Loading slide

The GPU Moment

  1. 01The GPU Moment
  2. 02One thing first
  3. 03Why training is slow
  4. 04What a GPU actually does
  5. 05The realization
  6. 06The scale this enabled
  7. 07Something unexpected
  8. 08Take a breath
  9. 09Data and compute
  10. 10Memory card
  11. 11Reinforce your understanding
  12. 12Question: Why GPUs and not faster CPUs?
  13. 13Question: Three ingredients
  14. 14Question: Why the field sped up
  15. 15Quiz: answer
  16. 16The hardware was ready. The data existed.
  17. 17Want to go deeper?
8 / 16
BackNext

Take a breath

That was a big idea, so let it settle before we move on.

You don't need to hold the details. Here is the whole thing in one breath: the right technology already existed, the graphics card built for video games. Once researchers pointed it at neural networks, training went from months to hours. And when they used that speed to build networks far bigger than before, the networks didn't just get a little better. They started doing things no one had built into them.

That surprise has a name, emergence, and it is the thread we will keep pulling on for the rest of this course. For now, that is all you need to carry forward.

One ingredient is still missing, though. Fast machines need something to learn from.

Citations