Loading slide

The GPU Moment

  1. 01The GPU Moment
  2. 02One thing first
  3. 03Why training is slow
  4. 04What a GPU actually does
  5. 05The realization
  6. 06The scale this enabled
  7. 07Something unexpected
  8. 08Take a breath
  9. 09Data and compute
  10. 10Memory card
  11. 11Reinforce your understanding
  12. 12Question: Why GPUs and not faster CPUs?
  13. 13Question: Three ingredients
  14. 14Question: Why the field sped up
  15. 15Quiz: answer
  16. 16The hardware was ready. The data existed.
  17. 17Want to go deeper?
BackNext chapter

By 2010, the ingredients had finally come together.

The algorithm, backpropagation through deep networks, had been known since 1986. The fixes for the vanishing gradient existed. The data, in the form of ImageNet, was assembled and ready. GPU training made it practical to use all of it in a reasonable amount of time.

None of this had been true before. Together, those changes made a new kind of AI possible.

In 2012, a team from Toronto entered a deep neural network into the ImageNet competition. The next chapter is what happened.

Citations