Loading slide

A Network That Learns

  1. 01A Network That Learns
  2. 02Stacking neurons
  3. 03Quick recall
  4. 04What stacking does
  5. 05Hidden layers
  6. 06The problem of credit
  7. 07Backpropagation
  8. 08Sharing the blame
  9. 09The search problem
  10. 10Fog on a hill
  11. 11Gradient descent
  12. 12Just to be clear
  13. 13What the hidden layer learns
  14. 14What the hidden layer discovers
  15. 15Did it actually learn?
  16. 16Generalization
  17. 17Limits in the 1990s
  18. 18Reinforce your understanding
  19. 19Question: Backpropagation
  20. 20Question: What the hidden layer learns
  21. 21Question: Memorizing vs. generalizing
  22. 22Quiz: answer
  23. 23The method existed. What was missing was scale.
  24. 24Want to go deeper?
13 / 23
BackNext

What the hidden layer learns

The network learns by fixing mistakes. But what exactly does it learn?

Before that: for any of this to work, every training example needs a correct answer attached to it.

The network makes a prediction, compares it to the right answer, and measures how far off it was. That gap is what drives the whole process.

If you're training a network to tell cats from dogs, someone had to go through thousands of photographs and label each one, "cat," "dog," "cat," "dog," before training could even begin.

The network never figures out the right answer on its own. It just learns to get closer to the answers it was given.

Citations(2)↓
  1. 1. deeplearningbook.org
  2. 2. developers.google.com