Loading slide

A Network That Learns

  1. 01A Network That Learns
  2. 02Stacking neurons
  3. 03Quick recall
  4. 04What stacking does
  5. 05Hidden layers
  6. 06The problem of credit
  7. 07Backpropagation
  8. 08Sharing the blame
  9. 09The search problem
  10. 10Fog on a hill
  11. 11Gradient descent
  12. 12Just to be clear
  13. 13What the hidden layer learns
  14. 14What the hidden layer discovers
  15. 15Did it actually learn?
  16. 16Generalization
  17. 17Limits in the 1990s
  18. 18Reinforce your understanding
  19. 19Question: Backpropagation
  20. 20Question: What the hidden layer learns
  21. 21Question: Memorizing vs. generalizing
  22. 22Quiz: answer
  23. 23The method existed. What was missing was scale.
  24. 24Want to go deeper?
12 / 23
BackNext

Just to be clear

A single neuron can only ask one question. That's not enough for real problems, which depend on combinations of things like the shape of an ear and the proportions of a face. No single rule can capture that.

So you stack neurons into layers. That gives the network more power, but it also creates a harder question: when the answer is wrong, which weights caused the mistake?

Don't worry if backpropagation and gradient descent felt like a lot. You do not need to remember the names.

Here is the idea worth keeping: when a network gets something wrong, there is a way to trace the mistake backward, give each weight its share of responsibility, and nudge the whole system toward a better answer.

The machine is not being handed the solution. It is correcting its way there, one mistake at a time.

Citations(2)↓
  1. 1. deeplearningbook.org
  2. 2. developers.google.com