Loading slide

A Network That Learns

  1. 01A Network That Learns
  2. 02Stacking neurons
  3. 03Quick recall
  4. 04What stacking does
  5. 05Hidden layers
  6. 06The problem of credit
  7. 07Backpropagation
  8. 08Sharing the blame
  9. 09The search problem
  10. 10Fog on a hill
  11. 11Gradient descent
  12. 12Just to be clear
  13. 13What the hidden layer learns
  14. 14What the hidden layer discovers
  15. 15Did it actually learn?
  16. 16Generalization
  17. 17Limits in the 1990s
  18. 18Reinforce your understanding
  19. 19Question: Backpropagation
  20. 20Question: What the hidden layer learns
  21. 21Question: Memorizing vs. generalizing
  22. 22Quiz: answer
  23. 23The method existed. What was missing was scale.
  24. 24Want to go deeper?
1 / 23
BackNext

Module IV

Learning from Data

Chapter II

A Network That Learns

In this chapter

  • Stacking neuronswhat happens when you chain layers together
  • The problem of credithow do you know which weights were wrong?
  • Backpropagationthe method that made deep networks trainable
  • Gradient descenthow the network finds its way to better answers
  • Generalizationwhy a trained network works on examples it's never seen

A single rule can only do so much.

It can sort one thing from another if the world cooperates, if the difference is clean, if the boundary is simple. Most real problems are not like that. The world is messier than any single rule can capture.

But what if rules could be stacked? What if the output of one fed into another, and another, each layer noticing something slightly more abstract than the one before? Not one line drawn through a problem, but many, overlapping and combining until something more like understanding emerged from the arrangement.

The hard part was not imagining the architecture. The hard part was figuring out how to teach it. How does a system of connected pieces know which part of itself was responsible when something goes wrong?

That problem had an answer. Finding it changed the field.

Citations