Loading slide

A Network That Learns

  1. 01A Network That Learns
  2. 02Stacking neurons
  3. 03Quick recall
  4. 04What stacking does
  5. 05Hidden layers
  6. 06The problem of credit
  7. 07Backpropagation
  8. 08Sharing the blame
  9. 09The search problem
  10. 10Fog on a hill
  11. 11Gradient descent
  12. 12Just to be clear
  13. 13What the hidden layer learns
  14. 14What the hidden layer discovers
  15. 15Did it actually learn?
  16. 16Generalization
  17. 17Limits in the 1990s
  18. 18Reinforce your understanding
  19. 19Question: Backpropagation
  20. 20Question: What the hidden layer learns
  21. 21Question: Memorizing vs. generalizing
  22. 22Quiz: answer
  23. 23The method existed. What was missing was scale.
  24. 24Want to go deeper?
4 / 23
BackNext

What stacking does

So what if you used more than one neuron?

The first neuron looks at the raw inputs and makes a simple call. Maybe it notices something about the shape of the ears. It doesn't know what a cat is. It just fires or doesn't based on what it sees.

The second neuron doesn't look at the raw inputs at all. It looks at what the first neuron decided. It takes that result, combines it with other results from other neurons in the first layer, and makes its own call. Maybe it's picking up on something about the overall shape of the face.

The third neuron looks at what the second found. And so on.

Each layer is asking a slightly more specific question than the one before, but no single neuron ever knows what a cat is. Each one just adds up its inputs, checks its threshold, and passes a signal forward.

Nobody writes the concept of "cat" down anywhere. It emerges from the layers, the same way the dog's decision to bark emerged from smell and sound and memory in the previous chapter. Once again, the understanding lives in the whole thing, not in any of the parts.

Citations(2)↓
  1. 1. deeplearningbook.org
  2. 2. doi.org
Citations(2)↓
  1. 1. deeplearningbook.org
  2. 2. doi.org