Loading slide
That was a lot. If the last few pages felt dense, that is not you missing something. This is genuinely some of the hardest machinery in the whole story, and the people who first worked it out spent years on it. So let us slow right down, drop every technical word, and say what actually happened in plain language.
A deep network learns by being corrected. It makes a guess, sees how wrong it was, and sends a correction backwards through its layers so each one can adjust. You met this in the last module: guess, measure, nudge.
Remember the whisper we talked about earlier, the message passed down a long line of people? That is the network, layer by layer, with the correction as the whisper. The trouble was that each person repeated it a touch more quietly than they heard it. So by the far end of the line, the front of the network, the message had faded to silence. Those front layers never really heard what to fix. And since they are the ones that learn the most basic things, the whole network stayed stuck.
The fix was almost silly in how small it was. Instead of everyone softening the whisper, each person now does one of two things: either pass it on at full voice, or stay quiet and say nothing at all. No more in-between, no more gentle fading.
You might wonder: if some people stay silent, doesn't the message die there? It would, through that one person. But a layer is not one person, it is a whole crowd of them side by side. At any moment plenty of them are passing it on at full voice, so the message always has a clear, unbroken path through the ones who are speaking. It reaches the front as loud as it started, and deep networks finally learn.
That is the entire chapter. You do not need to remember any of the names. Hold this one picture: the correction was fading as it passed down the line, and one small change kept a full-voice path open all the way to the front.