What made it work

AlexNet wasn't one breakthrough. It was several older ideas, combined and run at the right scale for the first time. Almost every piece is something we have already met.

The depth came from stacking layers of neurons, each one learning a slightly more abstract pattern than the layer before: raw pixels in, high-level concepts out. The fix that kept the training signal from fading on its way down all those layers was , the small fix from the first chapter. And GPU training, from the chapter you just read, made the whole thing feasible at all. Without it, training on 1.2 million images would have taken months.

Two more techniques helped the network learn the general idea instead of memorizing the exact training photos. Remember the danger from earlier: a network that memorizes its examples aces the practice set, then fails on anything it hasn't seen.

The first was . During training, the network randomly switches off some of its own neurons each round. Because no neuron can count on its neighbors always being there, the network can't rely on one fragile chain of cells to recognize a dog. It has to spread the knowledge around and build backups, and a network that isn't leaning on one brittle shortcut generalizes better.

The second was . Instead of showing each photo once, the network is shown many tweaked versions: flipped left to right, cropped tighter, shifted a little. A dog is still a dog whether it faces left or right, so these variations push the network to recognize the object rather than memorize one exact arrangement of pixels. From 1.2 million photos, it effectively sees many times more.

None of these ideas were brand new. The contribution was putting them together carefully, at scale, and showing that the combination worked dramatically better than the previous approach.

The result spoke for itself.