Loading slide
When researchers later peered inside trained vision networks like this one, they found something strikingly orderly: a ladder, built layer by layer, simple pieces at the bottom and whole objects at the top. This is where the abstract phrase "learned features" becomes something you can almost picture.
It is worth seeing why the network settled on a ladder rather than anything else. To make sense of an image, the most useful thing the first layers can do is spot the simplest pieces a picture is made of, then hand those up to be combined into something richer. Simple to complex, each layer standing on the one beneath it, is just the cheapest path from a grid of pixels to a name. Nobody designed it that way. The pressure to be less wrong drifted the network into it.
The earliest layers had taught themselves to spot the simplest things: edges. A line here, a sharp change from light to dark there. Tiny fragments, nothing more.
The next layers combined those edges into textures and small shapes. A corner. A curve. A patch of fur-like repetition. Still meaningless on their own, but a step up.
Deeper still, those shapes assembled into recognizable parts. Something that responds to wheels. Something that responds to eyes. Something that responds to the rounded top of a teapot.
And near the end, those parts came together into whole objects. A car. A cat. A face.
Remember the face example from the first chapter, edges into shapes into faces into a person? That was the prediction. AlexNet is where we got to open the box and see it had actually happened. That is what "learning its own features" really means.