The wrong question

The Perceptron was one neuron. A single unit taking inputs, multiplying by weights, making one decision.

Connecting multiple neurons together was not a new idea. You take one neuron's output and feed it as input into the next. Then another. Layer after layer, each one seeing a slightly transformed version of the problem. In theory, that kind of network could handle the combinations a single neuron couldn't.

The problem was nobody knew how to train one.

With one neuron, training is simple: it gets something wrong, you adjust its weights.

But a deeper network has hundreds of neurons across many layers, each with their own weights. When the whole network gets something wrong, which weights do you change? The error shows up at the end, but the cause could be anywhere inside. Nobody had figured out how to untangle that.

So the field asked the wrong question. They asked: can the Perceptron be fixed? No. So they gave up on the whole idea.

The right question was: can we figure out how to train a deeper network? Nobody had proved that was impossible. They had just stopped trying.