Backpropagation

In 1986, David Rumelhart, Geoffrey Hinton, and Ronald Williams published a solution to the credit assignment problem. The method was called backpropagation.

The idea: once you know the error at the output, you can work backwards through every layer to figure out how much each weight contributed to it. A small mathematical rule — one that had been around since the 1700s — lets you calculate, for any weight buried anywhere in the network, how much adjusting it would improve the final answer.

Think of it like blame flowing upstream. A mistake happened at the end. Backpropagation traces it back through every layer, all the way to the beginning, distributing responsibility to each weight in proportion to how much it contributed to the error.

Do this across thousands of examples. Adjust each weight slightly, every time. Over and over, the weights move in the direction that reduces mistakes.

That's backpropagation. No new mathematics was required. The insight was seeing how to apply an old rule to this specific problem — and doing it efficiently enough to be practical.