Backpropagation

In 1986, , , and published a solution. The method was called backpropagation.

Think of the whisper game. A message passes down a line of people, each one distorting it a little. By the end, the message is wrong. Now play it in reverse: the last person whispers the correction back up the line. Each person hears what they got wrong and by how much.

A network does the same thing. It makes a prediction, gets it wrong, then traces the error backwards through every layer. The mistake shows up at the very end, in the final answer. But the weights that caused it are scattered all through the network, including ones buried many layers back. Backpropagation is the procedure that carries the error back from the output, layer by layer, until it reaches every last one of them.

That backward journey matters. The error does not jump straight to each weight. It retraces, in reverse, the exact path the signal took on the way forward, passing back through the same connections that produced the answer. That is the part of the story we want to slow down and look at properly.