The answer is wrong. What caused it?

The gap you were just working with has a name. The difference between the answer the network gave and the answer it should have given is the error.

Six connections were few enough to hold in your head, even without being able to single one out. Real networks are not that size. In a large one, thousands or millions of weights may have contributed to a single wrong answer, and the wrong answer says no more about them than it did about the six.

This is the . The mistake appears at the end, but its causes are spread through the network.

Training needs a way to trace the error backward through the calculation.