Now move one step earlier

The output weight on the last slide changed the price directly. Now move one step earlier.

The cabernet weight entering hidden 1 is .20. Its effect must pass through the hidden unit and then through the output weight of 25 before reaching the price.

For this experiment, the output weight stays at 25 while only the incoming weight changes.

Press raise by .01. Watch three numbers change:

The weight changes from .20 to .21.
hidden 1 changes from .50 to .51.
The price changes from $23.75 to $24.00.

A .01 increase in this weight raised the price by $.25. Because the price was too low, increasing this weight helped.

Notice where that $.25 came from. The weight added .01 to what hidden 1 sends. Everything hidden 1 sends is then multiplied by its output weight of 25. So .01 x 25 = .25. The change did not reach the price directly. It had to cross the output weight to get there, and crossing it multiplied the change by 25.

The two experiments changed one weight at a time so the effects were easy to see. Training needs an effect for every weight in the network, and there is no time to test them one by one.

It does not have to. That multiplication by 25 is the clue. The same crossings that carry a change forwards can be walked in reverse to carry the error backwards.

Now move one step earlier

The output weight on the last slide changed the price directly. Now move one step earlier.

The cabernet weight entering hidden 1 is .20. Its effect must pass through the hidden unit and then through the output weight of 25 before reaching the price.

For this experiment, the output weight stays at 25 while only the incoming weight changes.

Press raise by .01. Watch three numbers change:

The weight changes from .20 to .21.
hidden 1 changes from .50 to .51.
The price changes from $23.75 to $24.00.

A .01 increase in this weight raised the price by $.25. Because the price was too low, increasing this weight helped.

The two experiments changed one weight at a time so the effects were easy to see. Training needs an effect for every weight in the network, and there is no time to test them one by one.

It does not have to. That multiplication by 25 is the clue. The same crossings that carry a change forwards can be walked in reverse to carry the error backwards.