Loading slide
The walk so far has crossed the output weights. Each one carries its effect, which turned out to be the signal its unit sent. Nothing has been fixed, and the machine still guesses $23.75.
The next crossing makes the numbers much larger, so it is worth knowing why before it happens. We are switching what we nudge.
On the last slide we nudged a weight and the guess moved $.50, to $24.25. Now we nudge the unit's output instead, and the guess moves $25, to $48.75. Same machine, same bottle, wildly different numbers, because +1 means something very different depending on what you add it to.
Adding 1 to a weight of 25 is a small proportional change. Adding 1 to a signal of .50 triples it.
From here on, the picture follows hidden 1 only, so the route stays readable. Keep in mind that the other three units are being handled at the same time, by the same steps, with their own numbers. Hidden 2 gets its own unit effect, its own incoming weights get theirs, and so on.
Press step into hidden 1. Now ask a different question. Not what the weight does, but what the unit does. If hidden 1 sent 1 more than it did, how much would the price move?
Work it through with the real numbers. Hidden 1 sent .50, so it contributed .50 x 25 = $12.50 to the guess. Send 1 more and it sends 1.50, contributing 1.50 x 25 = $37.50.
The guess would move from $23.75 to $48.75. The difference is $25.
Note the would. Nobody makes this change. Hidden 1 is not sending 1.50, and the guess is not $48.75. We imagined the change, read off how far the price shifted, and then put everything back. The machine still guesses $23.75.
That $25 is the answer. Be careful with it: it is not what the price becomes, and not what the unit is worth. It is how far the price shifts per unit of movement, a rate.
1 is a strange size to imagine moving something sitting at .50, and no real training step would ever move it that far. It is used here only because it makes the arithmetic easy to read. A rate is a rate at any size, so measuring per 1 costs nothing and keeps the numbers legible.
The starting point drops out, which is what makes this number useful. Going from .50 to 1.50 moves the price $25. Going from 2 to 3 would also move it $25. Every unit of output gets multiplied by 25 on its way to the price, so any +1 is worth $25, wherever it starts.
That is also why 25 turns up twice. The disc holds a weight whose value is 25. Hidden 1's effect on the price is also 25. They are different quantities, and they coincide because the unit's output is multiplied by exactly that weight on its way to the price.
That single multiplication is the whole trick, and it is worth naming what it did. We arrived holding a fact about the price. We crossed one weight. We now hold a fact about a hidden unit: moving hidden 1 by 1 is worth $25.
That is how a hidden unit gets a share of the blame without ever being told a correct answer. It was never handed a target. It was handed a rate, how much the final price responds when it moves.
One crossing remains. Hidden 1 is now worth $25 for every extra 1 it sends, and the four weights feeding it are the last stretch of wire.
Loading contents...