One more adjustable number

We kept one part out while the hidden-weight calculation was new.

After a hidden unit adds up its weighted inputs, it adds one more adjustable number before the floor at zero decides what it sends. That number is called a bias. For hidden 1 the bias has been 0 all along:

.50 + 0 = .50

Every earlier slide was still correct. A bias of 0 changes nothing, which is why it could wait.

A weight changes how much one particular input matters. The cabernet weight only contributes when the cabernet input is present, and contributes nothing when it is absent. A bias is different. It is not attached to any input, so it is added in full for every bottle, whatever the tastes happen to be.

What is it for? The answer sits in the floor at zero, the rule from slide 18 that turns a unit's total into the signal it sends. That rule is what makes zero matter: a total above zero passes through at its own value, and anything at or below zero becomes 0, silence.

So the zero line is not just a number on a scale. It is the unit's speaking threshold, the amount of weighted evidence that counts as enough. And a threshold is only useful if you can put it in the right place.

The weights alone cannot set that threshold. Each weight is tied to one input, so its influence comes and goes with the bottle. Nothing in the sum shifts every bottle's total by the same fixed amount. That is exactly the job the bias does.

The illustration shows what that buys. It puts all four wines side by side and reads out what hidden 1 makes of each one. Its weighted totals are .50, .40, .40, .30. Take the apricotty cabernet as a check: cabernet and apricotty are present, so .20 + .20 = .40.

All four totals sit above the floor, so the unit speaks for every wine. The plummy cabernet gets the loudest signal, which is the right wine to favour, but the other three are barely quieter. A unit meant to report one wine type is currently saying something about all four, and every one of those signals feeds the price.

Press bias to -.40. Every total drops by the same .40, becoming .10, 0, 0, -.10. Only the plummy cabernet still clears the floor. The other three wines now leave the unit silent.

Same weights, one extra number, and the unit turned from a murmurer into a specialist. That is what a bias is for: it sets how much evidence a unit needs before it speaks. A negative bias raises the bar. A positive bias lowers it, letting the unit speak on weaker evidence.

Nobody picks these values by hand. A bias is one more adjustable number, so the backward walk hands it an effect exactly as it hands one to every weight.

In fact the machine does not need to treat it as a special case at all. Every other number in that sum is an input multiplied by a weight, and a bias can be written the same way: give the unit one imaginary extra input whose value is fixed at 1, and let the bias be its weight. Then 1 x -.40 contributes the same -.40 the bias contributes. That fixed 1 is not a taste, and it is not saying a taste is present. It is a placeholder that never varies, there so the bias has something to be a weight of.

Writing it that way costs nothing and buys the whole backward walk. Slide 26 measured a weight's effect by multiplying by the input it carries, and this weight carries an input of 1, so its effect is whatever arrives from the unit above, passed through unchanged. Gradient descent then nudges it, a little per bottle, along with everything else.