Loading slide
Now let the machine make one complete pass through the four wines.
The first bottle is the plummy cabernet. Its middle unit reports 1, its weight is 25, and the machine guesses 25 dollars. The bottle actually costs 40, so the error is:
40 - 25 = +15
The learning rate is 0.1. Here the active middle unit reports 1, so the correction moves one tenth of the error. The rule from the last chapter now has all three numbers it needs:
weight change = 15 x 1 x 0.1 = +1.5
Press correct it. The plummy cabernet weight moves from 25 to 26.5. Its miss shrinks from 15 dollars to 13.50, so the total error falls from 60 dollars to 58.50.
The machine then brings in the apricotty cabernet. This one costs 10 dollars, so its error is negative:
10 - 25 = -15
Its weight moves down by 1.5, from 25 to 23.5.
Keep going until all four bottles have passed through. The two expensive wines move up to 26.5. The two cheap wines move down to 23.5. One pass brings the total error from 60 dollars to 54.
Now keep going. Run one epoch at a time, or run 10 at once to watch the error fall faster.
After 55 epochs, the total error is just 18 cents. Each weight is within 5 cents of the correct price. At 90 or so, the total error is 0. The weights are exactly right: 40, 10, 10, and 40.
Maybe you noticed that we could simply set the four weights to 40, 10, 10, and 40. We could.
In this hand-built machine, each bottle activates a different output weight. With a learning rate of 1, the expensive bottle would move its weight from 25 to 40 in one correction:
25 + (40 - 25) = 40
The cheap bottle would move its weight from 25 to 10:
25 + (10 - 25) = 10
It would still take four corrections, one for each bottle. But after one pass through the four bottles, all four weights would be correct.
We keep the learning rate at 0.1 to practise gradual corrections. In real networks, many examples usually share the same weights. Moving a weight by the entire error can improve one example while damaging others, as we saw in Chapter 1.
Loading contents...