The name for that loop

That loop has a name. It is called reinforcement learning.

Researchers gave each part of it a word. The rest of this chapter uses these words, so connect each one to the bicycle example first.

The learner is the . On the bicycle, that is you. An environment can contain other agents too.

Everything outside the agent that matters to the task is the environment. The bike, the road, the gravel, gravity. The agent affects it through actions, but does not control every part of what happens.

What the agent can perceive at one moment is an observation. How far the bike is leaning right now, how fast it is going, which way the wheel is turned. The full situation is called the state. An observation may reveal all of it or only part of it.

What it does about that is an action. Lean left. Turn the wheel. Pedal harder.

What comes back afterwards is the reward, a number saying how that went. Still upright, or on the ground.

Put those together and you have the loop from the previous slide: an agent receives an observation, takes an action, receives a reward, and adjusts how it will act next time. The bicycle gave us a picture of the loop. A reinforcement-learning system gives each part a precise mathematical role.

One of those words misleads. Reward sounds like praise, and it is not. It is a number, and it can be negative. A game score, a metre travelled without falling, a minus sign when the machine crashes. No approval is involved. Whoever set the problem up decided in advance what would be measured, and the number arrives by that rule whether anyone is watching or not.

And notice what has still not been mentioned. Nothing here has said anything about neural networks. Reinforcement learning names an arrangement for learning, not the machine that does it.

The name for that loop

That loop has a name. It is called reinforcement learning.

Researchers gave each part of it a word. The rest of this chapter uses these words, so connect each one to the bicycle example first.

The learner is the . On the bicycle, that is you. An environment can contain other agents too.

What it does about that is an action. Lean left. Turn the wheel. Pedal harder.

What comes back afterwards is the reward, a number saying how that went. Still upright, or on the ground.

And notice what has still not been mentioned. Nothing here has said anything about neural networks. Reinforcement learning names an arrangement for learning, not the machine that does it.