Reinforcement learning without a network

Here is a reinforcement learner with no neural network anywhere in it.

It is learning to escape a few rooms. Its entire memory is a short list. For each room, it can record which direction has tended to help. At the start, that list is blank. Every direction looks equally unpromising.

So it tries. One attempt runs into a dead end. Another doubles back. Another stops without finding anything useful. The failures do not reveal the route, but they are part of how the learner searches for one.

Eventually, an attempt reaches the exit. Now there is a good outcome to connect to the choices that led there. Across repeated updates, the estimated value of reaching the exit can spread back toward earlier rooms. The room before the exit becomes promising, then the room before that.

On the next attempt, the learner is not starting from nothing. It still may try other directions, but now it has a reason to prefer the route that worked.

Different reinforcement-learning methods pass that information backwards in different ways. The common loop is: act, see the consequence, then let that consequence change what happens next. A neural network is only one possible way to hold what was learned.