The old answer: repeat one network

Researchers needed a network that could accept any number of words and carry earlier clues forward.

For years, one common answer was a recurrent neural network, or RNN.

An RNN runs the same small neural network repeatedly.

On each turn, that network receives two lists of numbers:

1. The current word, represented as numbers
2. The numbers carried forward from the previous turn

The network combines both lists using its learned weights. The result is a new list of numbers, which moves forward to the next turn.

The word changes on every turn. The carried numbers change too. The network and its learned weights stay the same.

Running the same network again for every word is what recurrent means.

Researchers needed a network that could accept any number of words and carry earlier clues forward.

For years, one common answer was a recurrent neural network, or RNN.

An RNN runs the same small neural network repeatedly.

On each turn, that network receives two lists of numbers:

1. The current word, represented as numbers
2. The numbers carried forward from the previous turn

The network combines both lists using its learned weights. The result is a new list of numbers, which moves forward to the next turn.

The word changes on every turn. The carried numbers change too. The network and its learned weights stay the same.

Running the same network again for every word is what recurrent means.