Loading slide
Loading contents...
The list of numbers carried from one word to the next is called the hidden state.
"State" means the RNN's current condition after the words it has received so far. "Hidden" means the numbers stay inside the network.
Suppose this hidden state contains three hundred numbers. The illustration shows six of them as dials, one dial for each number shown.
After dog, the numbers form one pattern. When ran arrives, the RNN combines that token with the old hidden state and calculates a new hidden state.
The numbers can all change. The list still contains three hundred numbers.