Loading slide
Let's pause and make sure the one idea so far is solid, because the rest of the chapter leans on it.
Forget the names. RNN, hidden state, LSTM, you do not need to carry any of them. Here is the whole thing in plain language.
The old way of reading language was to go word by word, left to right, keeping a single short summary of the story so far. Read a word, update the summary, move on. The catch was that the summary could never grow. So in a long passage, the early parts got crowded out and forgotten, like a notepad with room for only one line that keeps getting written over.
That is the picture to hold. A reader with a good memory for the last few words and a fading grip on everything before that.
Now, there is a second, sharper version of this problem, and it is the one that really mattered. It is not just that early words fade. It is that two words far apart in a sentence have a hard time reaching each other at all. That is where we go next.