Each word has to wait

The running note creates another problem.

The RNN cannot process dog until the has produced a new hidden state. It cannot process ran until dog has produced the next one.

Each word waits for the word before it.

That matters during training. A computer may have thousands of workers ready, but the RNN cannot hand every word to them at once. Later words still need the notes produced by earlier steps.

Long sentences therefore create two costs. Earlier clues travel through a long path, and the work moves through that path one step at a time.