The running summary

Let's look closely at that scribbled summary, because everything in this chapter turns on what it is and what it isn't.

It has a name: the . Strip away the name and it is just a list of numbers, a , the same kind of thing we have been handling since words first became numbers. Say it is three hundred numbers long. That length never changes. Three hundred numbers before the first word, three hundred after the tenth, three hundred at the end.

Here is the move the network makes at every single word. Take the summary so far, take the new word, mix them together, and produce a fresh summary that now includes this word too. Then throw the old summary away and keep the new one. Read, mix, replace. Read, mix, replace. One word at a time, all the way down the sentence.

Picture passing a single notepad down a long line of people, one line of notes on it, no more. Each person may read what is written, add their own word, and rewrite the line to fit, but they may never make the notepad bigger. By the time it reaches the far end, the whole sentence is supposed to live in that one line of notes.

And you can already feel the squeeze coming. A fixed-length summary has only so much room. As more and more words pour in, something has to give. That something is the subject of the next slide.