The repeated work

Processing tokens is not just moving text from one place to another.

For every token, the model performs layers of numerical calculation. Earlier tokens shape how later tokens are understood.

So the first 1,000 tokens cannot simply be ignored.

But their earlier calculations may not need to be rebuilt.