What the task demands

You already have the sentence that settles it. The trophy and the suitcase have been following you since attention was introduced, and now they can be put to a different use.

The trophy did not fit in the suitcase because it was too small. The suitcase. Change the last word to big and it becomes the trophy instead, the rest of the sentence untouched.

You know why that happens inside a transformer. The question here is a different one. What does that sentence demand of anything trying to predict its last word?

To get that last word right, a machine has to know which object it refers to. To know that, it has to know that trophies go inside suitcases rather than the other way round, and that a thing fails to fit either because the container is too small or the contents too large.

That is not a fact about English. It is a fact about objects, containers, and physical space.

Now go back to counting. A counter looks at the last word or two and picks whatever followed most often. There is no arrangement of tallies that gets this right, not with more text, not with longer phrases. The information needed is not in the surface pattern at all.

So it is the same task, and the two systems are not on a spectrum. Doing this job well enough forces something into existence that counting never needed and could never produce.

So yes, it is autocomplete, and that turns out to be a much stranger fact than it sounds. The task never changed. Getting good enough at it required a machine that represents meaning as position, weighs every word against every other, and works over what it gathers layer after layer.

The dismissal is right about the task. It is wrong about what the task can demand.