Order is not decoration

There is a hole in the tower we just built.

Attention treats a sentence as a heap. It compares every word with every other word, weighing how well each matches, with no built-in sense of which came first. That was the point. It let any word reach any other in one hop, with nothing waiting on anything.

But The dog bit the man and The man bit the dog are made of the same words. Pour them into an unordered heap and they are the same heap, yet they describe opposite events, and only one of them is a normal afternoon.

Word order is not a decoration on top of the words. It carries who did what to whom. A machine that cannot see order cannot tell a victim from a culprit, and no amount of clever matching will recover it, because the information was thrown away before the matching started.

The old single-file reading got order for free and gave up speed for it. The transformer took the speed. So it owes the sentence its order back, by some other means.