Begin with it.

Its stored list of numbers is the same in both trophy sentences. Yet big points toward trophy, while small points toward suitcase. The answer must come from the other tokens in the sentence.

Attention gives the model a way to choose among them. The query from the token being updated is compared with every available key. Those matches become weights. The weights decide how much of each value enters the token's new list of numbers.

With full-sentence attention, later tokens such as big or small can help update it. A model generating text cannot see those later tokens yet. When it reaches them, they can look back at it, trophy, and suitcase. Across repeated uses of attention, the useful clues can meet.

Every available token has a short route to the token being updated. Many matches can be calculated at the same time. Several attention heads can also learn different ways of finding useful relationships.

Attention now solves one job. It chooses where information should come from and moves that information. It does not yet explain how the model works on what arrives or how it keeps track of word order.

The next chapter adds those missing parts and builds the transformer.