What can this token see?

Before attention scores any positions, the model decides which ones are available.

Some attention layers receive the whole completed sentence. When such a layer updates it, both earlier and later tokens are available. big can help connect it to trophy.

A model that generates text follows a stricter rule. At it, every later position is hidden, as if a sheet of paper covered the rest of the sentence. Attention can score trophy, suitcase, and the other earlier tokens, but not big.

This is called causal attention. Each position can use itself and anything before it, never anything after it.

The relationship is not lost. When the model later reaches big, that position can look back at it, trophy, and suitcase. What changes is the direction in which the information travels.

Why hide the future? During next-token training, seeing later tokens would reveal the answer the model is supposed to predict.

Attention scores only the positions available under the layer's rule.