More weight, more influence

Attention needs exactly that volume control, one per token.

Imagine that every token brings a pot of paint. Attention chooses how much paint to take from each pot.

When attention makes a new mix for it in the sentence ending in big, it may take more from trophy and big. It takes less from tokens that do not help identify what it refers to.

In the sentence ending in small, the mix for it should change. Attention should now take more from suitcase and small.

The model is not mixing paint. Each token supplies a list of numbers. A larger weight lets more of that list enter the mix. A smaller weight lets less enter.

The new mix becomes the next list of numbers for it. It now carries information from the tokens that mattered most.

But how does the model choose those weights?