Two earlier problems now meet.

A stored embedding gives bank the same starting point in river bank and bank account. The surrounding tokens must adjust that starting point so it fits the sentence.

An RNN offered one way to carry earlier information forward. But every clue had to fit inside one running summary and survive each update along the way.

A direct connection removes that long chain. But direct access creates a choice. If many tokens are available, which ones should contribute most?

Attention answers that question. It lets the relevant tokens count more when the model builds a new, context-sensitive representation. For bank in a sentence about a river, tokens such as river and shore can contribute more. In a sentence about money, account and deposit can contribute more.

Deciding which tokens those are is the whole problem.