This chapter explains the most important machine in modern AI, so let us be very clear about the one job it does before we open it up. If you hold this, the rest will make sense.

Two chapters ago, in the embeddings module, we hit a wall. A word like "bank" means different things in "river bank" and "bank account," but a fixed position in the space of meaning can only put "bank" in one spot. We said the fix was to recompute each word's position from the words around it, every time, and that doing so needed a special machine we had not built yet.

Attention is that machine. That is its entire purpose: to take each word and adjust its meaning based on the other words in the sentence, so "bank" lands near "river" in one sentence and near "money" in another.

That is all attention is for. Everything in this chapter, the leaning, the blending, the want-ads and folders, is just how it pulls that off. So as the mechanics arrive, keep asking one question: how does this help a word take on colour from its neighbours? Because that, and only that, is what attention is solving.