Loading slide

What Attention Means

  1. 01What Attention Means
  2. 02What attention is for
  3. 03Memory card
  4. 04Which word matters right now?
  5. 05It's just a weighted blend
  6. 06Looking, advertising, offering
  7. 07Catch your breath
  8. 08Watch it lean
  9. 09Take a breath
  10. 10All at once, not one at a time
  11. 11More than one kind of relevance
  12. 12Reinforce your understanding
  13. 13Question: Computing relevance
  14. 14Question: Why parallel matters
  15. 15Question: The distance problem, gone
  16. 16Quiz: answer
  17. 17What to carry out of this chapter
  18. 18Want to go deeper?
1 / 17
BackNext

Module VII

Paying Attention

Chapter II

What Attention Means

In this chapter

  • Which word matters right nowthe question attention is built to answer
  • A weighted blendhow attention is just the weights you already know, aimed at words
  • Looking, advertising, offeringthe three roles that make the matching work
  • All at once, not one at a timewhy this fit the hardware so perfectly

Understanding a sentence is, in large part, knowing what to ignore.

Read any line and you do not weigh every word the same. A few words carry the meaning of the moment, and the rest are scaffolding. When you hit "it," your mind reaches back and grabs the one earlier word it stands for, while the dozen words in between barely register. You do this constantly, effortlessly, without ever calling it a skill.

The last chapter left us needing a machine that could do the same thing: let any word reach straight to any other word, weigh how much each one matters right now, and pull in just the ones that count. That machine has a name. It is called attention, and this chapter builds it from the ground up, using nothing you have not already met.

The surprise of the story is how much came from this one idea. Not a clever trick bolted onto the old approach. A different way of reading entirely, and the thing almost everything that followed was built on.

Citations