Loading slide

What Attention Means

  1. 01What Attention Means
  2. 02What attention is for
  3. 03Memory card
  4. 04Which word matters right now?
  5. 05It's just a weighted blend
  6. 06Looking, advertising, offering
  7. 07Catch your breath
  8. 08Watch it lean
  9. 09Take a breath
  10. 10All at once, not one at a time
  11. 11More than one kind of relevance
  12. 12Reinforce your understanding
  13. 13Question: Computing relevance
  14. 14Question: Why parallel matters
  15. 15Question: The distance problem, gone
  16. 16Quiz: answer
  17. 17What to carry out of this chapter
  18. 18Want to go deeper?
8 / 17
BackNext

Watch it lean

Here is the idea you can actually play with. Below is the trophy-and-suitcase sentence. The word "it" is doing the looking, and the bars show how much it leans on each other word: how strong each match was, how big a pour each word makes into the blend.

Flip the last word between "big" and "small" and watch the weights move. Nothing about "it" changes. But change that one final word and the heavy bar jumps from "trophy" to "suitcase," because the want-ad "it" holds up now matches a different chest-card best. That jump is the machine doing what you did in your head a few slides ago, made visible.

Notice how little weight lands on "the," "in," "because." They are scaffolding, and attention learns to all but ignore them. The lean goes where the meaning is.

Citations(2)↓
  1. 1. arxiv.org
  2. 2. research.google
Citations(2)↓
  1. 1. arxiv.org
  2. 2. research.google