Loading slide

Embeddings

  1. 01Embeddings
  2. 02Memory card
  3. 03Hold this one idea
  4. 04A space of meaning
  5. 05Don't panic about hundreds of directions
  6. 06Word2Vec and the arithmetic of meaning
  7. 07Let that land
  8. 08Memory card
  9. 09The same word, different meanings
  10. 10Reinforce your understanding
  11. 11Question: Geometry of meaning
  12. 12Question: What embeddings can't capture
  13. 13Quiz: answer
  14. 14What you now know
  15. 15Meaning became geometry
  16. 16Want to go deeper?
3 / 15
BackNext

This chapter has one big idea, and everything else is decoration on it. So before we start, let us state it as plainly as it will ever be stated.

A word's meaning is its position among other words.

That is the whole thing. Not a definition stored in a box. Not a label. A place, sitting in a vast space, surrounded by neighbours. "Dog" means what it means because of who it sits near (puppy, bark, leash) and who it sits far from (turnip, Tuesday, gravity). Move it somewhere else and you would have changed what it means.

We spent the first chapter arguing our way to this idea, that meaning comes from the company a word keeps. This chapter takes that idea and makes it real and measurable: actual positions, actual distances, actual directions you can walk.

If at any point the chapter starts to feel slippery, come back to this one sentence and stand on it: meaning is position. Everything that follows is just exploring what that buys us.

Citations