Loading slide

Embeddings

  1. 01Embeddings
  2. 02Memory card
  3. 03Hold this one idea
  4. 04A space of meaning
  5. 05Don't panic about hundreds of directions
  6. 06Word2Vec and the arithmetic of meaning
  7. 07Let that land
  8. 08Memory card
  9. 09The same word, different meanings
  10. 10Reinforce your understanding
  11. 11Question: Geometry of meaning
  12. 12Question: What embeddings can't capture
  13. 13Quiz: answer
  14. 14What you now know
  15. 15Meaning became geometry
  16. 16Want to go deeper?
5 / 15
BackNext

That slide said a word lives in a space of hundreds of directions, and that phrase can stop you cold. Nobody can picture hundreds of directions. So let us defuse it, because the idea is much friendlier than it sounds.

You already think in many directions at once. You do it every time you describe a person. Imagine a few sliders: how tall they are, how old, how loud, how kind, how funny. Five sliders. Each person is just a setting on each one, a point in a five-direction space. Two people who are "near" each other on all five sliders are similar people. Nobody finds that mysterious, and nobody needs to see a five-dimensional picture to use it. You just track the sliders.

A word embedding is the exact same thing, only with a few hundred sliders instead of five. Each slider captures some shade of how the word behaves, and we do not even need to know what each one means. "Dog" is just a particular setting across all those sliders. "Puppy" has almost the same settings, so it sits nearby. "Turnip" has very different settings, so it sits far away.

So when we say "hundreds of dimensions," do not try to picture it. You cannot, and you do not have to. Just hold the friendly version: a word is a long list of slider settings, and near means similar. That is all "high-dimensional space" ever means here.

Citations