Loading slide

Embeddings

  1. 01Embeddings
  2. 02Memory card
  3. 03Hold this one idea
  4. 04A space of meaning
  5. 05Don't panic about hundreds of directions
  6. 06Word2Vec and the arithmetic of meaning
  7. 07Let that land
  8. 08Memory card
  9. 09The same word, different meanings
  10. 10Reinforce your understanding
  11. 11Question: Geometry of meaning
  12. 12Question: What embeddings can't capture
  13. 13Quiz: answer
  14. 14What you now know
  15. 15Meaning became geometry
  16. 16Want to go deeper?
1 / 15
BackNext

Module VI

The Meaning of Words

Chapter III

Embeddings

In this chapter

  • A space of meaningwhy similar words end up near each other
  • Word2Vechow a model learns the geometry of meaning from raw text
  • The same word, different meaningshow context changes where a word lands
  • What embeddings can and can't capture about language

Meaning is not only in words. It is also in the company they keep.

We have the pieces now. The first chapter said a word needs a position, not a name tag. The second turned text into tokens. This chapter is the payoff: how those tokens get their positions, and what the resulting map of language looks like.

There is a satisfying echo here. Back in the deep-learning module, AlexNet taught itself a ladder of visual features, edges into shapes into objects, that no human placed there. Embeddings are the same trick aimed at language. Nobody decides where "dog" sits or that it belongs near "puppy." The model discovers the whole arrangement on its own, from patterns of use. The geometry that emerges carries more structure than anyone expected: relationships between words turn into relationships between positions, and something that looks almost like meaning becomes visible in the shape of the space.

It is not meaning in the way you experience meaning. But it is not nothing either.

Citations