Embeddings

Meaning is not only in words. It is also in the company they keep.

If you want to represent a word to a machine in a way that carries something of its meaning, a label is not enough. What you need is a position: a location in a space where words that mean similar things end up nearby each other, and words with different meanings end up far apart.

The surprising thing is that this can be learned. Feed a model enough text and let it figure out, from patterns of use, which words tend to appear in similar contexts. The geometry that emerges carries more structure than anyone expected. Relationships between words become relationships between positions. Something that looks almost like meaning becomes visible in the shape of the space.

It is not meaning in the way you experience meaning. But it is not nothing either.