Word2Vec and the arithmetic of meaning

In 2013, a team at Google published a paper introducing Word2Vec — a method for learning word embeddings from text.

The results were striking. Not just that similar words clustered together — that was expected. But that the geometry of the space encoded relationships.

The famous example: take the vector for "king." Subtract the vector for "man." Add the vector for "woman." The result lands close to the vector for "queen."

The model hadn't been told anything about royalty or gender. It had just seen a lot of text. Somehow, through exposure to how these words were used, it had arranged them so that the king-queen relationship was geometrically consistent with the man-woman relationship.

This was a demonstration that meaning — or at least something that behaves like meaning — could emerge from the statistics of language use. And it could be computed with arithmetic.

<!-- TODO: simple diagram showing the king - man + woman ≈ queen vector arithmetic — this is one of the most memorable visuals in all of AI education -->