Word2Vec and the arithmetic of meaning

In 2013, a team at Google wrote the recipe down and ran it at full scale. They called it Word2Vec, and at its heart is exactly the game from the first chapter: hide a word, guess it from its company, nudge, repeat, across billions of sentences. Out of nothing but that game came a position for every word: the map you just wandered.

Then came the part nobody expected. The space didn't just put similar words near each other. That was the goal. It had also turned relationships into directions.

Remember the square from the first chapter? King and queen on one row, man and woman below, royalty running one way and gender the other? Word2Vec built that square on its own. Take the vector for "king." Subtract "man." Add "woman." The result lands almost exactly on "queen." The step that carries man to woman, the "male to female" direction, is the same step that carries king to queen. Walk it yourself on this slide.

Nobody told the model about royalty or gender. It read text and played the guessing game, and a relationship between words became a piece of geometry you can do arithmetic with.