Why it felt sudden

The 2010s held more than one bet about where capable AI would come from.

became known for machines that learned by doing. Play a game, act in a world, receive feedback, improve. Its bet was that real capability needs contact with an environment.

OpenAI worked on that too, on games and robotics. But its GPT work placed a growing bet somewhere else. Predict the next token across enough text, with a large enough transformer, and useful structure about the world might appear inside the weights on its own.

argued that predicting text well is not a shallow trick. To guess the next word reliably across everything people have written, a system has to compress a great deal about what the writing is describing.

put more weight on grounding, the idea that understanding requires perceiving and acting in a world rather than only reading about one.

These were not sealed camps. Papers were shared, methods travelled, people moved between laboratories, and DeepMind built large language models of its own. The disagreement was about emphasis.

Hassabis later said GPT-3 surprised him. He had expected a text-only system to stay limited by its lack of grounding. Scale showed that text carried more reusable structure than he had expected, and left the grounding question open rather than settled.