Scale and Emergence

Sometimes more is not just more.

There is a certain kind of change that is gradual until it isn't. A material gets compressed slowly and then, at a certain pressure, it snaps. A city grows incrementally and then, past some threshold, new kinds of infrastructure become possible. The rules change.

Something similar happened with large language models. As they got bigger, more parameters, more data, more compute, they didn't just get better at the things they were trained on. They started doing things no one had trained them to do. Capabilities appeared that hadn't been engineered or anticipated.

This is what emergence means: properties that arise from scale and interaction, not from any single component. Nobody put those capabilities there. They showed up.

That is not magic. But it is genuinely surprising, and it raises a question the field is still working through.