Why it took over everything

Within a few years, transformers became the leading architecture for many language tasks. Several advantages arrived together.

Attention gave tokens short paths to one another. Training calculations for many positions could run in parallel, using GPU hardware efficiently. Blocks could be stacked deeply. Self-supervised objectives created targets from text without a person labelling each example.

Together, those properties let researchers train larger models on larger datasets. Transformers are not best for every possible task, and attention itself becomes expensive as sequences grow, but the architecture scaled unusually well for language.

And it kept rewarding the obvious move. Build it bigger, feed it more text, give it more computing time, and its predictions measurably improved, in a smooth and fairly predictable way.

What that improvement turned into, task by task, was far less tidy. Some abilities crept up gradually. Others seemed to sit at nothing for a long while and then arrive all at once. Making sense of that is the whole of the next module.