How vast this gets

A neural network is still the same idea at every size: simple units, connected, each connection carrying a weight.

Press "scale up" and the drawing keeps growing. By the last step the wires have stopped being wires. They are a grey field. Nobody could count them by hand.

On a computer, "step inside" opens a network you can fly through. It sorts small pictures into four kinds, and it is built from about eleven hundred units joined by ninety-five thousand weights. Flying the length of it takes a while.

Now imagine that network two million times bigger.

That is roughly , with its 175 billion adjustable numbers. Models built after it are larger still. The published counts climb into the trillions, and the largest figures usually describe designs where only a fraction of those numbers is used for any single word.

Two million times is not a number anyone pictures. If that flythrough were a single page, two million pages would be a stack about two hundred metres tall, taller than most buildings you have stood next to.

That gap is not a different kind of machine. It is the same pattern, grown until no one can hold the whole thing in their head.

Why build them that big? Because for a long stretch, size kept paying. Researchers measured the relationship and found it was regular enough to plan around: give a model more numbers, more text, and more computing time, and its error fell along a predictable curve. That finding is called a scaling law.

The curve was useful because it let you look ahead. A team could train a series of small models, watch where the curve was heading, and estimate how well a much larger one would do before committing to it. A training run can cost millions and take months, so knowing roughly what you will get before you start is the difference between a gamble and a plan.

Two things complicate the simple version. Size alone was never the recipe, since a model given far more numbers than text does worse than a smaller one trained on more, so the ingredients have to grow together. And the easy gains from sheer size have narrowed, which is why later work also grew the effort a model spends thinking about each question, not just the number of weights it holds.

Scale is what makes the next problem sharp. When the final answer is wrong, the mistake has passed through so many weights that it is no longer obvious which ones should change.

How vast this gets

A neural network is still the same idea at every size: simple units, connected, each connection carrying a weight.

Press "scale up" and the drawing keeps growing. By the last step the wires have stopped being wires. They are a grey field. Nobody could count them by hand.

Now imagine that network two million times bigger.

That gap is not a different kind of machine. It is the same pattern, grown until no one can hold the whole thing in their head.

Scale is what makes the next problem sharp. When the final answer is wrong, the mistake has passed through so many weights that it is no longer obvious which ones should change.