Learning without changing

Step back and notice something genuinely strange about what you just did. You taught the model a task by showing it examples, and it got better at that task, all without any training happening. Nothing inside the model changed.

Recall from the model module that learning, real learning, means adjusting the weights, the billions of numbers that get nudged during training. That process is enormous, slow, and over the moment training stops. The weights are frozen. So when a model picks up your format from three examples, those frozen weights are not moving a hair. The lesson lives entirely on the board, in the context window, and the instant you clear the window it is gone, as if it never happened.

This trick has a name: . And the name is almost a tease, because it looks like learning while breaking the one rule we said learning followed. The model generalises to new cases it was never given, the hallmark of having learned something, yet stores nothing.

How can frozen weights do that? The honest answer is that researchers do not fully agree. But the leading intuition connects straight back to pretraining. The model spent its training reading an ocean of text, and that text was full of little patterns followed by their continuations: lists, tables, question-and-answer pairs, "here are some examples, now do another." It got staggeringly good at one move: spot the pattern in front of me and extend it. In-context learning is that same muscle, flexed on a pattern it is seeing for the very first time, yours.

So this is not a separate magic power bolted on. It is the next-token predictor, doing the only thing it ever does, continuing the most plausible pattern, except the pattern is the task you just demonstrated. That is why a handful of examples can stand in for hours of fine-tuning, and why prompting is as powerful as it is.