Train once, adapt anywhere

The transformer came with a way of working that has shaped AI ever since, and you have already met both halves of it. It comes in two stages with very different costs.

First, the big, expensive, general stage: pretraining. Take a transformer and a large selected collection of text, then run a prediction task over and over. For a causal language model, the target is the next token.

This is the costly part. To improve its predictions, the model learns patterns in spelling, grammar, style, facts repeated in the data, and ways that explanations and arguments are written. What it learns is uneven and can be wrong.

What you get is a . Its direct is text prediction, not being a safe and helpful assistant.

Then a more targeted stage: post-training or fine-tuning. Train the base model further on selected examples, , or task data. This can improve instruction following or adapt behaviour for a .

This later stage is usually smaller than pretraining, but it can still be technically demanding. It shapes how the model uses its learned patterns. It does not guarantee correct knowledge or safe behaviour.