Two stages, not one

Building a large language model takes two stages.

Pre-training is the stage you know. Months of predicting the next word across an enormous pile of text. Effectively all of the cost, the time, and the knowledge come from here.

Post-training comes after. Beside pre-training it is tiny, a fraction of the time and expense.

The sizes are misleading. Pre-training on its own does not produce anything you would want to talk to. Post-training is what does, and it is the rest of this chapter.