Show it what a good answer looks like

The first step is the obvious one. If you want the model to answer questions, show it people answering questions.

So people write them. Thousands of example exchanges, by hand. A question, then the reply it should have given. Another question, another reply. Each one written to be helpful, honest, and clear, which are words that will turn out to be doing a lot of work.

Those examples become a new training set. Tiny beside the internet the model learned from, a few tens of thousands of exchanges against hundreds of billions of words, but pointed at one specific thing.

Then the model trains on them exactly as it always has. Look at the input, guess what comes next, measure how wrong that was, nudge the weights. Same loop, same machine. Only the reading material has changed. Now the inputs are questions and what follows them is a good answer, so a good answer becomes the likely continuation.

That is the whole trick. The model was never taught what helpfulness is. It was shown enough text where the helpful thing came next that helpfulness became the pattern to continue.

Training a finished model further on a small set of examples, to point it at one particular job, is called fine-tuning. It is cheap next to pre-training, because the examples number in the thousands rather than the billions. The job it is being pointed at here is being an assistant.

It gets you most of the way. It does not get you all the way.