Loading slide
There is a gap between what GPT-3 could do and the assistant you actually talk to, and it is worth being clear about.
Straight out of training, a language model has exactly one instinct: continue the text. That is all the loop ever rewarded. So if you typed a question into the raw model, it would not necessarily answer it. It might continue your question with three more questions, because that is what a list of questions looks like in its training data. Ask it for advice and it might write what looks like a forum thread arguing with itself. It is not being difficult. It is doing the only thing it was trained to do: produce the most likely continuation.
This is the base model, the same idea we met back in the learning module: a model trained on a huge general pile of data, capable but unaimed. Enormously knowledgeable, and pointed at nothing in particular.
Turning that into something that answers you, helpfully and on purpose, takes a second, much smaller phase of training laid on top. That phase is where a raw text-continuer becomes an assistant, and it is the next two slides. It is also, as we will see, where one of the model's most talked-about flaws gets built into its behaviour.