Talk to a base model

What pre-training leaves behind is called a base model.

Say hello to one and it does not say hello back. It carries on, in fluent English, as somebody else. A woman from Leeds with a question about her sister's wedding, perhaps, several sentences of her, arriving from nowhere.

It has not misunderstood, refused, or broken. The sentences are perfectly good sentences. They are simply not a reply.

The model asked itself the only question it has ever been asked. Given this text, what usually comes next?

And Hello, who are you? is not, across most of the internet, a line that gets answered. It is a line near the top of forum posts, introductions, and chat logs. What usually comes next is somebody introducing themselves and getting to their point. So that is what the model wrote. It invented a person because the shape of the text needed one.

Put the same words to a post-trained model and it tells you what it is and offers to help.

The knowledge underneath is the same. Post-training is what changed.