That was a lot of strange ideas in a row, so let us set them down and hold the shape, not the details.

The model's knowledge is real, but it is a strange kind of knowing. It is smeared across billions of numbers with no index, so the model cannot look anything up or tell you where a fact lives. It is fenced by an edge the model cannot feel, so it does not know what it does not know. And it is frozen at the moment training stopped, so it has no sense of anything newer.

Put those together and one thing falls out, the thing this whole chapter is really about: the model can be confidently, fluently wrong, and it has no inner alarm to warn you, because nothing inside it is keeping track of what is solid and what is invented.

That is not a flaw to be patched away. It is the shape of what these models are. And knowing the shape is exactly what lets you use one well: lean on it where it is strong, check it where it is thin.