Loading slide
We met hallucination back in the model module, where we saw why it happens. Here we care about what it means for you, sitting at the keyboard, trying to decide whether to believe what came back. So a quick recap, then the consequence.
The recap, in one breath: the model writes the most plausible-sounding continuation, and "plausible-sounding" is not the same as "true." Text shaped like a citation is exactly what plausibly follows a request for sources, whether or not the cited paper exists. There is no separate step where the model checks its answer against reality, because it has no reality to check against, only patterns from text. So, as the memory card just said, producing a false fact and producing a true one feel identical from the inside. Both are simply high-probability completions.
Now the part that matters for using it. Because both feel identical from the inside, they come out identical too: same fluency, same confidence, same steady tone. The model has no inner wobble it shows you when it is guessing. There is no nervous catch in its voice, no "I think," no slowing down. The made-up citation arrives in precisely the font of the real one.
Sit with that, because it overturns a lifelong habit. With people, confidence carries real information. Someone who is unsure usually sounds unsure: they hedge, they pause, they say "I might be wrong." We have spent our whole lives reading those signals. A language model strips them out. Its confidence is not a readout of how solid the answer is; it is just its default style, applied evenly to truth and fiction alike.
This does not go away. It can be reduced, with retrieval tools that paste real documents onto the board, with careful prompting, with better training, but the pull toward plausible-over-true is part of what the machine is, not a bug waiting on a patch. Which means the job of telling solid from shaky cannot be handed back to the model. It falls to you. The rest of this chapter is about doing that job well.