What hallucination actually is

"Hallucination" is the word the field uses for when a language model confidently produces false information. Invented citations. Made-up statistics. Plausible-sounding facts that aren't true.

The word implies something visual and distorted — like a person seeing things that aren't there. The reality is more mundane and more structural.

The model generates text by predicting what would most plausibly follow from its prompt and context. "Most plausibly" is determined by patterns learned from training data. Text that looks like a citation is what plausibly follows a request for sources. It doesn't matter whether the cited paper exists.

There's no separate verification step. The model doesn't check its outputs against reality before producing them, because it has no access to reality — only to patterns learned from text. Generating a false fact and generating a true one feel identical from the inside. Both are high-probability completions.

This is not going away. It can be reduced with better training, with retrieval tools that ground outputs in real documents, with careful prompting — but the structural tendency toward plausible-sounding rather than true is intrinsic to how these models work.