You cannot see every step either

Think of a word being on the tip of your tongue. You know that you know it. You may remember its first sound, guess how long it is, or reject several suggestions. Still, the word will not come.

Then you give up. Half an hour later, while washing the dishes, it appears.

If you can picture the face of someone you know well, the image may appear without you consciously choosing the eyes, placing the nose, and assembling the expression.

Psychologists can study word retrieval, and neuroscientists can measure activity related to mental imagery. From the inside, you still do not watch the steps that recover a word or assemble a face.

This does not mean a brain and a language model work in the same way. A brain is part of a living body, shaped by a lifetime of experience. A language model is a trained machine.

The point is narrow. A process can produce a result without giving you a readable account of each step.

You cannot see every step either

Think of a word being on the tip of your tongue. You know that you know it. You may remember its first sound, guess how long it is, or reject several suggestions. Still, the word will not come.

Then you give up. Half an hour later, while washing the dishes, it appears.

If you can picture the face of someone you know well, the image may appear without you consciously choosing the eyes, placing the nose, and assembling the expression.

Psychologists can study word retrieval, and neuroscientists can measure activity related to mental imagery. From the inside, you still do not watch the steps that recover a word or assemble a face.

This does not mean a brain and a language model work in the same way. A brain is part of a living body, shaped by a lifetime of experience. A language model is a trained machine.

The point is narrow. A process can produce a result without giving you a readable account of each step.