Loading slide

What the Weights Contain

  1. 01What the Weights Contain
  2. 02Memory card
  3. 03Billions of numbers
  4. 04A strange kind of knowing
  5. 05The edge it cannot feel
  6. 06Frozen in time
  7. 07Why the wrongness is built in
  8. 08Take a breath
  9. 09Reinforce your understanding
  10. 10Question: Why hallucination is structural
  11. 11Question: Earned vs. assumed trust
  12. 12Quiz: answer
  13. 13Knowledge without a knower
  14. 14Want to go deeper?
4 / 13
BackNext

A strange kind of knowing

A neural network can know something and be completely unable to explain it.

Picture a face-recognition model. It can pick out a person across different lighting, angles, and ages, sometimes more reliably than a human can. But it cannot tell you what it looked for. There is no checklist it consulted, no chain of reasons you could read back. The knowledge lives in the geometry of billions of numbers, and that geometry does not translate into anything a person can inspect.

This is the sharp break from the old AI we met in the long-winter module. Back then, an expert system held rules a human had written down: if the ears are this shape and the nose this proportion, then it is probably a cat. Those rules were readable. You could argue with them, and you could fix the wrong one. A neural network has nothing like that. It is a black box, not because anyone hid the inside, but because distributed, learned knowledge simply has no readable inside. The understanding is in the whole, not in any piece you can point at.

Most of the time this does not bite. When the model is right, you do not care that you cannot see why. The trouble comes when it is wrong. You usually cannot find the mistake, because there is no single place the mistake lives. The error is spread across billions of weights, none of which can be opened up and questioned. That gap between what the model can do and what we can read out of it is not a flaw waiting for a patch. It is the price of this kind of learning.

Citations(2)↓
  1. 1. distill.pub
  2. 2. distill.pub
Citations(2)↓
  1. 1. distill.pub
  2. 2. distill.pub