Loading slide

Loading contents...

[██████░░░░░░░░░░░░][█████████░░░░░░░░░░░░░░░░░░░]6 / 18
<back>next

Your memory does this too

Everything you have just watched fail is already true of you.

Your memory of your grandmother is not filed anywhere. There is no cell in your head holding it, no address to visit. It is a pattern spread across an enormous number of connections, most of which are also participating in your memories of kitchens, of a particular armchair, of being small, of a hundred other people.

Three ordinary things follow from that, and you have probably noticed all of them.

Damage degrades rather than deletes. Injury or illness rarely removes specific memories the way deleting a file removes a file. Things get patchy, faint, harder to reach. That is what damage to a spread-out pattern looks like from the inside.

A fragment can pull back the whole. A smell drags an entire afternoon back, unbidden and complete. Nothing looked anything up. Part of a pattern was triggered, and the rest of the pattern came with it.

Recognition survives partial information. You recognise a face in bad light, at an angle, half hidden, thirty years older. No stored photograph would match. A pattern that is mostly present is enough.

And you have a version of that slider. Try to forget one specific fact about a person you know well, without losing anything else about them. You cannot, and not because you lack willpower. There is no row to remove.

One warning, because this comparison gets overclaimed constantly.

The similarity is about storage, and it stops there. A model is a fixed set of numbers that does not change until someone retrains it. Your brain is alive and chemical, physically rewiring itself while you read this, attached to a body that has spent years in the world. Neurons are not weights. Learning is not training.

So when you hear that these systems work like the human brain, the honest version is much narrower. Both hold information as a pattern spread across many connections rather than filed in one place. That one thing is true. Almost everything else people mean by the comparison is not.

The one thing they share has a name. Knowledge stored as a pattern across many units, where each unit takes part in carrying many different things, is a distributed representation. It is the idea this whole chapter has been circling, and it was worked out for brains before anyone built it into a machine.

# citations(2)↓
  1. [1]deeplearningbook.org
  2. [2]nature.com