Distributed knowledge

A neural network stores knowledge everywhere and nowhere.

A database has rows. You can find the record that says "Paris is the capital of France" and point to it. A neural network has nothing like that. The knowledge is distributed across millions or billions of weights — each one contributing a tiny amount to many different outputs, none of them meaningful in isolation.

This is why the model can generalize. It hasn't memorized a list of facts; it has absorbed a pattern that extends to cases it's never seen.

It's also why the model can be confidently wrong. The same distributed pattern that produces correct answers most of the time will occasionally produce something that sounds just as confident and is completely wrong. There's no flag inside the network that knows the difference.