The question that breaks it

Here is the question. Take "dog" and "puppy," two words that obviously belong together, and ask the network's only question about them: how close are these two rows of switches?

Let's actually compare them. "Dog" has its single 1 at slot 2,742, and 0 everywhere else. "Puppy" has its single 1 at slot 6,015, and 0 everywhere else. Lay one row over the other and look for anything they share. The 1s are in different slots, so they don't share that. Everywhere else, both are simply 0, 0, 0. There is no slot where both light up. By this representation, "dog" and "puppy" have nothing in common.

Now run the exact same comparison on "dog" and "skyscraper," two words with nothing to do with each other. "Dog" lit at 2,742, "skyscraper" lit at 8,390, both 0 everywhere else. Again, no shared switch, nothing in common.

Sit with what just happened. Dog-and-puppy came out exactly the same as dog-and-skyscraper: nothing in common, in both cases. There is no measurement you could run on these rows that would call "dog" any nearer to "puppy" than to a skyscraper. Every word is the same distance, the maximum distance, from every other word. One-hot makes everything equally far from everything else.

And now connect it back to how a network actually learns. Learning ran on spreading a lesson from one word to its near neighbours. But here no word has a near neighbour; they're all infinitely far apart in exactly the same way. So when the network learns that "barked" follows "dog," there is no nearby word for that lesson to spread to. Nothing catches it. Every word is an island, and the network has to learn each one from scratch, as if "puppy" had no relationship to "dog" whatsoever.

So look at the two failures side by side. The single-number street lied about closeness, putting strangers next door. One-hot tells no lies, but only by refusing to say anything: it erases closeness completely. A network needs closeness to learn, and we've handed it either a false map or a blank one. Neither works.

But now we can finally name what we actually want, because it sits right between those two failures: numbers built so that "dog" genuinely lands nearer "puppy" than "skyscraper," not by accident, but on purpose, with closeness that tells the truth. Finding numbers like that is the whole rest of this module.