So rearrange the street

So the behaviour is not the thing to fix. Treating close numbers as close cases is what makes a network able to handle anything it was not explicitly trained on, and a model without it would be far worse.

The numbers are the thing to fix, and they are ours. Which points straight at the answer, and you may already have it. If the addresses are the problem, change the addresses. Nothing forces us to keep them alphabetical. We chose those numbers, so we can choose better ones.

Put dog at 2 and puppy at 3, side by side, so that everything the model learns about one it half knows about the other. Move dollar out to 90, far enough that nothing about dogs leaks into it. The arithmetic has not changed at all. The same neighbour-sharing that was ruining everything is now doing exactly what we want.

That is genuinely the right idea. The question is how far it gets.