A network of neurons

What you've been looking at has a name.

Neurons connected in layers like this are called a network. When there are many layers, people call it a deep network. That's all the word "deep" means, many layers, stacked.

With a single neuron, training is straightforward. It makes a wrong guess. You compare the guess to the right answer. You adjust its weights a little. You do it again. Over time, it gets better.

A whole network is harder to train. Here's why: the wrong answer comes out at the end, at the output, but the weights that caused it are buried inside, separated from the output by layer after layer of other neurons. Which weight was responsible? One near the output? One three layers back? There's no obvious way to know. The mistake is visible. The cause is hidden.

So the field asked the obvious question: can the Perceptron be fixed?

Minsky and Papert had already proved: no.

But that was the wrong question. The right question was: how do you train the whole network?

Nobody had proved that was impossible. They had just stopped asking.