Loading slide
One word is about to do a lot of work in this module, so let's pin it down now.
A network's layers sit between the input and the answer. In the last module, we stacked a few of them and watched each one find a slightly more specific pattern than the one before. A shallow network has just one or two of those layers. A deep network has many: ten, fifty, sometimes hundreds, stacked one after another.
That is the entire meaning of the term. Deep learning is just this idea: training a network with many layers, so each layer can build on the patterns the layer before it found.
The word makes it sound exotic. It isn't. "Deep" is a comment about the shape of the network, nothing more. Tall, if you prefer. A lot of layers, one on top of the next.
So why didn't people just stack the layers and walk away? Because for years, the moment you went deep, training broke. The next slide is why.