Why add more layers?

We have seen why more layers are useful. Each one can pick up on patterns in what the layer before it found. A network recognising a handwritten 8 might spot short strokes in one layer, join them into curves in the next, and close those curves into two loops in the one after that.

More layers offered more stages in which to build one pattern from another.

Adding layers to a design was easy. Training some deep was not.

In these networks, adding stages could make the early layers learn slowly or stop improving. Greater depth offered more representational steps in principle, while creating a longer and more fragile path for corrections during training.

That is the problem this chapter is about.