What to keep

Depth gave a network more stages in which to build useful patterns.

It also gave the training correction a longer route back. Repeated multiplication could make that correction almost disappear before it reached the earliest layers. This was one important reason deep networks were difficult to train.

Better starting weights, layer-by-layer preparation, and less-squeezing activation functions all helped.

You do not need to memorise every technique. Keep the central problem: the correction must stay useful all the way back through the network.