A small step inside each neuron

To see why the signal faded, we have to zoom in on a single neuron. And to do that, we need one small idea first: a function.

A function is just a fixed rule for turning one number into another. You put a number in, you get a number out, and the rule never changes its mind. "Double it" is a function: put in 3, you get 6. Put in 10, you get 20. That is all a function is. A little machine with a number going in one end and a number coming out the other.

Now back to the neuron. Remember what it does from the last module: it adds up its inputs into a single total. But it has one more step after that, before it passes its answer along. It runs that total through a function. The number that comes out is what it hands to the next layer.

This final rule has a name, the . Every neuron has one. It is a tiny step, easy to overlook.

Which raises a fair question. If the neuron already added everything up, why not just pass that total straight on? Why run it through anything at all?