The rulebook disappeared

One widely used kind of machine learning, a neural network, changes what an explanation looks like.

In a short program made from written rules, a mistake may lead back to one faulty instruction. A person can read that instruction and change it.

During training, a neural network repeatedly adjusts many numbers called parameters. Together, those numbers shape how inputs are transformed into outputs.

The result is not a readable list of rules. There may be no single line that contains the complete reason for one answer.

That does not make explanation impossible. Researchers can test the system, trace some of its internal activity, and discover patterns in what it learned. But they cannot inspect it in the same way they inspect a list of written rules.

Neural networks made some difficult tasks possible. In exchange, an individual result can be harder to explain than a decision produced by a short list of rules.

The rulebook disappeared

One widely used kind of machine learning, a neural network, changes what an explanation looks like.

In a short program made from written rules, a mistake may lead back to one faulty instruction. A person can read that instruction and change it.

During training, a neural network repeatedly adjusts many numbers called parameters. Together, those numbers shape how inputs are transformed into outputs.

The result is not a readable list of rules. There may be no single line that contains the complete reason for one answer.

Neural networks made some difficult tasks possible. In exchange, an individual result can be harder to explain than a decision produced by a short list of rules.