The brittleness problem
The same pattern appeared in MYCIN. In an early evaluation, its main weakness was a lack of rules for considering how severely ill a patient was.
Severity mattered to the treatment. But until someone represented it in the knowledge base, MYCIN could not use it. The team had to add rules and test the system again.
This is
The system's skill ended where its written knowledge ended.