Back to the feeling

Remember ELIZA. Some replies sounded attentive. Then a strange repetition revealed the prepared rules underneath.

With ELIZA, the gap was easy to notice. A more capable machine makes it harder.

Suppose every reply fits. The machine follows the conversation, handles a new example, and explains its answer clearly. What would that behavior prove?

It would show what the machine can do. It would not let us look directly at what the words mean to it, or whether there is any experience behind them.

This chapter asks what should count as understanding when behavior is all we can see.