Loading slide
Loading contents...
In January 2023, weeks after ChatGPT arrived, people began sending the musician Nick Cave songs it had written in his style. Dozens of them. He answered publicly, and the answer is the sharpest statement anyone has made of one side of this question.
Songs arise out of suffering, by which I mean they are predicated upon the complex, internal human struggle of creation and, well, as far as I know, algorithms don't feel.
And then the line that does the real work:
ChatGPT has no inner being, it has been nowhere, it has endured nothing, it has not had the audacity to reach beyond its limitations, and hence it doesn't have the capacity for a shared transcendent experience, as it has no limitations from which to transcend.
Read that last clause slowly, because it is easy to skim past and it is the whole argument.
It has no limitations from which to transcend.
The claim is not that the output is bad. Cave allows elsewhere in the same letter that a model might produce something you could not tell apart from the real thing. He calls that replication, and says it would be a kind of burlesque.
The claim is about what a song is. On this account a song is not an arrangement of words and notes that produces an effect. It is a record of a specific person hitting the edge of what they could do or bear, and going past it. The struggle is not preparation for the work. The struggle is the material the work is made of.
And a system with no inside has nothing to make it from. Not because it is not good enough yet. Because there is no limit it strains against, no cost to producing anything, nothing at stake in the attempt. Give it more parameters and more training and the gap does not narrow, because the gap was never about capability.
That is a fact about the argument's structure, whatever you make of the conclusion.
Almost every objection to these systems is a claim about what they cannot do yet, and those claims have a poor track record. They lose ground each year. This one does not, because it never made a capability claim. Improve the model as much as you like and the argument is untouched.
Which means it either identifies something real about what creation is, or it defines art in a way that rules the machine out by construction, before any of its output is examined. People who have thought hard about this land in both places, and the disagreement is not a failure to think clearly.
This journey will not settle it for you, and it would be dishonest to pretend the mechanism could. But the machinery is all in front of you now, so when you weigh it you are weighing something real. You know what the model does, what it is made of, what it was trained on, and precisely what is and is not inside it while it writes.
What Cave puts a finger on is that the interesting question may never have been whether the machine can make something good. It may be what we think making is for.