Someone had to read it

Step away from the output for a moment and look back at how these systems are made, because there is a part of that story this journey described as machinery and never described as work.

Recall preference training. A raw model finishes its training on internet text and it is fluent, but it is not yet usable: it will answer a question with a list of similar questions, and it has absorbed everything the internet contains, including the parts nobody wants read back to them. So the model is shown a prompt, produces several answers, and a person marks which answer is better. Do that across enough examples and the model shifts toward the kind of reply people actually wanted.

The mechanism was covered. The word carrying the weight in it is person.

Somebody sat and did that, hundreds of times a shift. And the job is not only ranking helpful answers. To build the filter that stops a model producing descriptions of abuse, you need examples of that material, labelled, so the filter learns the shape of what it is refusing. Those examples come from the same internet the model was trained on. Someone has to read them and mark what they are.

That labelling has largely been contracted out, by several of the companies building these systems, to firms employing workers in countries where wages are low.

One case is documented in detail. An investigation by TIME followed a contract that began in late 2021, in which workers were paid take-home wages reported between $1.32 and $2 an hour to read and label passages describing violence, abuse against children, self-harm, and torture. Workers were expected to get through well over a hundred passages in a nine-hour shift. The outsourcing firm billed around $12.50 for each of those hours. Several workers described lasting psychological effects, and some later organised over the conditions.

Hold that next to the thing it produced. The reason a model will decline a request for something horrifying, the behaviour that makes it safe enough to put in front of the public, is not only a clever technique. It rests on people who were paid very little to absorb the material so the system would recognise it.

None of this is hidden, exactly. It is simply not what anyone means when they say a model was trained. The sentence sounds like something that happened to a machine.

The same shape shows up elsewhere in the industry and it is easy to miss. An impressive technical result, and underneath it, work that was made cheap by moving it somewhere the wage floor is lower. That is not a fact about neural networks. It is an old arrangement, wearing new clothes.