What the three letters stand for

The name can be read now, word by word.

The paper that introduced that first model was called Improving Language Understanding by Generative Pre-Training, and two of the three words are sitting right there in the title. The third is the machine it was built on, the transformer. Put them together and you get GPT, which is a name assembled the way engineers assemble names, by listing the parts and taking the initials. Nobody was picturing it on the side of a bus.

Generative. It makes new text rather than sorting text that already exists.

Pre-trained. The broad, expensive stage happens first. Aiming it at a particular job comes after.

Transformer. The tower. Attention, then the desk work, floor after floor.

Generative Pre-trained Transformer. The whole machine in three words, and none of them are new to you.