Using the model: inference

We will come back to the model's mistakes later. First, something simpler and more reassuring: what actually happens when you use one.

Here is the surprising part. Training is enormously expensive. Each use of the model, by comparison, is cheap.

Training involved adjusting billions of weights across trillions of examples. That took weeks or months of computation on specialized hardware, consuming enormous amounts of electricity. It happened once.

Using the model is called . You give it an input. It runs the input through the fixed, frozen weights. It produces an output. No learning happens. The weights don't change. It's just a very fast mathematical calculation on a fixed set of numbers.

This is why you can type a question and get an answer in under a second. The hard work was done once during training. What remains is a fast calculation, using the exact same weights, that can be repeated for millions of different users.

Cheap, though, is relative. Each answer can be cheap on its own, almost nothing next to the long, costly training behind it. But serving many people with a large model becomes expensive in bulk. This is the other reason you connect to a company's servers instead of holding the model yourself: cheap every single time, expensive in bulk.

When a company "deploys" a model, that's what they're doing: running inference at scale. The same file, the same weights, serving everyone.

Build it once, slowly. Run it forever, fast. That is the whole shape of a trained model in use.