The Work Already Done

A chat can grow by one sentence while the request behind it grows by thousands of pieces of text.

The application sends the instructions and earlier conversation again. The model needs that material to answer, but processing the same beginning from scratch would repeat work the system has already done.

Prompt caching is a shortcut for that repeated beginning.

It keeps useful working results from an earlier request. If a later request starts the same way, the system can reuse those results and continue from the first new part.