Prompt caching is an optimization where the model provider stores the internal computation for the beginning of a prompt, the prefix, and reuses it when subsequent requests start with exactly the same text. Production prompts are mostly static: the same system instructions, policy text, tool definitions, and few-shot examples are resent on every single call, with only the user's question changing at the end. Without caching, the model reprocesses all of that identical content every time; with caching, it picks up from where the prompts diverge.
The savings are substantial because the static prefix often dwarfs the dynamic suffix. Cached input tokens are billed at a steep discount, Anthropic charges roughly a tenth of the normal input price for cache reads, and OpenAI applies automatic discounts on repeated prefixes, while time-to-first-token drops because the prefix computation is skipped. The technique matters most for agents, whose long tool definitions are resent on every step of a loop, for chatbots that reprocess growing conversation history on each turn, and for any application with a large standing context. The one design requirement is prefix stability: caches match exact leading text, so prompts must be structured with static content first and variable content last, and a timestamp inserted at the top of a prompt silently destroys every cache hit.
At arosplatforms we structure prompts for cacheability as a matter of routine and measure cache hit rates in production. For agent-heavy and high-volume systems, this single optimization frequently cuts inference spend by half or more, which is often the difference between a use case that clears its ROI bar and one that does not.