How Prompt Caching Works and Why Cached Input Pricing Drives Coding Agent Costs

A coding agent usually makes multiple model calls as it inspects files, edits code, and runs tests. Each call can include the conversation so far, from instructions and tool definitions to earlier messages and tool results. The provider processes those input tokens again, which can make them a large part of the task’s cost.

Prompt caching lets a provider reuse model-computed state for the unchanged beginning of a prompt, called its prefix. The model still processes new content and generates a fresh response. When the provider finds and reuses matching cached input, called a cache hit, it can charge less for those input tokens. For coding agents, the savings depend on how much context stays the same, how often the agent reuses it, and the provider’s cache pricing.

How agents build up repeated input

A coding agent typically works through a series of model calls. It might ask what to inspect, receive file contents from a tool, propose an edit, run tests, and then use the test results to decide what to do next. Each new call may include the earlier conversation as context. Claude Code’s description of its sessions illustrates how an agent resends that history as it continues working.

As the agent adds turns and tool results, its context grows. A later request can therefore contain a large amount of earlier input alongside a smaller amount of new information. The provider charges for those input tokens and the model’s output according to its pricing rules.

What prompt caching reuses

As a model processes a prompt, it calculates intermediate values used to process the text. In transformer models, these values are commonly called the key-value (KV) cache. During prompt caching, a provider saves this computed state for eligible input. When a later request starts with the same prefix, the provider can reuse the saved state instead of calculating that part again. The provider still processes any new or unmatched input and generates a new response. OpenAI’s prompt-caching documentation describes this reuse of model state.

Prompt caching reuses the model’s earlier calculations, not a completed answer. After reusing the state for matching input, the model responds to the latest request.

For an agent, the reusable prefix can include instructions, tool definitions, and earlier conversation turns. Because matching depends on the prefix, order matters: put stable content early and changing task details later to make reuse easier. A change near the beginning can prevent later content from matching the earlier prefix. The provider and model determine which prefixes and request settings qualify, so continuing a session alone does not guarantee a cache hit. OpenAI’s cache diagnostics describe conditions that affect whether a request reuses cached input.

How the cache rate changes the bill

Providers may charge different rates for ordinary input and output. They may also set separate rates for cache writes and cache reads. A cache write stores eligible input so a later request can reuse it; a cache read uses that stored input again. The first request can incur a write charge, while a later matching request receives the provider’s cache-read rate. New input and generated output remain separate costs. Anthropic’s prompt-caching documentation lists these billing categories and model-specific rates.

For an illustration, Anthropic listed these rates for Claude Opus 5.5 on its prompt-caching page as of November 9, 2026: $4 per million ordinary input tokens, $5 per million tokens written to the five-minute cache, and $0.20 per million cached-read tokens. Rates vary by model and provider.

Suppose an agent sends the same 10,000-token prefix in 10 requests, and all nine later requests reuse the prefix within the cache lifetime:

Under those assumptions, caching cuts the cost of that prefix by about 83%. It does not cut the full task bill by 83%: the calculation excludes new input on each turn, generated output, cache misses, and other billable activity.

That distinction matters because agent bills reflect both the amount of context sent and the number of calls that resend it. Anthropic’s cost guidance reports that caching reduced costs in its own coding and agent-task comparisons, with savings varying by task and session length. Those results show that caching can lower costs when an agent repeats context across a long session. They do not predict the savings for every agent or provider.

What lowers cache savings

Familiar text alone does not guarantee a cache hit. The prefix must meet the provider’s matching rules. The model and relevant settings must also be compatible, and the cached state must still be available. Providers also require prompts to meet minimum cacheable prompt lengths. Anthropic, for example, documents model-specific thresholds and five-minute and one-hour cache options. Its cost guidance explains how session length and the time between calls affect the economics.

Pauses and long tool runs can matter because generation time counts against some cache lifetimes. If the cache expires before the agent’s next request, the provider must process and write the prefix again. A longer cache lifetime can help when requests are far apart, but the provider may charge more to write input to that cache. Check the current pricing and retention rules for the model you use rather than assuming that one provider’s settings apply elsewhere.

Prompt caching also does not shrink the conversation. The agent still sends the context, and cached tokens still count toward the model’s context-window limit, the maximum amount of input it can handle in a request. Cached reads also cost money. OpenAI also notes that cached tokens count toward token-per-minute limits. Caching reduces the cost of reusing input; it does not remove input or eliminate other capacity constraints.

How to measure the savings

Track ordinary input tokens, cache-write tokens, cache-read tokens, output tokens, and requests per completed task. Those figures show whether the lower price for cache reads offsets the cost of the context the agent sends repeatedly. Then compare total cost per completed task across realistic sessions, including long tool calls and pauses. A prefix-level discount alone cannot show whether one model or agent setup completes the same work more cheaply.

For prompt design, place stable instructions and tool definitions before changing user details and tool results. Then check the provider’s usage data to see how many tokens it reports as cache reads. If the provider reports few cache reads, check whether the prefix changes between calls, whether it meets the model’s minimum length, and whether pauses exceed the cache lifetime. The specific fields and diagnostic tools vary by provider.

API token prices also do not directly determine what a subscription user pays. Subscription products may account for usage through limits or other product-specific rules, so API savings should not be presented as an equivalent reduction in a subscription bill.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com