DEV Community

Cover image for Prompt Caching Cost Is What Decides Your AI Coding Agent Bill
Nirali for Toolbit AI

Posted on Edited on

Prompt Caching Cost Is What Decides Your AI Coding Agent Bill

A coding agent sends your whole conversation back to the model on every single turn. So the price that decides your bill is often the smallest number in the pricing table: the prompt caching cost.

Max Woolf's post OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why? is getting a lot of discussion on Hacker News right now. It walks through a summer of price cuts. One line stuck with me. He writes that when you use agents for coding, cache read costs are "the vast majority (96% - 99%) of the total token expenditure." I wanted to see what that means for a real AI coding agent cost, using today's pricing pages instead of launch headlines.

Why agents live on cache reads

Every time an agent calls the model, it sends the system prompt, the tool definitions, every file it has read and every message so far. Then it adds a little new content at the end: a tool result, or your next message.

Providers cache that repeated prefix. Reading it back from the cache is much cheaper than sending it fresh:

  • On Claude Opus 5.5, a cache hit costs 5% of the base input price.
  • On GPT-6.1 Sol, it's also 5% ($0.10 vs $2.00).
  • On Claude Fable 5.1, it's 2.5%.

The new tokens at the end usually get written to the cache. That costs a bit more than normal input: 1.25x on both Anthropic's default 5-minute cache and OpenAI's default 30-minute cache.

LLM API pricing 2026: the numbers that matter

Here is what the official pricing pages say today. Prices are USD per million tokens, standard tier.

Model Input Cache read Cache write (default) Output Note
Claude Opus 5 $5.00 $0.50 $6.25 (5 min) $25 previous Opus
Claude Opus 5.5 $4.00 $0.20 $5.00 (5 min) $20
Claude Fable 5.1 $10.00 $0.25 $12.50 (5 min) $50
GPT-5.6 Sol $4.00 $0.40 $5.00 (30 min) $20 promotional price
GPT-6.1 Sol $2.00 $0.10 $2.50 (30 min) $10
Gemini 4 Argon $2.00 $0.10 not listed (implicit caching only) $10 until 30 Nov 2026

Checked 26 Oct 2026 on the Anthropic, OpenAI and Google pricing pages. OpenAI prices are for prompts up to 272K input tokens.

(I work at Toolbit, and we keep a running comparison of these prices here.)

Look at Opus 5 to Opus 5.5. Input and output dropped 20%. Cache reads dropped 60%, from $0.50 to $0.20. If you only read the headline, you'd plan for a 20% saving. An agent gets a lot more than that.

A worked example: one hour of agent work

These are made-up but realistic assumptions for one coding session:

  • 160 model calls in about an hour
  • the context averages about 100K tokens over the session (it grows, and the agent compacts it now and then), so each call re-reads about 100K cached tokens on average
  • about 2K new tokens per call (tool output, your messages), written to the cache
  • about 500 output tokens per call (mostly tool calls and short replies)

That adds up to 16M cache-read tokens, 0.32M cache-write tokens and 0.08M output tokens. Cache reads are 97.6% of all tokens, right inside Woolf's range.

Model Cache reads Cache writes Output Session total
Opus 5 $8.00 $2.00 $2.00 $12.00
Opus 5.5 $3.20 $1.60 $1.60 $6.40
GPT-5.6 Sol (promo) $6.40 $1.60 $1.60 $9.60
GPT-6.1 Sol $1.60 $0.80 $0.80 $3.20
Gemini 4 Argon (until 30 Nov) $1.60 $0.64* $0.80 $3.04
Gemini 4 Argon (from 1 Dec) $3.20 $1.28* $1.60 $6.08

Google lists no cache-write price for Argon, so I priced the new tokens at the normal input rate. Its caching is implicit, so your hit rate isn't guaranteed.

Three things stand out.

  1. Opus 5 to 5.5 is a 47% cut for this session, not 20%. The 60% cache-read cut does most of the work. A headline-only estimate would have said $9.60. That lines up with Woolf's estimate of a 40-60% total cost cut.
  2. Cache reads are the biggest line item, but not 96-99% of the money here. They're 97.6% of the tokens. In dollars they're 50-67% of the bill, because writes and output cost far more per token. If your agent writes long answers, output matters more than you'd think.
  3. Caching itself is the real saving. Run the same session on Opus 5.5 with no caching and the input alone is 16.32M × $4 = $65.28. That's $66.88 in total, against $6.40 with caching.

Stacked bar chart for an example Claude Opus 5.5 coding session: cache reads are 97.6% of tokens but 50% of the $6.40 cost; cache writes and output are 25% each

Which cuts are temporary

Not every price in that table will still be there in January.

Timeline of promotional API prices ending: GPT-5.6 Sol promo at least through 21 Nov; Gemini 4 Argon $2/$10 until 30 Nov, then $4/$20; Gemini 3.8 Flash $0.75/$3.75 until 31 Dec, then $1.50/$7.50

  • GPT-5.6 Sol: OpenAI calls its current price promotional, "available at least through November 21, 2026."
  • Gemini 4 Argon: $2 input / $10 output through 30 Nov 2026, then $4 / $20 from 1 Dec. Cache reads go from $0.10 to $0.20. The example session doubles.
  • Gemini 3.8 Flash: $0.75 / $3.75 through 31 Dec 2026, then $1.50 / $7.50 from 1 Jan 2027.
  • Claude Sonnet 5 went the other way. Its $2 / $10 launch price was due to rise to $3 / $15 on 1 Sep 2026. Anthropic made $2 / $10 the standard price instead.

If you pick a model on price, put those dates in your calendar.

How to estimate your own prompt caching cost

You don't need my assumptions. Your usage logs have the real numbers.

  1. Pull token counts per call. Anthropic's API returns cache_read_input_tokens and cache_creation_input_tokens in the usage block. OpenAI reports cached_tokens.
  2. Split them into four buckets: cache reads, cache writes, uncached input and output.
  3. Multiply each bucket by its own price. Don't use the input price for everything.
  4. Check your hit rate. Woolf says the cache tends to be hit around 90-100% of the time, depending on the provider. If yours is much lower, fix that before you switch models.
  5. Watch for cache expiry. Anthropic's default cache lives for 5 minutes and is refreshed each time it's read. Step away for a coffee and the next call writes the whole context again. On Opus 5.5, rewriting 100K tokens costs $0.50, while reading them costs $0.02. OpenAI's default is 30 minutes. Both offer longer lifetimes for a higher write price: 1 hour at 2x input on Anthropic, 12 hours at 2.5x on OpenAI.

Then redo the math for the date each temporary price ends.

The short version

Input and output prices make the news. For coding agents, the cache-read column and your hit rate decide most of the bill. Cheaper cache reads are why Opus 5.5 costs about half as much as Opus 5 in the example above, even though its headline prices only fell 20%.

AI tools helped me research and draft this post. I checked every number against the linked sources on 26 Oct 2026.

What does your agent's split look like? If you've checked your usage logs, I'd love to know how much of your bill is cache reads and how much is output.

Top comments (6)

Collapse
 
pomereneio profile image
Kyle Ma •

Price the mix, not the headline — that's the right unit. Two things worth keeping next to the $6.40 vs $66.88 number: prompt caching reuses a prefix and still generates, so the saving lands on input tokens while output and writes keep their price — a replay that skips the whole run is a different saving, and the two shouldn't be read as the same line item. And the comparison holds the session fixed: a cheaper input mix that needs more turns to finish the same task hands the saving straight back. Cache share next to turns per finished task is the pair that keeps the math honest.

Collapse
 
nirali_toolbit profile image
Nirali Toolbit AI •

Thanks, this is a really useful way to put it. You’re right that the saving only lands on the input side. In the table I kept cache writes and output at full price for that reason, so the drop mostly comes from cheaper cache reads. The part I didn’t cover is your second point: I held the session at 160 calls for every model. If a cheaper model needs more turns to finish the same task, some of that saving goes straight back. Do you track turns per finished task in your own setup? I’m curious how much it varies between models.

Collapse
 
pomereneio profile image
Kyle Ma •

Yes — turns per finished task is exactly the number we pair with cache share, for the reason you just named: a fixed 160-call session hides the case where the cheaper model needs more turns to finish the same task. In our setup the honest unit is turns-to-completion on the same task set, logged per model, next to cache share.

On how much it varies between models: I don't have a per-model turn count I can quote — we haven't published that comparison, so I won't invent one. The variation is real in principle (models differ in how much they re-plan or re-derive), but the number worth trusting is the one measured on your own task set: same tasks, same finish criteria, turns logged per model. That's the measure-first approach I wrote up here: dev.to/pomereneio/how-much-of-your... — it walks through logging repeat rate before deciding what to cache, and turns-per-task is the same kind of log.

Thread Thread
 
nirali_toolbit profile image
Nirali Toolbit AI •

That’s a fair point. Keeping every model at 160 calls was just to keep the comparison simple, but counting turns to finish the same set of tasks is the better test. I’d like to rerun it that way. I’ll read your post on logging repeat rate as well. Quick question: how do you decide a task is “finished”? Tests passing, or someone signing off on the result?

Thread Thread
 
pomereneio profile image
Kyle Ma •

Good question — it's the one that decides whether the turns number means anything. Short answer: tests passing where a test exists, human sign-off where one doesn't. The rule that matters is that the finish criterion is fixed before the run and identical for every model.

In our setup a task counts as finished when the run reaches its goal under the task's own completion check: where the task has an automated check — tests, an expected output, a state assertion — that check passing is the finish line. Where it doesn't, the run isn't logged as finished until someone signs off on the result. And a run that doesn't finish under its criterion isn't a trajectory we'd replay: only a completed trajectory gets recorded for next time, so a half-finished run never becomes a cached answer.

The reason to be strict about it: turns-per-task is only comparable across models if 'finished' means the same thing for all of them. If one model gets signed off loosely and another has to pass a test suite, you're measuring the criterion, not the model. So log the finish criterion next to the turn count, per model, on the same task set — same tasks, same finish criteria, turns logged per model. That's the same measure-first log I wrote up here: dev.to/pomereneio/how-much-of-your...

Thread Thread
 
nirali_toolbit profile image
Nirali Toolbit AI •

That makes sense. "Measuring the criterion, not the model" is the bit I'd have got wrong, since a loose sign-off on one model would quietly skew the whole table. When I rerun it I'll set the finish check per task first and log it next to the turns. Thanks for spelling it out.