DEV Community

Vaibhav Krishali
Vaibhav Krishali

Posted on Edited on

Prompt caching saved 91% on 7,407 coding-agent calls, and real logs found two bugs in my cost tool

I built a small open-source CLI, llm-cost-audit, that reads LLM call logs and ranks where money could be saved. Synthetic test data said it worked. So I pointed it at 22 days of my own Claude Code sessions to see what real data would say.

Short version: at list API prices, the 7,407 calls come to $566. Without prompt caching, the same calls would have cost $6,301. The run also caught two bugs in the tool that the synthetic tests never hit.

(I'm on a subscription, so these are API-price figures, not money I paid. The tool reads only token counts, model names and timestamps, never prompt text.)

1. Caching is doing almost all the work

A coding agent re-sends its whole conversation on every turn. The median call read 191,278 input tokens to write 358 output tokens.

98.3% of those input tokens (1.69 billion) came from the prompt cache, billed at 5% of the normal input price on the models I mostly used. Priced at the full input rate, the same 22 days come to $6,301. Caching cut the bill by 91%.

2. Where the remaining money goes

Token type Cost Share of spend
Cache reads $303.82 54%
Cache writes $190.53 34%
Output $71.43 13%
Full-rate input $0.06 0%

Cache writes are 1.7% of input tokens but 34% of spend. 92% of them are 1-hour writes, which cost 2x the input price, against 1.25x for the default 5-minute writes. Whether 5-minute writes would be cheaper here depends on how often calls are more than five minutes apart; I have not measured that yet.

3. Bug one: the tool undercounted spend by 12%

The tool priced every cache write at the 5-minute rate. On this log it reported $497.59 instead of $565.85. Any team using 1-hour caching would have got the same undercount.

The fix reads the 1-hour count Anthropic reports in each response (usage.cache_creation.ephemeral_1h_input_tokens) and prices it separately. If a model has no 1-hour price, the report now says so instead of quietly undercounting.

4. Bug two: a placeholder guess outranked a measured finding

Six of the seven findings were "extreme input:output ratio", each with a saving based on a fixed assumption that 25% of the input could be cut. They ranked above the one finding computed from the log.

For an agent, a huge input and a short answer is normal. The input is the conversation history, and 95 to 98% of it was already cached. The tool now reports this pattern as "long reused context", where the lever is compacting history, not trimming a prompt. Any finding whose saving is a placeholder is marked as illustrative and ranks after every measured one.

What I would check first on a team's agent

  1. Is caching on, and is it hitting? Here it was worth 91%.
  2. Which cache lifetime are you paying for? The write price differs by 60% between the two.
  3. Once caching works, cost grows with context length times the number of turns. Compaction and shorter sessions are the next lever.
  4. Run any cost tool, including mine, on your real logs before trusting its numbers.

Limits

  • One user, one product, 22 days. Agent cost patterns differ from chat or batch workloads.
  • Dollar figures use list prices checked on 11 November 2026, standard tier, no batch or regional pricing.
  • Calls were deduplicated by message ID from local transcript files; calls missing from those files are not counted.
  • Shares are rounded, so they do not sum to exactly 100%.

If you're running an LLM product and want the same breakdown on your own logs, the tool is free and open source. I also do this as a paid audit: audit on Fiverr, or message me.

Top comments (2)

Collapse
 
eqvero profile image
Eqvero •

The 5-minute vs 1-hour TTL math has a clean break-even point in interactive agent sessions.

A 1-hour write costs 2x base input tokens versus 1.25x for 5-minute writes. In human coding workflows, inter-turn pauses between 5 and 30 minutes (reading generated diffs or running test suites locally) are frequent. If you use 5-minute TTL, every test run that exceeds five minutes triggers a cold cache miss. Paying full input price on a 190k context wipes out the 0.75x write premium across dozens of cached calls.

One other subtle drag we found in production logs: dynamic tool schema ordering. If an agent framework re-serializes tool definitions with non-deterministic key ordering between turns, Anthropic prefix matcher treats it as a cache miss at the tool block, invalidating everything downstream and forcing a full-context write.

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

Caching makes each re-read cheaper, and the history being re-read stays just as long. Tokenade is mine: on THOL, the public benchmark I run, it reduces token costs by 39% on average on long sessions, with 170 of 170 runs passed, same as plain Claude Code.