Anthropic emailed our org last week: prompt cache hit rate is low, and caching repeated content "could save up to 64% of direct API spend." The email is careful with scope — direct API only, Claude Code excluded because it manages caching itself. I went to check what that 64% was made of.
Our direct API traffic over the period: 16 calls, $0.25 total, average prompt 90 tokens. They come from traceguard's rerun harness, which replays self-contained consults as fresh single-turn calls — every prompt unique by construction. A workload like that cannot hit cache: there is no repeated prefix across calls, and 90 tokens is far below the minimum Anthropic will cache at all (1,024 tokens on Opus 4.8, the model those calls ran on; 512 on Opus 5 / Fable 5). Hit rate 0%, structurally. 64% of $0.25 is $0.16.
The metric was right. The money wasn't there.
The side the email excluded
The interesting numbers sit in the traffic the email correctly excluded. We ingest our own Claude Code session logs into traceguard's trace store (Claude Code transcripts carry no cost field, so we compute list price from usage). 71 days, 158 sessions, 58,753 API messages. This is section 1 of the command output, unedited:
| model | messages | prompt tok | hit rate | input cost | no-cache | saved | saved % |
|---|---|---|---|---|---|---|---|
| claude-opus-4-8 | 23,759 | 5,535,807,891 | 96.1% | $4,616.39 | $27,691.44 | $23,075.05 | 83.3% |
| claude-fable-5 | 15,291 | 3,583,420,473 | 96.2% | $5,847.08 | $35,834.20 | $29,987.13 | 83.7% |
| claude-opus-5 | 6,316 | 1,750,760,343 | 97.7% | $1,205.08 | $8,753.80 | $7,548.72 | 86.2% |
| claude-sonnet-5 | 11,139 | 1,326,223,701 | 95.1% | $422.96 | $2,652.45 | $2,229.49 | 84.1% |
| claude-haiku-4-5-20251001 | 1,573 | 57,312,161 | 93.9% | $9.76 | $57.31 | $47.56 | 83.0% |
| claude-opus-4-7 | 64 | 11,045,966 | 96.6% | $9.04 | $55.23 | $46.19 | 83.6% |
| claude-sonnet-4-5-20250929 | 2 | 42,410 | 0.0% | n/a | n/a | n/a | n/a |
| (none) | 609 | 0 | n/a | n/a | n/a | n/a | n/a |
| TOTAL | 58,753 | 12,264,612,945 | 96.2% | $12,110.31 | $75,044.44 | $62,934.13 | 83.9% |
Hit rate is token-weighted; costs are the input side only, at list price. The two n/a rows are deliberate — no published price for a model means no guessed money, tokens counted anyway.
Caching nobody at our org configured cuts the input side by 83.9%. With output included, the whole bill lands at 1/5.5 of the no-cache counterfactual. There is nothing left to optimize here; the remaining spend is mostly first-write premiums on genuinely new content.
The keep-alive ping idea
There's a folk optimization going around: ping your session on a timer so the cache stays warm and the next turn hits. The mechanics say it could work — the cache TTL is 5 minutes by default, every hit refreshes it for free, and a 1-hour TTL costs 2× on writes. Instead of arguing, we ran the counterfactual on our own gap data.
97.3% of our in-session gaps are under 5 minutes; they need nothing. The addressable prize is 422 gaps longer than an hour: the context rewrites right after them cost at most $1,913 over 71 days (an overestimate, since it counts genuinely new content as rewrite). Bridging those same gaps with a ping every 55 minutes would have taken 6,765 pings costing $2,009 in cache reads alone, before counting the pings' own output tokens. The strategy loses money on our data, under assumptions tilted in its favor.
The reason is boring: the 1-hour TTL removed the ping's niche. Our logs show Claude Code already routes nearly all cache writes into the 1h bucket (opus-4-8: 155.9M tokens written at 1h TTL vs 56.4M at 5m). Pinging is a hack from before extended TTL existed. If a tool sells you auto-ping in 2026, ask it for the counterfactual on your logs first.
Reading is not measuring, again
While writing this I checked our own SDK and found the same class of bug I keep filing against other trackers. traceguard's wrap_anthropic recorded tokens_in = usage.input_tokens. On Anthropic's API, input_tokens excludes cache reads and writes — and on our traffic, cache reads run about 1,000× larger than raw input, so the wrapper under-counted by three orders of magnitude. Every field read correctly, the quantity still wrong (the splitrail #220 pattern). Fixed in PR #39, and the fix is what makes the next part possible.
A rate without a denominator is not a finding. Percentages travel well in emails; dollars make decisions. Before acting on a metric email, multiply it by the money it applies to.
Check yours
This analysis is now one command, merged in PR #40: python -m traceguard.routing_audit.cache_audit. Point it at your ingested Claude Code transcripts or a traceguard trace store, and get the table above for your own data — per-model hit rate, actual vs no-cache cost at list price, gap distribution, and a yes/no on whether keep-alive pings would pay for you (ours printed NOT WORTH IT). Ships in traceguard 1.2.0 — pip install -U traceguard. The last line of ours came out like this:
Claude Code caching already saves us 84% ($12,110.31 vs $75,044.44 list). Checked with: python -m traceguard.routing_audit.cache_audit
If you run it, send me your numbers. Especially if pings win somewhere — I'd like to see the traffic shape that makes them rational.
Li Zhuojun
Top comments (2)
The keep-alive ping section is the useful bit. Cache advice often stops at the hit rate, but the gap distribution decides whether the trick has any economic shape. I'd probably graph cost by idle gap next to the token table, because that's where the 1h TTL quietly changes the decision.
64% of $0.25 is a brutal punchline. I'm from the SEO side and our tools pull the same trick, site health drops 8 points and the email never says what that means in actual traffic or money. Also had no idea Claude Code was already routing most writes to the 1h TTL. I had the keep-alive ping saved in my notes from some Twitter thread and apparently never needed it. Going to run cache_audit on my own logs, curious what a mostly vibe coded workload looks like.