A 30-turn measurement of how the saving amortises — and why the "one-line fix" quietly decays as conversations get longer.
Last week I publishe...
For further actions, you may consider blocking this person and/or reporting abuse
The fixed 16,896-token hit count for B explains the divergence more clearly than the percentage alone. One additional variable for the session-length analysis is the time between turns: thirty turns sent together and thirty turns spread across a day need not have the same prefix reuse in a best-effort cache.
I would stratify the next run by both length and inter-turn delay, then compute the saving from total spend across the observed session mix rather than averaging each session's percentage. That would preserve the useful B/C comparison while showing how much the append-only design delivers under actual user pacing and cache eviction.
Ran it. Four independent probes, each priming its own prefix — every prime came
back as a full miss, which is what confirms no probe was warmed by an earlier
one.
immediate re-send 99.1% hit
2 minutes 99.1%
15 minutes 99.1%
1 hour 99.1%
3 hours 99.1%
So within three hours eviction isn't the variable that matters; the prefix
survives intact. The whole sweep cost about a cent.
Two caveats I'd rather state than have you find:
Three hours is not the documented ceiling — DeepSeek says entries are cleared
"usually within a few hours to a few days", so the interesting region is past
where I stopped. A cliff may well exist; I just haven't reached it.
And my four probes ended up sharing almost identical body content because of a
seeding bug (I passed a string where the generator wanted a number, so every
probe produced the same text and only the leading marker differed). It doesn't
change the result — a cold prime each time is what establishes independence —
but I'd fix and re-run before treating the numbers as final.
Your point about total spend across the observed mix still stands. If sessions
are spread over days rather than hours, this says nothing about that regime.
The cold prime for every prefix is a useful control, and I agree the result only bounds the interval you actually tested. Thanks for flagging the seeding bug rather than presenting the sweep as final.
For the rerun, I would log a hash of each generated body alongside its seed and leading marker. That makes unintended duplicate bodies visible before the requests are sent. Repeating a few prefixes within each delay would also show whether the observed survival is consistent across requests, without turning the sweep into a claim about real traffic.
Hashing each generated body alongside its seed and leading marker is the fix I should
have had from the start — the duplicate bodies were visible in the output and I didn't
look at it that way until the numbers came back suspiciously identical.
Repeating a few prefixes within each delay is a good addition too. One request per
condition can't distinguish "the cache survives" from "that request happened to hit".
The duplicate-body check is worth making an assertion before any requests are sent. Distinct seeds alone would not catch a generator that silently collapses them, whereas distinct body hashes would expose exactly the issue you found.
For the rerun, separate independent prefixes from repeated requests within each prefix. Reporting both counts prevents a large number of repetitions on one cached body from looking like broad evidence across bodies. The cold-prime outcomes can then remain a control rather than being folded into the warm-hit estimate.
The body-hash assertion is the right shape: distinct seeds are an intention, distinct
hashes are an observation, and only the second one fails loudly. I'll assert it before
the first request rather than checking afterwards.
Separating independent prefixes from repeated requests within a prefix is also right,
and it fixes a reporting problem I'd have walked into — one prefix hit thirty times
would have looked like thirty observations.
For the Anthropic gap in your caveats, one data point from the agent benchmark of an MCP server I maintain, where the tool definitions and system prompt are identical on every turn. On gpt-5.4-mini, 48% to 90% of prompt tokens were served from cache depending on the run, with no setup. On Claude Haiku 4.5 and Sonnet 5 the same harness reported 0 cached tokens across 144 runs (about 950,000 prompt tokens), because it never set a cache_control breakpoint. So on Anthropic, layout C is necessary but not enough. Two details for your delay sweep: the default cache lifetime there is 5 minutes (1 hour at a higher write price), so 15 minutes and up would come back cold; and tools render before the system prompt, so a tool list that changes between turns acts like your variant A header. Run records: github.com/arhancanli/canlicapital...
This is the most useful thing anyone has added to either post. Three things in
it I did not know and could not have measured.
The Anthropic zero is the striking part. 144 runs and 0 cached tokens because no
cache_control breakpoint was set — that means on Anthropic the failure mode
isn't "the prefix moved", it's "nobody ever asked for the cache". Layout C is
necessary but not sufficient, exactly as you put it. That's a different bug from
the one I wrote about, and probably a more common one.
The 5-minute default lifetime also reframes my own sweep. I measured 2 minutes
through 3 hours on DeepSeek and got 99.1% at every interval, which now looks
like a provider-specific result rather than a general one.
And "tools render before the system prompt" settles a question PromptAlo raised
in the other thread. If the tool list changes between turns there is nothing
ahead of it to preserve, so it's worse than the header rather than equivalent.
Thanks for the run records link — reading them now. If the harness is public I'd
like to run the same questions against DeepSeek to see whether the entry-point
difference survives a provider change.
It's public: github.com/arhancanli/canlicapital... (run.mjs, and tasks.mjs with the ground truth for each task). Two notes for a DeepSeek run. run.mjs calls the OpenAI and Anthropic endpoints directly, so you'd point the OpenAI path at DeepSeek's OpenAI-compatible base URL. And the cached count comes from each provider's usage field, which isn't named the same everywhere, so check where DeepSeek reports cache hits before comparing numbers across providers. I'd like to see what you get.
Thanks for making it public — I've got the repo open. Two things in your note I'd have
got wrong on my own: the usage field naming, and pointing the OpenAI path at DeepSeek's
compatible base URL rather than rewriting the call.
I'll check where DeepSeek reports cache hits before I trust any cross-provider
comparison, and I'll post the numbers here either way — including if they don't
reproduce.
One thing I should say before running it: my own sweep had a seeding bug that produced
identical bodies across conditions, and Ahmet pointed out I should be hashing the
generated bodies rather than trusting the seed. So if the DeepSeek run disagrees with
your OpenAI and Anthropic numbers, the bug is at least as likely to be mine as a real
provider difference.