A customer told us prompt caching on our gateway was not kicking in at the token count the model's own documentation promised. We reproduced it, then re-ran every model against a second, independent serving path, one of the largest AI gateways, where the same gaps reproduced to the token. The documentation, not any one gateway, was the optimistic party: the published minimum is an eligibility floor, not the length that earns a cache hit. For the automatic-cache families the two differ by 1.4 to 2.4x. OpenAI's effective first-hit threshold measured near 1,456 tokens against a documented 1,024; Gemini 2.5 Flash first read from cache near 5,000 against a documented 2,048; Claude, which caches only where you place an explicit marker, hit its documented per-model minimum within a few percent.
TL;DR
- OpenAI's documented 1,024-token cache minimum under-states the effective threshold of about 1,456 tokens, measured on two paths.
- Gemini 2.5 Flash documents 2,048 but first read near 5,000 tokens, about 2.4x higher.
- Claude's explicit
cache_controlhit its documented minimum within a few percent (Opus 1,073 vs 1,024). - GLM 5.2 and DeepSeek V4 publish no minimum and read from about 800 tokens; MiniMax M3 reports roughly 114 cached tokens at any length.
- Automatic caches also need a 2-to-8 call warm-up before the first read.
We ran every measurement through two serving paths, our own gateway and one of the largest independent AI gateways, and treated a result as model behavior only where the paths agreed. The point of the second path is attribution: a discrepancy that reproduces on an unrelated vendor's stack is the model's, not ours. That cross-check worked cleanly for OpenAI, Gemini, and GLM, which cached on both, at the same effective thresholds. It does not work for every model: on that second gateway the open-weight models are largely served by GPU hosts that do not implement the vendor's prompt cache, which the gateway's own endpoint metadata confirms per provider, and unpinned routing drifts across those hosts so cache affinity is lost. Where the second path could not corroborate, the numbers below come from the path that reaches each vendor's own caching API. Lengths are each model's own tokens, calibrated from the returned usage, not characters. Every arm used a fresh prefix and recorded the first call index that produced a cache read, not a single hit-or-miss.
The gap between documented and effective
The documented minimum tells you when a prompt can be cached. The effective threshold is the length at which a repeated prompt actually comes back as a read. For the automatic-cache families they are not the same number.
| Family | Cache type | Documented min | Measured first-hit | Gap |
|---|---|---|---|---|
| OpenAI GPT-5.5 / 5.4-mini | automatic | 1,024 | ≈1,456 | +40% |
| Gemini 2.5 Flash | automatic | 2,048 | ≈5,000 | 2.4x |
| Gemini 3.5 Flash | automatic | 4,096 | ≈5,200 | +27% |
| Claude Opus 4.8 / Sonnet 5 | explicit marker | 1,024 | 1,073 | exact |
| Claude Haiku 4.5 | explicit marker | 4,096 | 4,206 | exact |
The OpenAI figure held to the token across both paths: a 1,356-token prompt never read back, a 1,456-token prompt did. Gemini was the widest gap. A sweep that topped out at 3,300 tokens saw zero reads and looked like caching was off; extending the sweep to 5,000 produced a clean read on both paths at the same length. The documented 2,048 is a floor the cache is eligible at, not one it serves reads at.
The pattern across the study: the cache you mark explicitly has an accurate spec, and the cache that happens automatically does not.
The minimum is not the only undocumented variable
Clearing the effective threshold is necessary but not sufficient. The automatic-cache families need a warm-up: the first read lands on a later call, not the second.
- OpenAI: first read on call 2 to 3.
- Gemini: first read on call 4 to 8.
This matters for cost modelling. A prompt at 6,000 tokens is above every documented and effective Gemini threshold, but a workload that sends it twice and moves on can still pay full price both times, because the cache had not warmed. Short or bursty traffic pays the uncached rate even when the length qualifies. We only concluded "does not cache" after at least twelve repeated calls with a settle between them; a shorter sweep produced a false negative on Gemini that a deeper one overturned.
Cached counts also snap to fixed blocks, which is worth knowing when you reconcile a bill: 128-token blocks on OpenAI, 64-token blocks on DeepSeek. A read of 4,073 cached tokens on a 5,014-token prompt is a partial-prefix hit rounded to a block boundary, not a bug.
The caches you control are exact
Claude caches only the segments you tag with cache_control, and that control comes with an accurate spec. Every Anthropic claim we tested held:
- Per-model minimum, to the token. Opus 4.8 and Sonnet 5 first read at 1,073 tokens against a documented 1,024; Haiku 4.5 at 4,206 against 4,096. The small overage is block rounding, not drift.
- Read rate of 0.1x input. We derived each model's input price from its own cold rows, then solved the cached rate from a hit row. Opus 4.8 and Haiku 4.5 both came out at 0.10, matching the documented multiplier.
- Five-minute refresh, free on every read. Seeding a prefix and re-reading at two, four, and six minutes hit on every read. A read inside each five-minute window keeps the entry alive with no extra write.
- Cascade invalidation. With a stable system prefix and one tool defined, changing only the tool's description forced a full rewrite of the system cache below it. Changing a tool definition invalidates the system and message caches, matching the documented hierarchy.
One doc-versus-doc conflict fell out of this. A third-party table listed Claude Opus's minimum as 4,096 tokens; the measurement read at 1,073, and Anthropic's own 1,024 is the correct figure.
The open-weight families usually document nothing
The families above at least publish a number to be wrong about. The open-weight and Chinese-lab models mostly publish no minimum at all, which leaves measurement as the only option. Our provider cache comparison covers how their published rates line up once a hit lands; the question here is only the length at which the first read appears.
| Family | Documented min | Measured first-hit | Granularity |
|---|---|---|---|
| GLM 5.2 (Z.ai) | none | reads from ≈800, both paths | 64-token blocks |
| DeepSeek V4 | none | reads from ≈800 against the vendor API | 64-token blocks |
| MiniMax M3 | 512 | reports a flat ≈114 cached at any length | non-standard |
GLM 5.2 publishes no minimum length and cached from about 800 tokens with 64-token block granularity on both paths, a lower floor than any of the documented families. DeepSeek V4 also publishes no minimum and read from about 800 tokens with the same 64-token granularity, but only against its own caching API. DeepSeek's docs call the cache best-effort with no hit-rate guarantee, and that is exactly what an intermediary exposes: the other gateway serves DeepSeek through a set of GPU hosts where only DeepSeek's own endpoint implements the cache, so routing that is not pinned to that endpoint returns no reads at all.
MiniMax M3 is the case where the reported number itself misleads. It documents a 512-token minimum, but it reports a constant count near 114 cached tokens from the first call at every length from 200 to 5,000 tokens. That number does not track prompt length, and it appears even on paths that perform no caching, so it is the model's own bookkeeping rather than a signal of what was reused. This is the same lesson the newer OpenAI models teach from the other direction: the usage token fields and the real caching can disagree, so when the savings matter, reconcile against usage.cost, not the token count.
The newest models are moving the rules
Two doc-level changes are worth flagging before you assume older behavior carries forward. On the GPT-5.6 family, OpenAI's guide states that cache writes cost 1.25x the uncached input rate, where earlier families wrote for free. The same guide describes implicit caching as placing a breakpoint on the latest message, which is a different shape from prefix caching a stable system block across turns. If you want a stable prefix reused across differing user turns on those models, mark it with an explicit breakpoint rather than relying on the implicit path. Confirm the write multiplier and the minimum per model, because both now vary by family in ways a single doc page flattens.
What to do with this
- Measure your own effective threshold. Sweep prompt length in your own tokens and record the first length that returns a cache read. Do not assume the documented minimum is where hits begin.
- Budget for warm-up. Treat the first two to eight calls on a new prefix as uncached in your cost model for automatic-cache providers.
- Prefer explicit markers where the provider offers them. Claude's cache_control gave an accurate, testable spec: a known minimum, a known read rate, a known TTL, and a known invalidation rule. That predictability is worth more than a lower documented floor you cannot rely on.
- Re-baseline on new model families. Minimums, write pricing, and breakpoint behavior shifted within a single vendor's lineup during this study.
For the read rates, TTLs, and keying rules that sit behind these thresholds, our prompt caching guide has the per-provider mechanics.
The short version: the documented minimum is an eligibility floor, not a hit threshold, and for automatic caches the two differ by 1.4 to 2.4x. Verify the number that governs your bill, in your own tokens, against your own traffic.
Top comments (0)