Drop one tool from your request: one GPT-5 version keeps 76% of it cached, another keeps nothing

Prompt caching is the cheapest performance win within an LLM based application. An application sends a long system prompt with each request and the model provider checks recognizes the part it had seen before (from the beginning) and it charges you a fraction of actual token cost instead of reprocessing them. An agent that uses tools where the system prompt and tool definitions can be upto several thousand tokens and barely change between turns, the discount is enormous.

The advice to benefit from the caching is simple - keep the front of your prompt stable. Have the static content first, variable content last, and freeze your tool definitions so the cached prefix stays intact. The advice is correct but it is also version-dependent in a way nobody mentions, and the dependence is severe.

I ran one experiment by dropping a single tool from the request and changing nothing else across four versions of the same model family for fifteen times per version, in three batches across two days. On gpt-5.2 the change cost me a quarter of my cache. On gpt-5.5, two versions later, the identical change threw away the entire cache, system prompt included. And on gpt-5.1, the same test wouldn't settle on one answer, which turned out to be a finding of its own and its the reason that this post reports replicated runs instead of single ones.

This post is the experiment, its results and what they mean for anything that changes its tool list at runtime.

Here is the short version, before the method. The same operation on every row - remove one tool, percent of the request still served from cache:

model	drop the last tool	drop the first tool
gpt-5.2	76%	46%
gpt-5.4	~80%	0%
gpt-5.5	0%	0%
gpt-5.1	wouldn't settle on one answer so it has its own section below

The rest of the post is how these numbers were measured, why they can be trusted and what to do about them. The whole experiment runs on a small synthetic rig which you can copy.

You'll get the most out of this if you're already comfortable with:

• how a chat completion request is structured - a system message, tools definitions and a user message
• the idea that the model sees your request as one flat sequence of tokens, not as separate fields
• roughly what prompt caching does (charges less for a repeated prefix). If that's new to you then the OpenAI and Azure caching docs are a fine ten-minute primer first

Caching is a prefix match

Prompt caching is not semantic matching as the providers do not lookup by context instead it walks your token sequence from the first token, left to right and then finds the longest run that is byte-for-byte identical to something already in its cache. That longest byte-for-byte identical sequence is the prefix and everything after the identical sequence mis-match/break is processed as fresh, at a full price.

1. Position is everything - A change near the begining of your prompt invalidates the front and everything after it because the match stops at the first difference. A change near the back invalidates only the back.
2. Identical-but-unreachable content is worthless - If two requests differ at token 50, then the 3,000 identical tokens sitting at position 51 onward cache nothing as they are past the break.
3. Size floor - Providers only cache a prefix once it clears a minimum length of 1,024 tokens on the OpenAI/Azure models. Anything below that, you get no caching, no error, just a bill.

Synthetic agent with nothing proprietary in it

To measure this cleanly I needed an agent shaped request that I could publish. So the domain is a fictional public library catalogue. Structurally it mirrors a real tool using agent, and three properties that are baked in:

• Long static system prompt (~1.8k tokens) - the analyst instructions, workflow rules, formatting rules.
• Small varying block near the top - a few lines describing which collection we're working over. It carries counts, not lists, so it is nearly constant width and differs between collections only in its digits. This is an analogue of a per-user or per-dataset block.
• 8 tool definitions - passed in as tools= which are deliberately verbose so the block clears the floor of 1,024 tokens on its own (~1.5k tokens).

One measurement that matters comes from the API itself:

def cached(resp) -> int:
    # prompt_tokens_details can arrive as a plain dict on older SDKs,
    # so read it dict-safely rather than via attribute access.
    d = resp.usage.model_dump().get("prompt_tokens_details") or {}
    return d.get("cached_tokens", 0) or 0

cached_tokens is how many tokens of this request were served from cache. That single number is the whole experiment.

Note

On older SDK versions, prompt_tokens_details is returned as a raw dict instead of a typed object, so getattr(details, "cached_tokens", 0) silently returns 0 and every request looks like a cache miss. If you are seeing zeros everywhere, try printing response.usage.model_dump() first and confirm the field is actually there before believing the miss.

Measurement: two traps, a control, and replication

Trap 1 - Run the same request twice, the first run wrote the cache and the second run hits the cache of the first run, and a genuine miss looks like a hit. The fix for this is a nonce: a fresh unique comment prepended to the shared text on every run, so each run starts cold. It goes at the head of the shared block, so the requests within a run still share everything after it.

Trap 2 - The write isn't visible yet as a cache write takes a moment to become readable. Fire the second request too soon and a real hit reads as a miss. The fix is a 30 second sleep between the write and the read for this experiment.

Positive control - After the test requests, I re-send the very first request unchanged. If that doesn't come back as a hit, the run is void as the write never became visible, and a zero on a test request proves nothing. Void runs are excluded, and I report how many there were.

Replication - Every cell below was run fifteen times, in three batches - the third batch three days after the first two, to test whether the numbers survive time. Because I anchored the first draft of this post on a single clean run of gpt-5.1 and five runs later, that cell turned out to be bimodal. A single run of a nondeterministic process hands you one of its modes with total confidence.

The question worth asking

Can I change which tools I send from one request to the next without paying for the whole prompt again?

This is not hypothetical and it is the normal life of an agent, and it's a question developers have asked on OpenAI's own forum without an answer.

• MCP server connecting or disconnecting mid-session changes the tool list with no action on your part; a subprocess exits, a session expires, a server reconnects
• Skill loader that filters tools by context sends a different subset each turn
• Any per-turn tool selection logic reshapes the array as it goes

A February 2026 evaluation of prompt caching for agentic workloads (Lumer et al., "Don't Break the Cache", arXiv:2601.06007) states the rule plainly: when tool definitions are in the prompt, "any change to the available tool set invalidates the cached prefix" and recommends a fixed tool set as the mitigation. (Their experiments control cache boundaries with inserted UUIDs - the same trick as my nonce, arrived at independently.)

So the test holds the system prompt byte-for-byte identical and changes only the tool list, in two specific ways:

a       = call(system, TOOLS)        # all 8 tools - cold, writes the entry
# ... 30s sleep ...
keep_17 = call(system, TOOLS[:-1])   # drop the LAST tool  -> a strict prefix of TOOLS
keep_28 = call(system, TOOLS[1:])    # drop the FIRST tool -> diverges at the top of the tools definition block
a2      = call(system, TOOLS)        # all 8 again - positive control. Same call as first (a)

TOOLS[:-1] keeps tools 1 to 7 and is a strict prefix of the full list. If the tools block prefix-matches internally, this should stay mostly cached. TOOLS[1:] keeps tools 2 to 8 and diverges right at the top of the tools block and this tells us whether a change at the front of the tools reaches back and kills the system prompt above it.

Result on gpt-5.2

Fifteen runs, three batches, spanning two days. Every run, to the token:

keep 1-8  (control)    cached = 3,328
keep 1-7  (drop last)  cached = 2,560
keep 2-8  (drop first) cached = 1,536

Read those against each other, because the story is entirely in the comparison.

Dropping the last tool kept 2,560 tokens cached which is about 76% of the ~3.4k-token request. The system prompt and most of the tool block survived, because the shortened list is a byte-for-byte prefix of the original, the match runs down toward where the eighth tool would begin.

Dropping the first tool kept 1,536 which is roughly the system prompt and nothing else. Removing a tool at the top of the block changed the first tool-definition token, so the match broke there, and the remaining seven definitions reprocessed at full price.

Same operation - remove one tool. The only difference is which end you cut from, and it's the difference between keeping three-quarters of your cache and keeping half.

So on this version -

Order your tool definitions by how often they change. Put the stable ones first and the volatile ones last. A tool added or removed at the **tail* is cheap; the same change at the head costs you every tool behind it.*

Important Note

One wrinkle in the exact numbers: on a partial match, the cache sometimes gives up a block or two more than "shared span, rounded down to 128" would predict which is one to six blocks in my data, varying by version. An identical resend, by contrast it holds onto nearly everything. So I'm reporting the pattern, not a formula for it. What's rock-solid is this: every cached_tokens value this rig produced - over 300; 300 of them ship as raw CSVs with the code which landed exactly on a 128-block boundary, and wherever delivery was reliable, the ordering held without exception: full match > drop-last > drop-first. That ordering is what every conclusion here rests on. Exactly how many blocks survive at a partial boundary isn't documented as far as I know.

Result on gpt-5.5

Same script as previous. I just changed the model string.

keep 1-8  (control)    cached = 3,200
keep 1-7  (drop last)  cached = 0
keep 2-8  (drop first) cached = 0

The control hit so the runs are valid, caching is working, an identical resend is served from cache. But both tool-change variants dropped to 0. Its not reduced but Zero on every valid run across three batches.

Look at what keep 1-7 means here. That request shares a byte-for-byte identical ~1.9k-token system prompt with the original, and differs only by dropping the last tool - the 88 tokens at the very end. On gpt-5.2 that change kept 2,560 tokens. On gpt-5.5 it kept nothing. Removing one tool from the end of the list destroyed the cache for the entire request, system prompt and everything.

On gpt-5.5 the tools block is effectively atomic, and touching it reaches all the way back to the front of the prompt. Any change to the tool list like one tool added by an MCP server, or the MCP server reconnecting (with tool definition update), one dropped by a skill filter - reprocesses everything, every turn, with no error telling you so it's reprocessing at full price. This is the version where the literature's blanket rule that says "any tool-set change invalidates the prefix" is exactly right. On 5.1 and 5.2, the same rule is measurably incomplete.

Was it the model, or the provider?

One confound needed clearing early is that my first runs were against Azure OpenAI, later ones against the OpenAI API directly. So I ran the identical test on gpt-5.1 against both providers. The four values came back identical to the token, every one landing on the same match boundaries.

Two caveats: Those were single runs, made before I discovered that gpt-5.1's cache hit delivery is nondeterministic (next section), so "identical" is partly luck of the draw, and the durable claim is that both providers produce values on the same boundaries. And once provider had shown no effect, I did all replication on one provider (OpenAI direct) to keep it a controlled non-variable. I found no provider-dependent behaviour anywhere in this work, but the deep replication is single-provider.

The trend across versions

Here is the full set where each cell is the value from fifteen runs.

model	drop last tool	drop first tool	identical resend (control)	runs
gpt-5.1	3,072 or 0	1,664 or 0	3,328 (mostly)	15 - see next section
gpt-5.2	2,560	1,536	3,328	15/15 identical
gpt-5.4	2,688	0	2,688	14/15 identical; one delivery miss
gpt-5.5	0	0	3,200	13/15 valid; all valid runs agree

(gpt-5.3 wasn't available on the API as a pinned model, so there's a gap in the middle. The two excluded gpt-5.5 runs were void as their controls hadn't landed at 30 seconds and one gpt-5.4 run in the third batch blanked both lookups while its control hit; all three carry the delayed-write signature the next section pins down.)

Read the columns separately, because two different things change across versions.

Tolerance to a change at the head of the tools collapses first. On 5.2, changing the first tool still holds the system prompt (1,536 cached). On 5.4 that drops to 0 which means a head change now reaches back and kills the system prompt above it. That happens a full version before the block goes completely atomic.

Tolerance to a change at the tail survives longer, then it also dies. keep 1-7 is still 2,688 on 5.4 - trim the last tool, keep most of the prefix. Then on 5.5 even that goes to 0.

So it is a staged regression, not a flip:

• 5.2 - forgiving at both ends. Tail changes cheap; head changes cost only the tools.
• 5.4 - forgiving only at the tail. A head change is now catastrophic.
• 5.5 - atomic. Any tool change, either end, throws away everything.

One number that is not noise: gpt-5.4 retains 2,688 on a tail-trim where gpt-5.2 retains 2,560. 5.2 produced exactly 2,560 fifteen times out of fifteen; 5.4 delivered exactly 2,688 in fourteen of fifteen and never a different number. No run on either version ever produced any other nonzero value, so that one-block gap is a real, stable difference between versions.

Important Note

The control column deserves its own glance. On gpt-5.4 an identical resend caps at 2,688, where the other versions retain 3,200–3,328 for the same request which is replicated across all fifteen runs, including a batch three days later, so it's real and stable across days: 5.4 retains a shorter maximum span even on a perfect match. It doesn't line up with the tool-change trend, so I'm flagging it. One thing it does not undermine: the zeros in that row. The control's only job is to prove the cache write was visible when the test requests ran, and a non-zero control proves exactly that, whatever its ceiling. The 5.4 zeros stand.

Version that didn't hold still: gpt-5.1

My first run with gpt-5.1 produced the cleanest numbers in the whole family - 3,072 on a tail-trim, 1,664 on a head-trim, 3,328 on the control. Then the five-run replication came back:

run 1:  keep1-7= 3,072   keep2-8=     0   control= 1,664
run 2:  keep1-7= 3,072   keep2-8= 1,664   control= 3,328
run 3:  keep1-7=     0   keep2-8=     0   control= 3,328
run 4:  keep1-7=     0   keep2-8= 1,664   control= 3,072
run 5:  keep1-7=     0   keep2-8= 1,664   control= 3,072

A second five-run batch flipped differently where three runs where only the head trim hit, two where only the tail trim hit, controls clean throughout. Same minutes, same account, gpt-5.2 sitting at zero variance in between. A third batch, three days later, flipped again and this time toward clean where nine of the ten lookups delivered, the tenth was a single familiar miss, and the hit values never moved an inch.

Two observations turn this from a mess into a result.

First: every value that ever appears is exactly a predicted boundary value. 3,072 is the tail-trim boundary. 1,664 is the head-trim boundary. 3,328 is the full match. Across fifteen runs of this cell - and more than 300 readings in this whole investigation - not one intermediate value ever appeared. The nondeterminism is in whether a hit is delivered, never in where the match breaks. Boundaries are deterministic; delivery is not.

Second: the failure modes are identifiable - and the rates move. I ran a stripped-down diagnostic - one cold write, thirty seconds, one identical resend, then a second resend one second after that - ten rounds, interleaved with gpt-5.2 so both saw the same minutes and the same load. Then I ran the identical script again three days later. On the first day, gpt-5.1 hit ten out of ten while gpt-5.2 - flawless in the full protocol - missed twice, each miss caught by the follow-up resend one second later. Three days on, the same script inverted: 5.2 ten out of ten, 5.1 missing three, every miss again caught one second later. The flaky one swapped.

Across both days, forty simple rounds: not one entry ever went missing. Five first-resends came back empty, and a resend one second later found every single one. Whether those writes landed a beat late or the first lookup simply missed them isn't separable from outside - either way it's the same signature that voided the two gpt-5.5 runs in the main table and blanked one gpt-5.4 run in the third batch. One mode did stay version-specific across every batch: only gpt-5.1, and only with several near-identical entries in flight, ever missed a lookup against an entry that was demonstrably being served seconds on either side of the miss.

So the honest summary is not "5.1 is the broken one." It's this:

Cache delivery is best-effort on every version, and the failure rates belong to the moment, not the version - the model that flaked on one day was the clean one three days later. OpenAI's own documentation describes routing by a hash of the prompt's initial tokens and makes no guarantee a hit will occur, which makes distributed-delivery effects the plausible family of explanations but the mechanism isn't observable from outside. What is observable with zero exceptions across more than 300 readings and two days is the boundary behaviour, where a match breaks when you change your tools is deterministic, replicable, and version-scoped. That's the part the practical advice rests on, and the part the rest of this post measured.

It also yields the meta-lesson I had to learn mid-draft: a single measurement of a bimodal process will hand you one of its modes with total confidence. I would recommend to replicate before you believe your own number.

This also forces a question back onto the main table: if delivery can silently fail then how do I know the zeros on 5.4 and 5.5 are boundaries rather than strings of delivery failures? Because failure, where it exists, is visible and those cells don't look like it. Within the gpt-5.4 runs, the other partial-match lookup delivered fourteen of fifteen at 2,688 and there is one miss carrying the delayed-write signature, both lookups blank while the control hit, while keep 2-8 returned zero fifteen out of fifteen, across three batches. A lookup that misses roughly once in fifteen does not miss fifteen straight times on its sibling. On gpt-5.5, the control hit in all thirteen valid runs while both variants sat at zero in every one. And gpt-5.1 shows what delivery flakiness actually looks like with mixed outcomes inside a five-run batch. Fifteen straight zeros beside a sibling that misses once in fifteen is a rule.

A note on the newest version

The obvious next question is gpt-5.6 and it sits outside this comparison, and the reason is: on the newer model, sending function tools on the Chat Completions endpoint is rejected outright unless you disable reasoning, and tool use is steered toward the Responses API instead, which as per the documentation caches statefully on a different mechanism. So it can't be dropped onto the same axis without changing several variables at once.

Why this matters in practice

Pulling it back to decisions you actually make:

• If your tool list is fixed for the life of a session, none of this touches you as the prefix never changes, and you get the discount on every version. Freeze-and-forget is fine.
• If anything reshapes your tool list at runtime like MCP servers connecting and dropping, context-driven skill loaders, per-turn selection then on the atomic versions you may be paying full price for your entire prompt on every turn, silently. The cached_tokens field is the only place it shows. On this rig's own numbers: one atomic invalidation re-bills ~3,200 tokens at full price that would otherwise it would have cost a tenth which is roughly 2,900 token-equivalents per event. Multiply by how often your tool list actually changes, then by your model's input price, and decide whether that's a noise or a line item.
• On versions where the block prefix-matches internally (5.1, 5.2 here), order your definitions by volatility: stable tools first, the ones that come and go last. Append at the tail, never insert at the head. On atomic versions this costs nothing and buys nothing which is exactly why the advice has to name its version.
• Sorting your tool array alphabetically - this is the advice that circulates for making the list deterministic which fixes non-determinism but does nothing about insertion cost: a newly added tool lands wherever the alphabet puts it, often mid-block, invalidating everything below it on the versions where position matters, instead sort by (volatility, name): deterministic and cache-friendly, where the version rewards it.
• Measure on the model you actually ship and then replicate. This is the real lesson, three times over. The same code, a few versions apart, went from keeping three-quarters of its cache to keeping none. A single run of one version handed me a clean number that five runs dissolved. And the same script on a different day returned different failure rates with every boundary value intact. Any caching claim that doesn't name a version, a sample size, and a date has an expiry date of its own, and you won't get a warning when it passes.

The whole thing is a small self contained repo - the rig, a few short scripts, and the raw per-run CSVs with nothing proprietary in it, so you can point it at your own model and read your own numbers, including the messy ones.

Every number here was measured on 2026-07-17 and 2026-07-20, on one rig shape (~1.9k-token system prompt, ~1.5k tokens of tools, eight definitions); cache behaviour is server-side configuration and can change under the same model string with no version bump, so treat these as a dated snapshot and the rig as the durable artifact.

The only caching behaviour you can trust is the one you measured more than once, on the model you're actually running.

Thank you for reading!

---

References & Further Reading

1. Code + raw run data
2. Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks
3. OpenAI documentation