DEV Community

Jangwook Kim
Jangwook Kim

Posted on • Originally published at effloow.com

GPT-5.6 Prompt Caching: The 24-Hour Cache Is Gone

#openai #promptcaching #gpt56 #apicost

Say your product sends the same long block of instructions to an AI model on every request. A returns policy. A tone-of-voice sheet. A product catalogue extract. That block doesn't change between customers. You still paid for it on every single call, the way you'd pay a narrator to read the same chapter aloud to every visitor who walks in.

Prompt caching is the fix. The provider keeps the unchanging front section of your request in fast storage and charges you a fraction of the normal rate to reuse it. On OpenAI's GPT-5.5 models that reuse window stretched to 24 hours by default. Then GPT-5.6 shipped, and the rules changed underneath the feature.

We ran the API on both model families to see exactly what changed. One finding matters more than the rest: on GPT-5.6 the 24-hour window is gone, replaced by 30 minutes. And filling the cache is itself a billable line item, priced above plain input.

What actually changed between the two model families

OpenAI's API changelog is short about this. The July 9, 2026 entry lists "explicit prompt caching controls" as part of the GPT-5.6 release. The May 29, 2026 entry says that for organizations without zero-data-retention, cache lifetime defaults to 24 hours instead of a few minutes in memory.

Read together, those two entries suggest the feature only got better. The documentation tells a different story. GPT-5.6 introduces a new prompt_cache_options block next to the older prompt_cache_retention setting, plus a way to mark exactly where the reusable section of your prompt ends instead of letting the system guess. That marker is the "explicit control" the changelog means.

What the changelog does not mention: the new options block accepts a much shorter lifetime, and GPT-5.6 reports cache writes as their own billable quantity. We wanted both confirmed by the API itself rather than inferred from a docs page.

What we built and ran

Effloow Lab ran an OpenAI API check on 2026-07-20 using scripts/gpt56-explicit-cache-probe.py, a small budget-guarded script in this repository.

The setup is deliberately boring. We wrote a 9,208-character fictional operations handbook (invented warehouse rules, no real company, no customer data) and sent it as the unchanging front section of every request, followed by one short question about it. That prefix measured 2,032 tokens on arrival, comfortably over the 1,024-token minimum OpenAI documents for caching to engage at all.

Then four configurations, same text every time:

  • A: GPT-5.6 with caching left on automatic
  • B: GPT-5.6 with the explicit mode switched on and the end of the handbook marked as the cache boundary
  • B2: same as B, but asking for a 24-hour cache lifetime
  • C: GPT-5.5 with the old 24-hour retention setting, as a control

Four calls per configuration, roughly two seconds apart. Every call passed through the repository's token budget guard before and after. The full commands, raw console output, and limitations are in the public lab note.

What actually happened

Caching worked immediately in every GPT-5.6 configuration. The first call was a miss, as expected, and every call after it reused almost the whole handbook.

Configuration Tokens sent per call Tokens reused on repeat calls Share of the prompt reused Charged as a cache write on call 1
GPT-5.6, automatic 2,032 2,029 99.9% 2,029
GPT-5.6, explicit marker 2,032 2,001 98.5% 2,001
GPT-5.5, 24h retention 2,032 1,792 88.2% field not present

Three things stand out.

The 24-hour cache is not available on GPT-5.6. Configuration B2 came back with a flat rejection: Invalid value: '24h'. Supported values are: '30m'. So a reuse window that used to last a full day now lasts half an hour. In business terms, a workload that fires once an hour used to ride the cache all day. On GPT-5.6 it never hits at all.

Filling the cache is its own bill. On the first call of each GPT-5.6 run, the response came back with a field our GPT-5.5 calls never returned: cache_write_tokens. It read 2,029 in automatic mode and 2,001 in explicit mode. OpenAI's pricing page charges cache writes at $1.25 per million tokens against $1.00 for plain input. Plainly: the call that loads your instruction block into the cache costs 25% more than an ordinary call would have. We did not test whether GPT-5.5 bills the same thing without reporting it, so read this as "5.6 makes the write visible and prices it," not as proof the charge is brand new.

The explicit marker buys certainty, not volume. Marking the boundary yourself reused 2,001 tokens instead of 2,029. That is 28 tokens less, about 1.4% of the prompt, and at GPT-5.6 Luna's list price the gap is a rounding error. What you get in exchange is a cached section that is identical on every call because you defined its edge, instead of one the system redraws for you. For anything you have to forecast or explain to a finance team, that predictability is the point.

What this means for your business

Here is the arithmetic that decides whether the upgrade helps or hurts you. Using OpenAI's published prices for GPT-5.6 Luna ($1.00 per million tokens sent, $0.10 per million reused, $1.25 per million written to cache) and our measured 2,029-token block, per 1,000 calls that share that block:

Traffic pattern What happens Cost per 1,000 calls Versus no caching
Busy: several calls every half hour One write, the rest reused about $0.21 roughly 90% cheaper
Quiet: calls more than 30 minutes apart Every call misses and pays to write about $2.54 roughly 25% more expensive
No caching engaged at all Plain input every time about $2.03 baseline

Those are modelled numbers, not invoices. The quiet row assumes a miss still pays to write, which is what the first call of every run did in our test. We never let a cache actually expire and watched what happened next.

The break-even is low. Reuse the same block twice inside a 30-minute window and you're already ahead. But the downside case is real, and it's the one nobody puts in a launch post: a low-traffic service on GPT-5.6 can pay a quarter more than before, for a feature sold as a discount.

So the one thing to do differently after reading this: before you migrate, plot the gaps between your calls that share a prefix. If most gaps are under 30 minutes, turn the explicit marker on. If most are over, treat caching as a cost you have to justify, not a saving you get for free.

Scale that up. A support assistant on steady daytime volume, sharing a 2,000-token policy block, saves close to 90% of its instruction cost. An internal report generator that runs eleven times a day across office hours saves nothing and pays the write premium on every run. Same feature, opposite outcome. The variable isn't your prompt, it's your traffic shape.

There's a migration trap here too. Carry your old prompt_cache_retention: "24h" setting from GPT-5.5 across to 5.6 and it won't do what it used to. Anyone who budgeted for a day-long cache needs to redo that number against half an hour.

Can this survive your workflow?

Run your own traffic through the same question before you migrate. The pattern to check is simple: how often does the same unchanging prefix get sent, and are those sends closer together than half an hour?

  • Customer support and ticket triage. Usually a clean win. Steady volume, one shared policy block, cache stays warm through the working day.
  • Order and billing automation. Depends entirely on volume. A few hundred orders an hour caches beautifully. A few dozen a day pays the write premium on nearly every call.
  • CRM enrichment and batch back-office jobs. Group them. Scattered across the day they lose; run as one batch and they win.
  • Overnight or weekly reporting. The 30-minute window will not carry you between runs. Assume no cache benefit and check whether the write charge is quietly costing you extra.
  • Anything with zero-data-retention. Extended caching behaves differently for ZDR organizations, so verify against your own account settings rather than the default documentation.

If you want this measured against your real workload instead of a synthetic handbook, that is the kind of claim-bound run Effloow's Proof Studio exists to produce, and you can see the services side for scope.

When to use it, when to skip it

Switch the explicit marker on when: you have a stable prefix over 1,024 tokens, your traffic keeps it warm, and you need the cached region to be the same on every call so your cost forecast holds.

Skip it when: your prompts are short, your prefix changes per user, or your calls are spread thinner than one every 30 minutes. In the last case the honest move is to measure whether caching is costing you money before you optimize anything else.

Do not assume it carried over when: you are moving from GPT-5.5 or earlier. The setting name changed, the maximum lifetime shrank, and a new charge appeared. Three separate things to re-check.

Honest limitations of this test

One prefix, one account, one region, four calls per configuration, all on a single day. That's enough to show whether the cache engages, what the API accepts and rejects, and which new fields appear. It isn't enough to characterise eviction, routing across machines, or hit rates under production concurrency.

We tested gpt-5.6-luna only. Sol and Terra were never exercised. The pricing structure looks the same shape across all three, but we did not verify their cache behaviour.

Our repeat calls sat about two seconds apart, so nothing here tests the 30-minute boundary directly. The rejection message is our evidence that 30 minutes is the ceiling. We never watched a cache expire.

Finally, every dollar figure above is arithmetic over OpenAI's published price table and our measured token counts. No billing statement was read. No dollar amount was directly measured. Treat the percentages as the shape of the economics, then confirm against your own invoice.

What Effloow added

The changelog says "explicit prompt caching controls" and stops there. What is not in any vendor document is a side-by-side measurement of how much of the same prompt each mode actually reuses, the rejection that proves the 24-hour window is gone on GPT-5.6, and the break-even table showing the traffic pattern where the feature turns into a 25% surcharge. Those three came from our own run and are reproducible from the public lab note.

For related measured work, see our earlier run on the 24-hour prompt cache on GPT-5.5 and the broader token optimization guide for production LLM systems.

FAQ

Q: Can I still get a 24-hour prompt cache on GPT-5.6?

We did not test prompt_cache_retention against GPT-5.6, so we can't tell you whether the old parameter is still accepted there. We can tell you the thing that actually matters for your bill: on GPT-5.6, prompt_cache_options.ttl took 30m and rejected 24h with an HTTP 400. Whatever the old parameter does on 5.6, a day-long cache is not on the menu.

Q: Why did the explicit mode cache fewer tokens than automatic mode?

Because we told it where to stop. The marker sat at the end of the handbook block, so the cached region ended there. Automatic mode extended a little further into the request. The difference was 28 tokens, 1.4% of the prompt.

Q: Is caching ever worse than not caching?

By the price table, yes. A cache write bills at 1.25 times the normal input rate, so if nearly every call misses, you pay a 25% premium for a feature that never pays back. Low-traffic services should check this before enabling anything.

Q: What is the minimum prompt size for caching to engage?

OpenAI documents 1,024 tokens. Below that, the reused-token count comes back as zero. Our prefix was 2,032 tokens, which is why every configuration engaged the cache on the second call.

For your engineers

Exact configuration from the run:

  • Endpoint: POST https://api.openai.com/v1/responses
  • Models: gpt-5.6-luna, gpt-5.5-2026-04-23
  • Prefix: 9,208 characters of synthetic text, SHA-256 80761fcc454ee750..., measured at 2,032 input tokens
  • max_output_tokens: 48, 4 calls per configuration, ~2s apart
  • Cache fields read from usage.input_tokens_details: cached_tokens, cache_write_tokens

Reproduce:

python3 scripts/gpt56-explicit-cache-probe.py
Enter fullscreen mode Exit fullscreen mode

Explicit-mode request shape (trimmed):

{
  "model": "gpt-5.6-luna",
  "prompt_cache_key": "effloow-cache-probe-explicit-v1",
  "prompt_cache_options": { "mode": "explicit" },
  "input": [
    { "role": "developer",
      "content": [ { "type": "input_text", "text": "<prefix>",
                     "prompt_cache_breakpoint": { "mode": "explicit" } } ] },
    { "role": "user",
      "content": [ { "type": "input_text", "text": "<question>" } ] }
  ]
}
Enter fullscreen mode Exit fullscreen mode

The rejected TTL call, verbatim:

{
  "error": {
    "message": "Invalid value: '24h'. Supported values are: '30m'.",
    "type": "invalid_request_error",
    "param": "prompt_cache_options.ttl",
    "code": "invalid_value"
  }
}
Enter fullscreen mode Exit fullscreen mode

Note that prompt_cache_key matters more on GPT-5.6 than it did before. OpenAI's caching guide describes it as required for reliable matching on 5.6 and later, and recommends keeping traffic on a single key to roughly 15 requests per minute. We used a distinct key per configuration so the three runs could not contaminate each other.

Evidence: public lab note, with the machine-readable artifact saved alongside it in the repository.

Sources

Top comments (0)