Stop Paying for the Same Tokens Twice: A Practical Guide to Prompt Caching

You've built a chatbot. Every turn, you re-send the whole conversation — the 8,000-token system prompt, the uploaded PDF, the 15 messages of history — just so the model can answer "and what about Mars?"
The model re-reads all of it. Every. Single. Time. You pay full price for all of it. Every. Single. Time.
Prompt caching fixes this. It's roughly one extra line of JSON, and it can cut your input costs by ~90% on the repeated part while making responses noticeably faster.

Let's walk through it.

The one-liner version

Add cache_control at the top level of your request:

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},   # <-- this is the whole trick
    system="You are an AI assistant tasked with analyzing literary works...",
    messages=[
        {"role": "user", "content": "Analyze the major themes in 'Pride and Prejudice'."}
    ],
)
print(response.usage)

That's automatic caching. The API caches everything up to and including the last cacheable block in your request. Next time you send a request that starts with the same content, that prefix is read from cache instead of reprocessed.
Same thing in curl, if that's more your speed:

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-5",
    "max_tokens": 1024,
    "cache_control": {"type": "ephemeral"},
    "system": "You are an AI assistant tasked with analyzing literary works.",
    "messages": [{"role": "user", "content": "Analyze the major themes in Pride and Prejudice."}]
  }'

---

The mental model: it's a prefix cache

This is the single most important thing to internalize.
Your prompt is read in a fixed order:

tools  →  system  →  messages

Caching works on prefixes of that sequence. When you mark a block with cache_control, you're saying: "cache everything from the start of the request up to and including this block."
Which means:

• ✅ Static stuff at the front = cacheable.
• ❌ Change something early = everything after it is invalidated. So the golden rule is: put the boring, unchanging stuff first. Tool definitions, system instructions, that 50-page contract, your 20 few-shot examples. Then the volatile stuff — the user's actual question — goes last. ---

What it costs (and saves)

Three price tiers instead of one:

	Multiplier vs. base input
5-minute cache write	1.25×
1-hour cache write	2×
Cache read (hit)	0.1×

So for Claude Opus 5 ($5/MTok input):

• First request writes the cache: $6.25/MTok
• Every subsequent hit: $0.50/MTok You pay a 25% premium once, then 90% off forever after. If you reuse a prefix even twice, you're already ahead. Cache breakpoints themselves are free. You're never charged for having a breakpoint — only for tokens actually written and read.

---

Reading the usage fields (this trips everyone up)

"usage": {
  "cache_read_input_tokens": 100000,
  "cache_creation_input_tokens": 0,
  "input_tokens": 50
}

input_tokens is not your total input. It's only the tokens after your last cache breakpoint. Total is:

total = cache_read_input_tokens + cache_creation_input_tokens + input_tokens

So the example above processed 100,050 tokens, not 50. Handy side effect: cache hits don't count against your rate limits the way fresh input does, so effective throughput goes up too.

Quick sanity check: if both cache_creation_input_tokens and cache_read_input_tokens are 0, nothing was cached. Most likely you're under the minimum length (see below) — the API won't error, it just silently skips caching.

---

Minimum sizes — don't skip this

Prompts shorter than a per-model floor simply won't cache:

Model	Minimum cacheable tokens
Opus 5, Fable 5, Mythos 5	512
Opus 4.8, Sonnet 5 / 4.6 / 4.5	1,024
Opus 4.7, Haiku 3.5	2,048
Opus 4.6, Opus 4.5, Haiku 4.5	4,096

If you're just under the line, it's often worth padding the cached section (more examples, more context) to get over it. Cache reads are cheap enough that the extra tokens pay for themselves.

---

Multi-turn conversations: let it drive

With automatic caching, the breakpoint walks forward on its own as the conversation grows:

Request	What happens
1	System + U1 + A1 + **U2** → everything written to cache
2	... + A2 + **U3** → System→U2 read from cache, A2+U3 written
3	... + A3 + **U4** → System→U3 read from cache, A3+U4 written

No bookkeeping. No moving markers around. Each turn reads the whole prior conversation from cache and only writes the new bit. This is the single best default for chat apps.

---

Explicit breakpoints: when you need the wheel

Put cache_control on individual blocks when different parts of your prompt change at different rates. You get up to 4 breakpoints.
Classic RAG-agent layout:

{
  "tools": [ /* ... */, { "name": "get_document", "cache_control": {"type": "ephemeral"} } ],
  "system": [
    { "type": "text", "text": "You are a research assistant...", "cache_control": {"type": "ephemeral"} },
    { "type": "text", "text": "# Knowledge Base\n## Doc 1...", "cache_control": {"type": "ephemeral"} }
  ],
  "messages": [
    { "role": "user", "content": [
      { "type": "text", "text": "Tell me about Perseverance.", "cache_control": {"type": "ephemeral"} }
    ]}
  ]
}

Four independent segments:

1. Tools — basically never change
2. Instructions — change on deploys
3. RAG documents — change daily
4. Conversation — changes every turn Swap the RAG docs and you keep segments 1 and 2. Add a turn and you keep 1, 2, and 3. Changes only invalidate their own segment and everything downstream.

---

The mistake literally everyone makes

Here's the bug I want you to remember, because it's expensive and silent.
Your prompt: blocks 1–5 are a big static system context. Block 6 is f"[{timestamp}] {user_message}". You put cache_control on block 6, because it's the end and that seems right.

• Request 1: cache written at block 6. The hash includes the timestamp.
• Request 2: different timestamp → different hash → miss. The system walks back through blocks 5, 4, 3, 2, 1 looking for entries... but no request ever wrote an entry there.
• Result: a fresh cache write every single request. You pay the 1.25× premium forever and never get a single read. The lookback does not find stable content behind your breakpoint and cache it for you. It only finds entries that earlier requests wrote — and writes happen only at breakpoints. The fix is one line: move cache_control to block 5, the last block that's identical across requests. > Rule of thumb: put the breakpoint on the last block whose prefix is identical across the requests you want to share a cache. (Note: automatic caching falls into the same trap here, since it targets the last block. If your final block has a per-request timestamp, use an explicit breakpoint on the static prefix instead.)

---

The 20-block lookback window

A related gotcha. When looking for a cache hit, the system checks your breakpoint's position and then walks backward — but only 20 blocks.

• Turn 1: 10 blocks, breakpoint at 10. Entry written at 10.
• Turn 2: 15 blocks, breakpoint at 15. Walks back to 10, finds turn 1's entry. Hit! Only blocks 11–15 processed fresh.
• Turn 3: 35 blocks, breakpoint at 35. Checks blocks 35 down to 16, finds nothing. The turn-2 entry at block 15 is one position outside the window. Miss. Full reprocess. If your conversation can jump by 20+ blocks in a single turn, add a second breakpoint further back so a write accumulates there before you need it.

---

The 5-minute vs 1-hour decision

Default TTL is 5 minutes, and it refreshes for free every time you hit the cache. An active chat session basically keeps itself warm.

"cache_control": { "type": "ephemeral", "ttl": "1h" }

Reach for 1h when:

• Follow-ups are likely to land after 5 minutes but within an hour (a user who steps away; an agent sub-task that runs long)
• Latency matters on those delayed follow-ups
• You're batching, where jobs commonly take 5–60 minutes Stick with 5m when your prompt is used more often than every 5 minutes — refreshes are free, so you'd be paying 2× writes for nothing. Mixing TTLs in one request is allowed, with one rule: longer TTLs must come first. 1-hour blocks before 5-minute blocks.

---

Bonus: pre-warm the cache

Latency-sensitive app? The first user of the day eats the cache-miss penalty. Unless you warm it up first:

SYSTEM_PROMPT = [{
    "type": "text",
    "text": "You are an expert software engineer...",
    "cache_control": {"type": "ephemeral"},
}]
def prewarm_cache():
    client.messages.create(
        model="claude-opus-5",
        max_tokens=0,                                  # no output generated
        system=SYSTEM_PROMPT,
        messages=[{"role": "user", "content": "warmup"}],
    )

max_tokens: 0 reads your prompt in, writes the cache at your breakpoint, and returns immediately with an empty content array and stop_reason: "max_tokens". Zero output tokens billed. (You still pay the cache write, naturally.)
Two things to get right:

1. Put the breakpoint on the shared content (your system prompt), not on the "warmup" placeholder — otherwise the entry is keyed to the placeholder and real traffic never hits it. This is why pre-warming needs an explicit breakpoint rather than automatic caching.
2. Use the same thinking config and effort setting as your real requests. Those get rendered into the prompt, so a mismatched pre-warm writes an entry nobody uses. max_tokens: 0 is rejected with stream: true, extended thinking, structured outputs, forced tool_choice, or inside a Batches request.

---

What breaks the cache

Cache hits need a 100% byte-identical prefix. Things that invalidate:

Change	Blast radius
Tool definitions	Everything
Toggling web search / citations	System + messages
Switching fast mode	System + messages
tool_choice	Messages
Adding/removing images anywhere	Messages
Thinking config or effort	Messages (and sometimes more)

One sneaky one: some languages (Go, Swift) randomize map key order when serializing JSON. If your tool_use blocks come out with shuffled keys, your cache never hits and you'll have no idea why. Pin the ordering.
Also: caches are isolated per organization, and per workspace on the Claude API. And a cache entry only becomes available after the first response begins — so firing 10 parallel requests with the same prefix gives you 10 misses. Send one, wait, then fan out.

---

Troubleshooting checklist

Cache not hitting? Run down this list:

• [ ] Is the cached section byte-identical across calls?
• [ ] Are you over the minimum token count for your model?
• [ ] Is the breakpoint on a block that stays the same (no timestamps, no user input)?
• [ ] Are calls landing within the TTL?
• [ ] Are tool_choice, image presence, thinking config, and effort consistent?
• [ ] Is your JSON key order stable?
• [ ] Has a growing conversation pushed you past the 20-block lookback?

---

Got a caching setup that surprised you — good or bad? Drop it in the comments. 🚀