If you run Claude Code or Cline daily, prompt caching controls your API spend. Anthropic charges 1.25x base price to write a cache checkpoint and 0.1x to read it, yielding up to a 90% discount on large contexts. While building upstream routing for our multi-model gateway, I audited over a dozen third-party relays. Many claim caching support, but break it via flawed architecture or forge telemetry. Here are the four most common traps, the break-even math, and a test script.
The Naive Key-Pooling Trap
Anthropic caches context per upstream account with a 5-minute TTL. Low-cost relays pool dozens of keys using round-robin routing. Request 1 hits key A (+25% write fee). Request 2 routes to key B. Key B has no cache, triggering another +25% write. Without session affinity, you pay 125% on every turn instead of saving 90%.
Stripping Ephemeral Cache Control
Some resellers advertise 50% off while claiming full compatibility. Behind the scenes, their proxies strip the cache_control block before forwarding to cheap non-caching channels. They then charge standard uncached rates, pocketing an unearned cache-creation markup.
Fabricated Usage Telemetry
Proxies control the returned JSON. Some inspect prompt prefixes with memory hashes. When they detect repeated prefixes, they inject fake numbers into cache_read_input_tokens in the usage object. Your CLI displays a green cache hit, but the backend bills at full input rates.
Dynamic System Prompt Injections
Anthropic requires an exact byte-for-byte prefix match. Some middleman gateways inject internal tracking tags, safety prompts, or dynamic timestamps into the system prompt. Even one altered byte invalidates the cache breakpoint, forcing a complete context re-read.
The Break-Even Reality Check
Write costs 1.25, read costs 0.10, and uncached is 1.0. If your cache hit rate is H, average cost per turn is 1.25 * (1 - H) + 0.10 * H. Setting this equal to 1.0 gives a break-even hit rate of 21.7%. If proxy routing instability drops your cache hit rate below 22%, prompt caching costs you more than keeping it disabled.
How to Audit Your Provider
Send two sequential requests two seconds apart with a static block above 1024 tokens:
Request 1 returns cache_creation_input_tokens > 1024.
Request 2 returns cache_read_input_tokens > 1024 with lower latency.
Check your dashboard to verify request 2 was billed at 0.1x, not standard input rates.
Python Audit Script:
import time, requests
URL = "https://your-provider-endpoint.com/v1/messages"
KEY = "YOUR_API_KEY_HERE"
MODEL = "claude-sonnet-4-6"
prefix = "System Ref: " + ("Alpha Beta Gamma Delta " * 200)
def probe(q):
t0 = time.time()
r = requests.post(URL, headers={"x-api-key": KEY, "anthropic-version": "2023-06-01"},
json={"model": MODEL, "max_tokens": 50,
"system": [{"type": "text", "text": prefix, "cache_control": {"type": "ephemeral"}}],
"messages": [{"role": "user", "content": q}]}).json()
print(str(round(time.time() - t0, 2)) + "s | " + str(r.get("usage")))
print("--- Turn 1: Expect cache_creation ---")
probe("List 1")
time.sleep(2)
print("--- Turn 2: Expect cache_read & fast latency ---")
probe("List 2")
Checklist for Any Third-Party Gateway
Demand per-request transparency. If a platform hides request IDs, upstream route switches, or granular cache read and write counters, your balance is vulnerable. Start with a small deposit ($1 to $5) and verify your usage logs before scaling up.
Disclosure: I got tired of encountering these routing problems, so we built API Route — an OpenAI and Anthropic compatible routing gateway with session stickiness and granular log inspection. We also recently got listed in the official Docker Agent Docs and Dify Marketplace. Happy to answer any questions about proxy architecture below!
Top comments (0)