What I Learned Cutting Claude Code's Token Bill by 77%

What building a profiler for AI coding agents taught me about the hidden river of data flowing to Claude on every turn, and how to cut most of it without changing a single result.

Dashboard live demo · Source
(The live dashboard profiles a real DeepSeek session — the plumbing is identical for Claude Code.)

⚠️ Early results. Single-sample runs on one fixture (Claude Code + Opus 4.6 on Bedrock). The direction is strong and reproducible; treat individual percentages as ballpark. Caveats are at the end, that honesty is part of the point.

---

You can't fix what you can't see

An AI coding agent feels like magic: ask it to fix a bug, it reads files, writes code, runs tests, reports back. Underneath, it's just making HTTP requests to a model, over and over — traffic you never see. I built the AI Agent Profiler (aap): a transparent proxy that records every request byte-for-byte and passes it through untouched. The first thing that jumps out is the volume.

The model has no memory

Every call to the model is stateless — it remembers nothing between turns. So the agent re-sends the entire history, from the top, on every turn. Turn 30 doesn't send turn 30; it sends turns 1–30. Again. It's not a chat, it's a snowball:

[ System:  "You are Claude Code. Here are your 15 tools…" ]  ~5,000 tok — re-sent every turn
[ Tools:   full JSON schemas ]                               ~7,000 tok — re-sent every turn
[ User:    "fix the failing tests" ]
[ Tool:    <full 2 KB of scheduler.js> ]                     read once — re-sent forever
[ Tool:    <200 lines of test output> ]
   … and on, and on …

Most of it is dead weight: a file you read and edited ten turns ago is still mailed on every later turn; 15 tool schemas ride along when the agent uses three. That's why long sessions get expensive, not the model "thinking hard," but re-mailing stale paperwork thousands of times. One 70-request session I measured pushed ~3.9M input tokens while the model wrote back a few thousand words.

How Anthropic's prompt cache works

Anthropic softens this with prompt caching — and it's an explicit model you control. You mark where a cacheable prefix ends with a cache_control breakpoint:

{ "type": "text", "text": "You are Claude Code…", "cache_control": { "type": "ephemeral" } }

What matters:

• Up to 4 breakpoints per request.
• Content before a breakpoint is cached ~5 min; a cache read is ~10× cheaper than uncached input.
• It's a prefix match — change one byte early and everything after it is billed as new.

Pricing (Opus 4.6 / Bedrock; verified against anthropic.com/pricing on 12 Jul 2026 — rates change): cache read $0.50/MTok vs input $5.00 vs output $25.00.

Claude Code already places its 4 markers well and hits ~99–100% cache. So the real question isn't "turn caching on" — it's how much of that cached snowball can I stop carrying without breaking the prefix?

Trimming the pile in-flight

Because the profiler sits in the middle of every request, it can do more than watch. With the optimize layer on, it tidies the outgoing pile just before each request leaves, never touching the reply, never changing what the agent does next. Biggest levers first:

• pruneStale (~57% of the savings). A file read fifteen turns ago and since edited becomes a one-line stub: [read scheduler.js earlier — 47 lines]. The model already acted on it.
• pruneUnusedTools (~23%). After ten turns, drop the schemas of tools never called (often 12 of 15).
• insertBreakpoints. After trimming, re-anchor Anthropic's 4 markers so the surviving prefix still cache-hits.
• Smaller ones: dedup repeat reads, truncate giant blobs, skip needless re-reads.

(A fourth lever, collapseSystem, stubs the repeated system prompt — see the caveats; I'm deliberately not leaning on it.)

The result: ~77% cheaper, identical output

To get real numbers I used a fixed benchmark fixture: a small JavaScript project with 9 planted bugs and 3 stubbed-out methods, ~54 visible tests, plus a set of hidden edge-case tests the agent never sees (used to grade quality afterwards). The task handed to Claude Code was blunt — fix everything and make the tests pass. I ran that identical task through the profiler three ways:

Run                    Reqs   Input tok   Cache hit   Cost    vs baseline
──────────────────────────────────────────────────────────────────────────
No optimization          70   3,917,198     100%     $2.04       —
Full optimization        94     776,366     100%     $0.48     −77%
"Cache-protective"*     118   4,978,066     100%     $2.67     +31% WORSE

* don't-prune-anything, to "protect" the cache — the worst result (more below).

Quality was identical across runs: 54/54 fixture tests, same 54/57 edge tests. The optimizer changed the cost, not the outcome.

The counter-intuitive part: pruning does NOT break the cache

The naive fear — "editing old messages shatters the prefix and costs more", is backwards here, and not because misses were outweighed by volume. The winning run held 100% hit with only 92 uncached tokens: pruning kept the hit rate and shrank the volume, because insertBreakpoints re-anchors the markers after the edit. Cached tokens carried per request fell from ~4.3M to ~776K.

Honest note, simulation vs reality. My offline simulator (a strict byte-prefix model) predicted pruning would tank the hit rate to ~67%. The live Bedrock runs held ~100%. Anthropic's real cache tolerates in-region edits far better than a naive model assumes; where they disagreed, I trusted the live runs.

So "protect the cache by not pruning" is exactly wrong on Anthropic-format traffic: carrying 4.3M cached tokens every turn costs more than carrying 776K — even though both are "cached."

Side note: it did not port to DeepSeek

I tried the same layer on DeepSeek (OpenAI chat format) and got the opposite result — the identical pruneStale edit shattered its automatic prefix cache and blew cost up. Different cache, different rules: mutating old context is safe-ish on Anthropic's explicit breakpoints and toxic on a pure prefix cache. I built a separate cache-safe DeepSeek profile instead. Full cross-provider story and data are in the repo. The lesson worth keeping: there's no universal "make it cheaper" button — the right one is provider-aware.

Caveats — read before quoting a number

• Single samples, one fixture. One-shot runs on one JavaScript task; agents are non-deterministic (baseline swung run-to-run). Directions held; treat percentages as ballpark.
• collapseSystem is unverified. It stubs the ~5K-token system prompt after turn 1. No effect on this fixture, but "you can drop the system prompt safely" is a claim I'm not making broadly — hence it's out of the headline numbers.
• Nobody fully solved the task. Even the best runs left 3/57 edge tests failing — ordinary agent bugs, identical with pruning on or off.
• Tooling honesty. Some scores were gathered by hand while two harness bugs were fixed. A profiler that hides its own measurement bugs isn't worth much.

Try it

Open source (MIT). In read-only mode it just shows you the firehose — how big the pile is, which files get re-read, where the money goes:

aap serve             # the meter on the pipe
aap run claude        # launch the agent through it
aap serve --optimize  # …and trim the pile (measure on your own provider first!)

• Browse real captured sessions in your browser: https://rguiu.github.io/ai-agent-profiler/
• Code + full benchmark reports: https://github.com/rguiu/ai-agent-profiler

The interesting question turned out not to be "how much can I save?" — it's "what is my agent actually sending, and why is it sending it a thousand times?"