I measured 8 weeks of Claude Code: 87% lower cost, 86% first-try typecheck

Diego Prieri — Tue, 03 Nov 2026 16:20:53 +0000

Everyone has an opinion about coding agents: they're too expensive, or they're magic, or the code they write is garbage. I lead the frontend of a ~120k-line TypeScript app where agents write most of the code, so I wanted numbers instead of opinions.

It turns out the data was already on my machine.

The data Claude Code already keeps

Claude Code writes every session to ~/.claude/projects/**/*.jsonl. Each line is an event, and two kinds of event matter here:

Assistant responses include a usage block with token counts: uncached input, cache writes (split by 5-minute and 1-hour TTL), cache reads and output.
Tool calls include every Edit, Write and Bash command the agent ran, with its output.

That's enough to answer two questions:

Is prompt caching actually working, and what does it do to cost?
When the agent edits code, how often does the first typecheck or test run pass?

I wrote a small CLI, claude-code-metrics, that reads those files and answers both. Nothing leaves your machine.

npx tsx src/cli.ts                     # everything
npx tsx src/cli.ts --since 2026-10-01  # a period
npx tsx src/cli.ts --project my-app    # sessions whose cwd contains "my-app"

Result 1: prompt caching

Over 8 weeks and 7.5B input tokens:

	Value
Input tokens read from cache	98.4%
Input cost vs. no caching	87.4% lower
Responses	30,439

How the savings are computed

Prompt caching changes the price of each input token, depending on where it comes from:

Token type	Price vs. base input
Uncached input	1×
Cache write, 5-minute TTL	1.25×
Cache write, 1-hour TTL	2×
Cache read	0.1×

So the savings are:

savings = 1 − actual / uncached
actual   = input×1 + write5m×1.25 + write1h×2 + read×0.1
uncached = (input + write5m + write1h + read) × 1

The base price cancels out, so the ratio holds for any model. In code it's a few lines:

export const MULTIPLIERS = { input: 1, write5m: 1.25, write1h: 2, read: 0.1 };

export function ratios(t: TokenTotals): CacheRatios {
  const uncached = t.input + t.write5m + t.write1h + t.read;
  if (!uncached) return { readShare: 0, savings: 0 };
  const actual =
    t.input * MULTIPLIERS.input +
    t.write5m * MULTIPLIERS.write5m +
    t.write1h * MULTIPLIERS.write1h +
    t.read * MULTIPLIERS.read;
  return { readShare: t.read / uncached, savings: 1 - actual / uncached };
}

One gotcha: Claude Code logs one line per content block, all with the same message id. If you don't deduplicate by id, you count the same response several times.

Why the number is so high

An agent session re-sends a long, mostly stable prefix on every turn: system prompt, tools, project instructions, the conversation so far. Only the tail changes. That's the ideal case for caching: you pay to write the prefix once, then read it at 10% of the price for the rest of the session.

Update (Nov 3): a reader pointed out that edits made from Bash (sed -i, cat >, tee) didn’t arm a check. They do now, and attempts went from 503 to 1,942. Updated rates: 84.8% typecheck and 76.2% tests on the first run; 86.5% of tests pass on unfiltered runs only (n=89). Thanks for the catch!

Result 2: first-try pass rate

This is the number I care about most, because it measures how good the agent's first attempt is, not how good it gets after five retries.

Check	Attempts	Passed on first run
Typecheck	205	86.3%
Tests	298	75.2%

How "first try" is defined

Each session is replayed in order:

An edit (Write, Edit, MultiEdit) to a matching file arms a check: .ts/.tsx/.vue arms typecheck, code files arm tests.
The next Bash command that runs that check (tsc, jest, vitest, npm test, …) is the first try. Its output decides pass or fail (error TS…, N failed, not ok).
Reruns before the next edit are ignored. They measure iteration, not the first attempt.

The caveat that matters most

Agents love to pipe long output through head, grep or tail, and a filtered output can hide the failing line. So the CLI also reports the rate on unfiltered runs only:

Check	Unfiltered runs	Pass rate
Typecheck	10	100%
Tests	21	52.4%

The samples are small, but the test number is a warning: treat 75% as an upper bound. This is also why, in my workflow, the agent never commits without a gate that re-runs typecheck and tests with full output.

Other caveats

Subscription plans aren't billed per token. There, the 87% means "what these tokens would have cost on the API": a measure of efficiency, not money saved.
This is observation, not an eval. Tasks aren't controlled. To compare prompts or context, you need a harness that runs the same tasks under each variant. That's what I built agent-context-evals for.
Pre-existing failures make every attempt look like a failure. Measure on a clean baseline.
Transcripts get cleaned up over time, so the period is whatever is still on disk.

What I took away

Caching is not something to tune. It's something not to break. Keep the prefix stable (instructions, tool definitions, project context) and the savings come on their own.
Agents are good, not reliable. 86% on the first typecheck is impressive. The other 14% is why verification gates aren't optional.
Measure your own workflow. Your numbers will be different from mine, and the data is already sitting on your disk.

The code is MIT-licensed: github.com/diego-prieri/claude-code-metrics. I'd love to see other people's numbers. If you run it, share yours in the comments.

DEV Community: Diego Prieri