DEV Community

Discussion on: We said a Claude Code subagent costs 436k tokens. A cleaner measurement says 54k — here is what fooled us

Collapse
 
vinhnguyenthanhdn profile image
Vinh Nguyen

Worth pinning one thing down before the probe becomes a standing regression test: cache_creation_input_tokens is only populated on a cold prefix. Run the probe a second time inside the cache TTL with the instruction files unchanged and the same 54k shows up under cache_read_input_tokens with creation near zero, so a check that reads only the creation field reports that spawn cost collapsed precisely when nothing changed. Summing creation plus read, and recording which one was non-zero, keeps the number comparable across runs and makes a real drop distinguishable from a warm cache. Otherwise the replacement inherits the failure of the number it replaced: right value, wrong noun attached to it.

Collapse
 
rulestack profile image
Rulestack

One reviewer's first request in a different session came in at cache_creation 2,971 and cache_read 90,689 — a creation-only check would have logged that as spawn cost falling to 3k, on the largest prefix I've looked at. So the check you're describing is the one to build, and there isn't one yet: the probe ran once, cold, and nothing re-runs it. When it does become a check it'll record creation + read plus which field carried it, so a warm run and a cheap run stop looking alike in the log. Though the naming is mine to fix first — I've been calling it 'spawn cost' when the number is prefix size plus whether the prefix was warm.

Collapse
 
vinhnguyenthanhdn profile image
Vinh Nguyen

Recording both fields fixes the token count, but summing them still hides a price gap that is larger than the measurement error you were chasing. Cache creation bills at 1.25x base input on the 5-minute TTL (2x on the one-hour), and cache reads bill at 0.1x. Your reviewer run is the clean case for it: 2,971 creation plus 90,689 read is about 93.7k tokens either way, but weighted that is roughly 12.8k base-input-equivalent, against about 117k if the same prefix had arrived cold. Same summed tokens, about 9x apart in what it actually costs, so the check probably wants the weighted number sitting next to the raw one rather than the raw one alone.

Thread Thread
 
rulestack profile image
Rulestack

You're right, and the 9x spread deserves to be printed. The summed count answered the question we were asking — was 436k a measurement artifact — but as a cost number it treats a 0.1x cache read like base input. Weighted base-input-equivalent next to the raw sum is the honest presentation, because the two diverge exactly when caching is doing its job.