I audited six token usage trackers. They disagree with each other by 2x to 8x.

Over the past five months I took apart the folding logic in six token usage trackers and recomputed each one with an independent implementation.

They don't agree. The gaps run from 2.00× to 8.09×, in both directions. Ten fixes have landed so far across five repositories.

The second finding matters more: none of these tools knows whether its own number is right. There is nothing for them to check against.

Here is each gap, why it happens, and how to check the one you use.

Disclosure first

Two of the issues below are my own submissions. Saying so up front, rather than letting you find out by clicking the link:

• tokscale issue #1011 (filed 2026-08-03). The parser fix landed three days later. It's still open for a different reason — see mechanism 4.
• claude-code-templates PR #754 (filed 2026-07-25, unmerged as of Aug 18). Checks green, no conflicts, no maintainer response yet; the thread is me and the bots.

For the second one, "unmerged" means "not picked up yet," not maintainer negligence. That's a 30k-star repo with a long queue.

How this started

I wasn't looking for bugs. I was auditing my own session logs and happened to read the same files with two tools. The numbers didn't match.

A small gap I would have written off as rounding. This was close to double, so I looked properly.

There's precedent for this, and it went well. In tokscale issue #76, johnpyp reported tokscale and ccusage differing by close to 2×. Maintainer junhoyeo labelled it a bug within twenty minutes, closed it within five hours, and traced the root cause to Claude Code upstream duplicating session history under stream-json (anthropics/claude-code#5034). Not a tokscale bug.

That case makes the point I want to make: when a discrepancy is large enough to see, it gets fixed, and fixed fast. The dangerous ones are the ones nobody can see.

They can't be seen because there is no third number. Subscription users especially. You pay a flat monthly fee, so Anthropic never sends you a per-token invoice. Claude Code's own dollar figure comes with a plain warning in the docs: usage for Max and Pro subscribers is included in the subscription, so that figure "isn't relevant for billing purposes."

So this whole category is a competition with no referee.

Five mechanisms

Every discrepancy I found falls into one of five buckets. Each one below has a measured number and the repo it came from.

1. The same usage is written to the log twice

The most common one, and the easiest to reproduce.

A streaming response emits a usage chunk, then assembles a complete message when the request finishes. Both carry the same usage. Sum everything you see and you get exactly 2×.

On the deepseek-harness corpus, naive folding came out at 2.000000× of the official projection. Two provider routes, computed separately, agreeing to six decimal places. Not approximately two. Two.

The same mechanism wears different clothes elsewhere:

• claude-code-templates (30,276 stars) counts once per content block. Measured 2.36×. My fix PR #754 is still unmerged as of Aug 18.
• Clawdmeter had the same per-content-block duplication. Measured 2.34–2.37×. The author fixed it in v3.0.1 on 2026-08-08, titled the release "token counts corrected (~2.5x lower)," and noted that the percentage progress bar reads Anthropic's rate-limit header and was never affected. That's candid handling and it deserves saying out loud.

Three projects, three implementations, one hole.

2. A forked child session physically contains its parent

This one hides better.

Sessions can fork, and the child's log file physically contains the parent's full prefix. If your aggregation walks every session file and sums, that prefix gets counted twice.

The two forks I caught were inflated 5.18× and 23.05×.

23× is not a typo. The later the fork point, the longer the inherited prefix and the shorter the child's own contribution, so the ratio gets worse. At the limit you are reporting an entire parent session as the cost of a child that did almost nothing.

The test has to be seq >= seedLength. Checking origin === 'subagent' alone is not enough, because an ordinary user-created fork inherits a prefix too.

3. Some model calls aren't in scope at all

Context compaction is a real model call and the provider reports usage for it. But it isn't a loop step, so it produces neither an assistant chunk nor an assistant message. Most folding logic matches only those two event types, so this cost was never counted at all.

Measured: 3 compaction events, 48,895 tokens, none of them counted. The largest summarize call reported 44,444 tokens (41,472 of them cache reads) to replace a history range of 19,962 tokens.

This omission has an unpleasant property. Compaction fires most often on long sessions, which are exactly the sessions where you most need the cost to be right.

4. Estimates and measurements share one field

In tokscale, the input field adds a character-count estimate of tool_result content (chars.div_ceil(4)) on top of the API-reported input_tokens. Claude Code never writes token metadata on tool_result blocks, so the fallback fires on every one of them. Measured before the fix: 87.6% of input values were the estimate, 8.09× inflation.

This is the best-handled case in the set, and it's worth telling in full.

I filed issue #1011 on 2026-08-03. Three days later junhoyeo merged the parser fix (#1037) and shipped v4.11.0, writing in the thread that "your measurement is what drove this." He deliberately left the issue open, because the correction isn't retroactive — already-cached totals stay inflated, and he laid out why bumping parser_version would be worse: it would discard retained assistant turns that compacted transcripts no longer contain.

What happened next is the part I'd point at. He designed a cache migration, then overturned his own design: an adversarial review of his plan showed the migration wasn't needed at all, because the provenance rebuild already clears stale estimates as a side effect. The blocker wasn't the design, it was that the commit hadn't been cut into a release. His words: "the remedy is a release, not a migration." He also apologised, unprompted, for a nine-day gap — "That's on me."

I re-measured on main (f169a1f0): input went from 10.10× to 1.00×, clearing 17,109,737 tokens, while output, cacheRead, cacheWrite and messageCount held identical. That second half is the load-bearing part — it says the estimate was recomputed away rather than records being retired.

So #1011 is still open, but not because nobody acted. Two things remain: the fix hasn't been cut into a release tag, and server-side stored submissions stay inflated because the merge guard refuses reductions.

Character estimation is fine on its own. The problem was that the estimate and the measurement were merged into one field and looked identical in the UI. A user had no way to tell whether the number in front of them was counted or guessed. That isn't an accuracy problem, it's a labelling problem. If a number contains an estimate, the interface should say so.

5. History gets rewritten silently

After resume and compact, historical totals drift. Nothing errors. The numbers just change.

I'm not putting a multiplier on this one, because it isn't a fixed multiplier — it depends on how many times you resumed. The mechanism is real; the direction varies.

Why nobody catches this

Because there is no ground truth.

Every tool compares its output to its own expectations. The person writing the tests and the person writing the implementation are the same person with the same mental model, so a passing test proves the code matches the author's understanding. It proves nothing about whether the author's understanding matches the log.

viberank's trust model shows the shape of the situation. It labels submissions verified, and that label is about identity — you are who your GitHub account says you are. The numbers themselves go through a separate set of server-side heuristics: token totals, cost-per-token ratio, date plausibility, and a review flag for outliers. Both things are done. They're just easy to read as one thing in the UI. It never claimed to verify the data, and I don't think this is deceptive — it's what happens when a category has no third-party baseline.

tokscale's README has a line that describes the situation precisely. Writing about subscription quotas, it says those numbers are vendor-reported and "are not independently verified against tokscale's own usage tracking." That sentence is about quotas rather than about its own token counting, but the structure it describes applies at every layer: each layer relays the layer above it, and no layer checks.

This is not a failing of these authors. The category is missing something to check against.

The method

Four steps. None of them needs a vendor account.

Build a synthetic corpus with hand-computed ground truth. Not "run it once and save the output as expected" — that records current behaviour, it doesn't test it. Work out on paper what the answer should be, then make the code match.

Make the probe check itself first. Before touching real data, run --self-test against the hand-computed fixture. If a fold disagrees with the fixture, that's a bug in the probe, and no number leaves the machine.

Import the subject's own entry point. Don't reimplement from its README — then you're testing the docs, not the code. For deepseek-harness I imported usage-projection.ts directly and ran it over a real log.

Get two implementations that share no code to produce the same number. This is the only thing that turns "I think this is wrong" into "this is wrong." In that case an independent fold and the official patch came out with zero difference across four buckets and two scopes. Past that point it stops being a matter of opinion.

The probe is stdlib-only Python, no dependencies, one command. Session logs stay out of the repository; only measurements get published.

What to check in your own tool

These are format-independent. They apply to any metering tool:

1. Can one usage event be written to two records? If so, what's your dedup key?
2. Does a child session's log contain its parent's content? Is your aggregation by file or by ownership?
3. Are there model calls outside the main loop? Compaction, summarization, title generation, search — those cost money too.
4. For every number in the UI: measured or estimated? Mixing them is lying to yourself.
5. Missing is not zero. No pricing model is an error, not $0. No sample is unknown, not free.
6. Don't stack your own estimate on top of a number the API returned. Either trust it or label it.

5 and 6 are the ones I've gotten wrong most often.

Closing

The point isn't the bugs. Most of these authors fixed things quickly: Clawdmeter shipped a fix in one release, tokscale merged a parser fix in three days and then overturned its own follow-up design. The two projects I wrote about at greatest length are the two that responded most seriously — that isn't a coincidence. People who engage leave a process worth writing down.

The point is that a number everyone uses and nobody can verify is a bad number, even when it happens to be correct.

These tools are being used right now for team cost allocation, subscription tier decisions, and public leaderboards. Until there's a third-party baseline, all of that rests on a number that vouches for itself.

Probe, protocol, and the full invariant catalog, Apache-2.0:
https://github.com/lizhuojunx86/traceguard/tree/main/usage-tracker-audit

Happy to answer questions. If you maintain one of these tools and want me to run the probe against it, open an issue.