Update, September 5, 2026. Session format v2 landed on September 1 and rewrote the event model these four findings are described in. I migrated the same corpus through it. Every number holds, to the token, and one of them holds for a reason worth writing down.
The commit is f99b06ea, "feat(session)!: embed assistant streams in format v2", September 1 at 20:00Z. It shipped through 0.1.2-alpha.5 and 0.1.2-rc.1; master today is d347e703, 0.1.3-alpha.1.
First, a sentence of mine to retract. In the August 28 update I wrote that usageOf() still matches assistant/chunk and assistant/message. That is wrong as of f99b06ea. It matches assistant/message and assistant/attempt now, and when the event carries no usage field it walks the embedded stream backwards for the last usage chunk.
How I re-measured. I converted the four original sessions with upstream's own sessionFormatV0ToV1 and sessionFormatV1ToV2 at d347e703, which is the same path DSH's persistence uses to open an old log. Both migrations validate their target, so the conversion either produced a v2 artifact upstream considers legal or it refused. Same four sessions, same ground truth, both formats folded by the same code.
One thing blocked it and is worth reporting on its own. The frozen released-v0 reader in the current tree refuses these logs: assistant/chunk 179 chunk replayState has unexpected member "kind". My corpus was written by 0.1.0-rc.6 in August, where replayState is {kind, version, api, provider, model, responseId, stopReason, blocks}. The frozen v0 disposition allows {response, blocks?}. So a current DSH cannot read a session log this old, and the failure is a validation refusal rather than a misread. replayState is an optional member of a finish chunk and carries no usage, so I dropped it from a copy, 72 occurrences across 8,654 lines, and confirmed the v0 fold is bucket-for-bucket identical before and after.
The corpus went from 8,650 events to 303. The number of places a usage number is written went from 75 to 75.
That is the whole result. The migration collapsed 8,350 assistant/chunk events into the settlements that follow them, a 96.5% reduction in log records, and it removed no duplicate at all.
| v0 | v2 | |
|---|---|---|
| events | 8,650 | 303 |
| usage sightings | 75 | 75 |
| naive fold, all events | 1,073,540 | 1,073,540 |
| naive fold, own work only | 550,416 | 550,416 |
| official projection | 536,770 | 536,770 |
| corrected fold | 324,103 | 324,103 |
| compaction portion | 48,895 over 3 events | 48,895 over 3 events |
D-1 survives at exactly 2.000000×, and it moved somewhere harder to see. In v1 the duplication was two events, a usage chunk and then the assembled message. In v2 live.settle() writes exactly one durable settlement per attempt, assistant/message when a surface message exists and assistant/attempt when it does not, in mutually exclusive branches at agent.ts:389, 407, 413, 430, 460. assistant/chunk is not deprecated, it is gone: absent from KNOWN_SESSION_EVENT_TYPES and filtered out of the frozen v2 inventory in session-format-v1-to-v2/src/dispositions.ts.
But the second copy did not go with it. Of the 36 assistant/message events in the migrated corpus, all 36 carry both a usage field and a stream whose last usage chunk holds the same numbers, and all 36 agree exactly. Expand the stream and read .usage and you are at 2.000000× again, now inside one record, where nothing about the file suggests you counted anything twice. The official usageOf() reads .usage first and falls back to the stream only when it is absent, so it takes one. A plugin author who flattens the stream because that is where the chunks used to live, and then adds the field because it is right there, gets the old bug with none of the old evidence for it.
D-2 is the one where I gave advice that no longer runs. I wrote that the only sound discriminator is seedLength. The v2 physical header requires isSeeded and stores no numeric cut, and the current Session package rejects a header carrying seedLength outright: packages/core/session/src/index.ts:98-100, "session header has invalid field seedLength". The cut is the seq of the last session/end-seed { inherited: true } marker, so that marker is the first event of the child's own work and everything before it is the parent's (session-format-v1-to-v2/src/codec.ts:125-136, cross-checked against validation.ts:88-127). I had this off by one on the first pass and the totals hid it, because the marker carries no usage.
The trap itself is untouched. The two forks migrate to cuts of 47 of 65 events and 83 of 104, and report the same inflation they did at 1,008 of 1,183 and 3,171 of 3,284: 11,418 tokens against 2,204 of own work, and 263,790 against 11,442. Still 5.18× and 23.05×. Only the field you check changed.
D-3 is unchanged, which is why the title is unchanged. At d347e703 the file is 221 lines, compaction/summary appears zero times, and usageOf() handles assistant/message and assistant/attempt and nothing else. The three compaction events still cost 48,895 tokens and are still counted by nothing. Measured at 47f9438 (v0.1.0-rc.5), still there at d347e703 (0.1.3-alpha.1). A breaking rewrite of the surrounding event model went straight past it.
D-4 is still fixed, and v2 shows the fix was structural rather than lucky. llm/retry appears twice in the file and tokenUsage.stateVersion is 2 at line 122. A retried step still settles twice under one (turn, step), once as assistant/attempt for the attempt that died and once as assistant/message for the one that finished, so the replacement slot and the llm/retry-started boundary are both still load-bearing. On this corpus the retry-aware and retry-blind folds agree at 536,770, because the one failed attempt here reported zeros. That was true in August and it is still the reason I have no number for D-4.
On the probe. It reads assistant/chunk and seedLength, so on a v2 log it finds neither and reports nothing. Nothing is not a pass. The v2 reader and the migration harness above are going into the repo next.
Update, August 28, 2026. The retry half is fixed upstream. The compaction half is not.
dsh-v0.1.2-alpha.1 shipped on August 27, commit cd5ef814. In it tokenUsage.stateVersion went from 1 to 2, and apply() now handles llm/retry-started: it clears the (turn, step) replacement slot so the retried attempt adds to the total instead of overwriting the attempt that died. That is D-4, closed. The projection's own comment now states the rule outright, that a repeated sample replaces its attempt's earlier value while llm/retry-started closes the replacement slot.
They keyed the boundary on the retry event rather than on an allowlist of failure kinds. That avoids the fragility I flagged on yha9806's patch: a future adapter with a third failure kind cannot silently revert this one. aborted appears nowhere in the file.
I checked the previous state instead of assuming it. At b150a551b8, the master I quoted on August 26, that file is 219 lines, stateVersion is 1, and llm/retry appears zero times. The change landed between the 26th and the 27th.
usageOf() is unchanged. It still matches assistant/chunk and assistant/message and nothing else, and compaction/summary appears nowhere in the file. D-3 is open at 0.1.2-alpha.1, which is what the title of this post says.
None of my numbers move. D-4 was the one finding I said I could describe but not measure, because every failed attempt in my corpus reported zeros. The fix corrects something that cost me nothing to observe, so the 2.000000×, the 23.05× fork, and the 48,895 compaction tokens all stand as measured at 47f9438.
One thing I framed wrong. I wrote that a defect surviving four releases is worth more than one pinned to a date. Half of it did not survive the fifth. The compaction gap now carries that claim alone: measured at 47f9438 (v0.1.0-rc.5), still there at cd5ef814 (0.1.2-alpha.1), five releases later.
On August 27, vpimshin posted a fork branch fixing both halves with tests and did not open a PR, because CONTRIBUTING still refuses external ones. The half that is still broken now has five independent implementations of a fix and no way in.
Update, August 26, 2026. Three things in this post went stale, and one of them was my own mistake. In order of how much they matter:
The baseline is 47f9438, v0.1.0-rc.5, not rc.6. I mislabelled the tag when I wrote this and corrected it in the conformance repo on August 25 (787a8f1). Every measurement below stands. Only the version label was wrong.
Superseded on August 28, see above. Upstream shipped four releases and did not touch the file. Master is b150a551b8 today, and the root package.json and packages/llm/token-meter/package.json both read 0.1.1-rc.2. GitHub Releases is no longer empty: four prereleases went out between August 17 and August 21, ending at dsh-v0.1.1-rc.2. Through all of it, usageOf() still matches only assistant/chunk and assistant/message, apply() still never sees llm/retry or a failed finish chunk, tokenUsage.stateVersion is still 1, and SESSION_FORMAT_VERSION is still 0. The claim gets stronger rather than weaker: measured at 47f9438 (v0.1.0-rc.5), still there at b150a551 (0.1.1-rc.2), four releases later.
The probe moved. It is at token-accounting-conformance now; the old traceguard/usage-tracker-audit/dsh-probe path is a stub and its results/probe-report.json returns 404.
One thing in the thread did change. On August 25, a137460387 ran the conformance checker from a fresh clone, the first person other than me to run it. Seven folds and the D-5 identity passed self-test, and his compaction-only patch failed with a residual of exactly gap_inherited − gap_superseded, the two dimensions he left out on purpose. A named residual is what I built the checker to produce, and that run is the first evidence it produces one for somebody who didn't write it.
Four numbers from a nine-day-old codebase, measured this week across two providers:
- Summing every usage record in a DeepSeek Harness session log gives 2.000000× the correct total. Not roughly two. Six digits, no remainder, reproduced independently on both routes.
- One forked session reported 263,790 tokens for 11,442 tokens of its own work, a 23.05× overstatement, because its log physically contains a copy of its parent's history. A second fork did the same thing at 5.18×. Nothing bounds the ratio.
- 48,895 tokens across three compaction events were counted by nothing, including the official projection that most plugins read from.
- A stream that died and retried left three usage samples under one step, the first of them all zeros. Keep-first reports that step as free.
The first and last are traps for people writing plugins. The middle two are gaps in DSH's own code, and the compaction one is what I would fix first.
Why I went looking
I spend a lot of time reading agent transcripts and adding up tokens. Over the past few months that turned into ten upstream fixes across four usage trackers: splitrail (216 stars, three issues), tokscale (4.6k), Clawdmeter, viberank, plus an open PR against claude-code-templates (30k). The pattern was always the same. Claude Code rewrites session files in place on resume and compact, so anything recomputing totals from live files inherits the drift. Streaming leaves partial snapshots that get summed as if they were separate calls. Subagent transcripts sit one directory deeper than a flat glob reaches, and on one corpus 54% of messages never entered any total.
Eleven of those lessons are written up as invariants in a catalog. DeepSeek released Harness on August 13, and the ecosystem produced thousands of plugin repositories within days. A curated registry snapshot on August 15, covering 457 of them, listed 27 that count tokens. I wanted to know whether the same class of bug had been reproduced at scale.
It hadn't. That surprised me, and it is worth saying before the criticism.
What they designed out
DSH's session log is append-only, and that is a written contract rather than an observation. The JSONL backend's README says "Flushed events are never rewritten." Every event carries a dense contiguous sequence number, checked on append. Compaction shadows old events in the surface projection and leaves the bytes alone: "The shadowed events remain in the raw log, so replay is deterministic."
That one design decision removes the failure that cost viberank 11% of a month-to-date total between two submissions sixteen hours apart.
Two more. Adapters emit one terminal usage value per step, never a growing snapshot, so there is nothing to mis-sum. Child sessions are siblings in the same directory rather than nested underneath, so a depth-limited walk cannot lose a third of the spend the way it did in splitrail.
Four of my eleven invariants are structurally satisfied here. I put them in the DSH catalog anyway, marked as satisfied, because a catalog that only adds rules is not a catalog. It is a list of fears.
Then I measured, and found four new ones.
Every usage sample is written twice
One model call reports its usage on two different events. Once as a stream chunk with chunk.type === 'usage', once again on the assembled assistant/message. Same numbers both times.
Fold naively and you get exactly double. The ratio printed as 2.000000 on the full corpus, on the local qwen route alone, and on the MiniMax route alone, each computed independently.
An approximate factor is arguable. An exact one is not, and that is the whole reason to report it this way. It also tells you the mechanism is universal rather than occasional, which a ratio of 1.9 would not.
The cache buckets do it too. 249,728 cache-read tokens on the MiniMax route, 81% of that route's corrected total, doubling exactly like input and output. That is the number I most wanted, because cache is usually the biggest bucket and the cheapest per token, so a doubling there moves a cost report further than a doubling of output does. Cache writes are still untested: neither provider populated the field.
DSH's own token-meter handles this correctly. It keeps one slot for the last (turn, step) and subtracts the previous buckets before adding the new ones. Its README spells it out. The official implementation going to that trouble is the evidence that the hazard is real, not theoretical.
One caveat that keeps the number honest, and it turned into the fourth finding. A step is not always one request, and when it isn't, the two-samples-per-step assumption breaks. More on that below.
The fork trap, and the field you must not filter on
A forked child's log contains a copy of the parent's completed prefix. The header carries seedLength, and every event below it belongs to the parent. Add sessions together without checking and you count that prefix twice.
I expected this to be a subagent problem. I was wrong, and being wrong is the interesting part.
A subagent child is stamped origin: 'subagent' and its delegation depth increments. But ctx.sessions.fork() is an ordinary user-facing action available on any session, and the child it produces has parentSession and seedLength set, delegationDepth: 0, and no origin key at all. Here is the header I actually got, from an ordinary fork that showed up in the course of using the web UI. I wasn't trying to make one:
{"type":"session","version":0,"id":"session-e61d64ec-…",
"parentSession":"session-001e8887-…","seedLength":1008,
"delegationDepth":0,"agentPreset":"standard"}
That one inherited 1,008 of its parent's 1,012 events and reports 11,418 tokens against 2,204 of its own work. The second fork I caught inherited all 3,171 events of a longer parent, asked one question, and reported 263,790 tokens for 11,442 tokens of work.
That is the shape of it. The error is not a factor, it is the ratio of inherited work to own work, and nothing bounds it. Fork a long session, ask one question, and the child reports the entire parent as its own.
So the obvious defence is the wrong one. DSH's own lineage index opens with "Ordinary forks terminate propagation" and starts with if (descendant.origin !== 'subagent') continue. That filter is correct for counting subagent descendants. Reused for token accounting it admits every ordinary fork silently. The only sound discriminator is seedLength.
Official telemetry gets this right by the right key, emitting session.parent_id and session.seed_length and expecting receivers to stitch on the pair. Nothing does that for you if you read files.
The one that is DSH's own gap
Compaction summarizes older history by making a model call, and that call costs real tokens. They land on compaction/summary.usage.
The official tokenUsage projection cannot see them. Its usageOf() matches assistant/chunk and assistant/message and nothing else, and the summarize call is not a loop step, so it produces neither.
This is worse than a plugin bug, because the plugins doing the right thing inherit it. Reading sessionProjections.tokenUsage instead of folding the log yourself is the correct, recommended approach. It is also how you miss this.
On my corpus that was 48,895 tokens across three compaction events. The largest single one is worth quoting in full: a MiniMax-M3 summarize call reporting 44,444 tokens (536 input, 2,436 output, 41,472 cache reads) to remove a range whose own shadowedTokenCount was 19,962.
Whether spending 44,444 tokens to shed 19,962 is a good trade depends entirely on what you pay for cache reads. I'm not going to tell you it's bad. I am going to point out that nothing in the official projection tells you it happened.
The usage? field is optional, so a provider reporting nothing on the summarize call produces no gap. That is a condition on the finding, not an escape from it. Both of my providers populated it.
A retried step is not one request
The last one I found by accident, chasing why one step had three usage samples instead of two.
seq 3203 chunk/usage {"inputTokens":0,"outputTokens":0}
seq 3204 chunk/finish {kind:'error', failure:{code:'TRANSPORT'}}
seq 3205 llm/retry retryId=afabacf1 provider=minimax-cn
seq 3206 llm/retry-started
… the whole response streams again
seq 3279 chunk/usage {"inputTokens":32,"outputTokens":1042,"cacheReadTokens":10368}
seq 3281 message usage {"inputTokens":32,"outputTokens":1042,"cacheReadTokens":10368}
The stream died, the harness retried under the same (turn, step), and the dead attempt left a usage chunk of zeros behind. Three samples, and one more retry would make four.
Keep-first reports that step as costing nothing. Not an approximation, the whole step, including 10,368 cache reads. If that sounds familiar, it is the DSH form of a Claude Code bug where keep-first lost 46.2% of output tokens on an agent-heavy tree.
It also means my 2.000000× survived that group by luck rather than structure. The dead attempt reported zeros, so summing three samples still gave twice the truth. That's why the probe prints the group-size distribution next to the ratio instead of the ratio alone.
There is a second consequence I can describe but not measure. The official fold replaces on a repeated (turn, step) rather than adding, and it never sees llm/retry (as of 0.1.1-rc.2; fixed in 0.1.2-alpha.1, see the August 28 update). So a failed attempt that reported real tokens before dying would have its cost silently dropped. Every failed attempt in my corpus reported zeros, so I have no number for this and I'm not going to invent one. The discriminator is sitting right there in the log if you want to handle it: llm/retry carries a retryId between the attempts.
Where the two upstream ones stand
I filed the compaction gap and the retry replacement upstream on August 15, as discussion #1886. Same day, yha9806 posted a working fix on a fork (63688b0): it folds compaction/summary.usage into the totals, treats a finish chunk with an error or aborted reason as an attempt boundary so retried attempts add instead of replace, and bumps tokenUsage.stateVersion from 1 to 2. I re-folded my corpus against it and the corrected totals matched independently, which is a better outcome than agreement on prose. The attempt-boundary approach is also better than what I proposed, because it needs nothing outside the package.
One note I left on it, in case it helps whoever lands this: keying the boundary on an allowlist of two failure kinds means a future adapter with a third failure kind silently reverts to the old behaviour. Testing for 'failure' in event.data.chunk.reason fails safe instead.
None of it is upstream yet, and the version keeps moving without the file moving. I measured all of this at 47f9438, v0.1.0-rc.5. Today master is cd5ef814 and both the root package.json and packages/llm/token-meter/package.json read 0.1.2-alpha.1. usageOf() still matches assistant/chunk and assistant/message and nothing else, so the compaction gap is where it was. The retry half is not: 0.1.2-alpha.1 bumped tokenUsage.stateVersion to 2 and made llm/retry-started an attempt boundary. So the claim narrows to one finding: measured at 47f9438 (v0.1.0-rc.5), still there at cd5ef814 (0.1.2-alpha.1), five releases later. Pull requests are turned off on the repository rather than merely empty, which is why fixes here pile up in fork branches and discussion threads instead. I'll update this section when it stops being true.
What I am not claiming
Four sessions, 8,650 events, 78 usage samples, two providers. The 2.000000× ratio is exact and does not need a large sample to mean what it says, and it reproduced independently on both routes. The fork trap has two observations, the compaction gap three, the retry one.
Cache writes are still untested. Reads are covered now, 249,728 of them, but neither provider populated cacheWriteTokens at all. MiniMax omits the key.
The compaction gap came to 15% of the corpus total, and I am deliberately keeping that percentage out of the summary. Three compactions over two short sessions inflates it. The citable facts are 48,895 tokens and three events.
Both routes are openai-completions-family. Whether an Anthropic-protocol or Responses-protocol route behaves the same, I don't know.
I got my own first pass wrong twice, which is why the probe prints four folds instead of two. My first attribution credited the compaction gap to the double-write line, because naive summing picks up compaction and the official projection doesn't. And I assumed the fork was a subagent until I read the header. Both corrections are in the probe now, along with a warning line that fires when a seed-bearing session has no origin.
Reproducing it
The probe is stdlib-only Python, about 400 lines, and runs against a session root:
python3 dsh_usage_probe.py --self-test # four folds vs a hand-computed fixture
python3 dsh_usage_probe.py --root <sessions> # the same four folds over your corpus
--self-test builds a parent/child pair exercising all three findings with constructed ground truth and asserts every fold against it. It takes under a second, touches no real data, and needs no vendor account. I ran it before every number in this post, and I would not quote one it hadn't preceded.
Write your probe corpus with compression: 'none' and packChunks: false and the log is line-readable by anything. One warning that cost me twenty minutes: a custom provider in DSH requires a credential even when the endpoint doesn't, because the provider id doubles as the credential name. An empty key fails the request with MISSING_CREDENTIAL rather than sending an unauthenticated one.
What would settle it
Three things, in the order I would do them.
Run it on a route that reports cache writes. Reads are settled; writes are the remaining hole, and it is an afternoon.
If you maintain a DSH plugin that counts tokens: fold per (turn, step), filter on seedLength, never keep-first, and add compaction/summary.usage. The first three you can fix today. The fourth needs the projection to change, or every consumer to fold the summary event themselves.
And if your corpus contradicts any of this, I want the numbers. A step whose two usage samples disagree, a seedLength that doesn't bound the inherited prefix, a provider that populates the summary usage into the projection, or a failed attempt that reported real tokens before it died. The Claude Code catalog was built entirely out of people sending me counterexamples, and six of its entries exist because someone did.
Probe, protocol, and the full invariant catalog are in the repository. Happy to answer questions about any of it.
— Li Zhuojun
A routing audit for your team, fixed price. The method in this series runs on any Claude Code trace store. Send me 30 days of your team's usage records (model, tokens, timestamps, agent and session ids; I'll send a one-line jq filter that drops prompt and answer text before anything leaves your machine) and within ten working days you get one number and a five-page report: what share of your spend ran on a tier your own routing policy would not have chosen, what that cost at list price, and which components caused it. On my own 26,131 traces the number was 22.6% and $1,248.13, all of it on subagents and none on the main thread. US$1,500, flat. Write to info@zhuojun.li with the subject "routing audit".
Top comments (4)
The 2.000000x is the most actionable number here and I would build the whole check on it. A drift bug gives you 1.98 or 2.07. An exact integer multiple with six digits of nothing after it says the same records are being visited twice by construction, which is a much narrower search than "the totals look high".
The compaction gap is the one I would expect to survive longest, because it is the only one of the four where the correct answer is not recoverable from the file you are reading. The other three are recomputable once you know the rule.
Do you have a case in the conformance catalog that asserts sum(events) == projection for a session that has compacted at least once? That is the assertion I would want failing in CI on day one, and it is cheap because it needs no ground truth, only internal consistency.
Yes, and the exact phrasing you used is the one it fails at.
That's D-5 in CONFORMANCE-DSH.md. Stated as
sum(events) == projectionit fails for the wrong reason: D-1 writes every usage sample twice and D-4 adds a third under a retried step, so a raw sum of sightings lands near 2x the projection on any session at all, compacted or not. The left side has to be a corrected fold first. Collapse per(turn, step), treat a failed terminal chunk as an attempt boundary, skip the inherited prefix onseedLength.Then the residual is not zero, and not one number either:
Bucket for bucket, nothing left over. It closes at +2,208 against +2,208 on the committed fixture, and -212,667 against -212,667 on my corpus, where the superseded term is 0 because both failed attempts there reported zeros.
The decomposition is the part worth having. A total that is merely wrong tells you to go looking; a residual landing exactly on one term tells you which invariant you missed, and a residual matching none of them says the catalog is incomplete.
I published the narrow form first, residual equals the compaction term alone, and corrected it. It only holds on a log with no forks and no token-bearing failed attempt. On my corpus the inherited-seed term is 261,562 tokens, so the narrow identity misses by more than it captures.
You're right that it costs nothing to run.
dsh-conformance/runs it against a committed synthetic fixture in under a second, stdlib only, so it works as a day-one CI gate needing nobody's corpus.On compaction surviving longest: half of that already broke. tokscale merged the compaction fold on 8/22 (#1162), before its DSH client ever shipped, so that number is correct the first time anyone runs it and there is no inflated history behind it. Upstream in DSH itself it is exactly as you predicted. Four independent implementations of the fix now exist and none can land, because CONTRIBUTING closes external PRs. Same defect in two trees, and what decides its lifetime is not difficulty.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.