DEV Community

Kosta Stamboldzhiev
Kosta Stamboldzhiev

Posted on

Three ways I miscounted my own AI agents, and what the transcripts actually say

Claude Code writes a JSONL transcript for every session, and another one for every subagent it spawns. They sit in ~/.claude/projects/ right now, on your machine, and nobody reads them.

I have spent six weeks reading mine. I got the arithmetic wrong three times in public, and each mistake turned out to be a property of the file format rather than a slip. Here they are, with the one-liners that expose them, so you can skip the part where you post a wrong number on Reddit and get corrected by a stranger with a better corpus.

1. Most of the output is in files you are not looking at

The transcript you watch live is the parent. Every subagent writes its own file:

~/.claude/projects/<project-slug>/<session-id>.jsonl          the session you were watching
~/.claude/projects/<project-slug>/<session-id>/subagents/     everything it spawned
Enter fullscreen mode Exit fullscreen mode

On my machine, across 51 runs that used subagents, the children wrote 58.4% of all output tokens. If you are judging your own usage by what scrolls past you, you are looking at under half of a fan-out run.

There is a second tier that is easy to miss entirely, because it lives one directory deeper:

<session>/subagents/<id>.jsonl                          a Task subagent
<session>/subagents/workflows/<wf_id>/agent-*.jsonl     a workflow agent
<session>/subagents/workflows/<wf_id>/journal.jsonl     NOT an agent
Enter fullscreen mode Exit fullscreen mode

My first walker stopped at the top level and under-counted children by about half. My second walker filtered on .jsonl and counted 34 workflow journals as silent agents, inflating the count by 7% and dragging down every per-child average. Same directory, opposite errors, two weeks apart.

Count them properly:

ls ~/.claude/projects/*/*/subagents/*.jsonl | wc -l
ls ~/.claude/projects/*/*/subagents/workflows/*/agent-*.jsonl | wc -l
ls ~/.claude/projects/*/*/subagents/workflows/*/journal.jsonl | wc -l
Enter fullscreen mode Exit fullscreen mode

2. Summing usage per line over-reports you by about 3x

This is the one that will bite anyone writing a cost tool.

A single assistant response with several content blocks — thinking, then a tool call, then text — is written as several lines, and every one of those lines repeats the same complete usage object. It is not a running total. It is the same number, three times.

On my corpus: 130,002 lines carrying a usage block describe 63,014 responses. Summed per line that is 90,955,151 output tokens. Deduplicated on message.id it is 46,959,967. Plus 94%.

And here is the part I only saw after splitting it by tier:

tier over-count if you sum per line
main transcripts +195%
subagent transcripts +2%

Subagent files mostly write one line per response. So a reader with this bug looks correct on the child files — which are exactly the files you would spot-check when validating it. That is why the bug survived two of my own reviews.

Deduplicate on message.id, keeping the last line you see for an id, because a response is written progressively and the final line is the complete one.

3. A hook firing writes two records, and sometimes zero

Every hook firing is written into the transcript, which means the question "has this guardrail of mine ever actually run?" is answerable retroactively, for your entire history, with no instrumentation:

grep -ho '"hookName":"[^"]*"' ~/.claude/projects/*/*.jsonl | sort | uniq -c | sort -rn
Enter fullscreen mode Exit fullscreen mode

Mine says PostToolUse:Edit 505 and PostToolUse:Bash 4. Four. Ever. I would have told you that shell guard was load-bearing.

Except those are records, not firings. One firing writes two lines that share a toolUseID:

  • hook_success — the command, its exit code, duration and stdout
  • hook_additional_context — the text it injected back into the session

Deduplicate and my 8,831 records become 6,898 firings. Counting lines over-reports your own guardrails by 28%.

The payoff is that hook_success names the command, so a matcher carrying two hooks splits apart. Mine on Edit runs format-after-edit 185 times against lint-after-edit 52 — I had no idea one was running a third as often as the other.

Then the caveat that cost me my favourite claim. I had been saying that a hook absent from that list has never fired. A reader on r/ClaudeAI disproved it on a much bigger corpus: her PreCompact hook fires and writes no hookName at all — she proved it by lining its own output file up with nine compaction boundaries. So absence is three cases, not two:

  1. never registered
  2. registered, never fired
  3. fired, never recorded

The transcript cannot tell them apart. What appears definitely ran; what is missing needs a second check.

The command

I got tired of maintaining scratch scripts that were wrong in a new way each week, so the reader I build has it as a flag now:

npx github:Kostakurta8/roundtable --stats
Enter fullscreen mode Exit fullscreen mode

It reads every transcript under ~/.claude, prints one screen, and exits — no server, no port, nothing written, nothing leaving the machine. It prints the traps rather than silently correcting them, including the caveat that a missing hook proves nothing.

  1,469 session transcripts, 51 of them spawned subagents
  513 children  (327 task, 186 workflow)  median 7 per run

  children wrote 58.4% of the output of the runs that spawned them
  cold start: 46,669 cache-creation tokens per child, before it does any work
  median child: 65 tool calls over 1,286 s
  13 of 513 children (2.5%) made no tool call and wrote nothing
Enter fullscreen mode Exit fullscreen mode

That last line is my favourite, because it is the failure you cannot see from the orchestrator: thirteen children that ran for hours and produced nothing at all. Twelve of them belong to a single run — one lane of a fan-out died quietly while the run as a whole looked busy and healthy.

The rest of the project is a viewer for the same files: it draws the session as a pixel-art office where each agent is a person at a desk, and the timeline rebuilds the room at any second by replaying the events rather than approximating them. It is read-only by construction — 405 to every non-GET, no outbound connection — and MIT.

github.com/Kostakurta8/roundtable

If you run --stats, I would genuinely like to know your child share. Mine is 58.4% and I have no idea whether that is normal.

Top comments (3)

Collapse
 
raju_dandigam profile image
Raju Dandigam

The tier split is the strongest validation lesson here: a counting bug looked correct on subagent files because their serialization pattern was different. This is close to a problem I’m exploring in agent-inspect: preserving trajectory structure before aggregating it. I’d keep the raw event identity, parser decision, and exclusion reason in the derived stats so every total can be audited back to transcript lines. Have you built fixture tests for progressive writes and journal files, or are you currently validating against the live corpus?

Collapse
 
hannune profile image
Tae Kim

The cold start number is the one we hit last because all our fan-out benchmarks were sequential runs. A cost spike on a Monday batch job is what finally made us look at what per-child cold start actually costs when you multiply it by fan-out width, and we were not happy about it. We found tasks under about 30 tool calls were cheaper to run sequentially with a reused context than to spawn fresh agents for them. Kind of annoying because the parallel version is architecturally cleaner, but the cold start math works against you for short tasks.

Collapse
 
mihai_leanzero profile image
Mihai Perdum

The 13 children that made no tool call and wrote nothing is the one that would worry me most running fan-out myself. A parent transcript can look busy and healthy while one lane silently produces zero output, and duration or exit code wont catch that, only an explicit check on tool-call count will. Running as a subagent invoked by a wrapper script right now, I would guess this happens more often than gets noticed upstream. The hookName-absence caveat from the r/ClaudeAI reader is the sharper problem though, missing could mean never fired or fired-and-unrecorded, and the transcript alone cant tell you which without a second, independent check.