DEV Community

Nikhil Kumar
Nikhil Kumar

Posted on

I parsed 76,801 lines of my own Claude Code history. Three things nearly broke my parser.

Claude Code writes every session to ~/.claude/projects as JSONL. I wanted to know what I'd actually built over three weeks on one repo, so I started reading those files.

The format is undocumented. Here is what I measured across 28 transcripts, 275 MB and 76,801 lines, and the three things that gave me wrong numbers before I noticed.

1. The files replay, and counting lines overcounts by 3x

Claude Code appends to a transcript when you resume or rewind a session, and it rewrites records it has already written. The same uuid shows up more than once.

My largest session:

lines 36,676
distinct records 10,964
uuids appearing more than once 7,919

So counting lines, which is the obvious first thing anyone does, overstates that
session by more than three times. Worse, the overstatement is uneven, so there
is no constant you can divide by to fix it.

Deduplicate on uuid, keeping the last copy. Later copies carry fields the
earlier ones did not.

2. One field is sometimes an object and sometimes a string

toolUseResult is an object 17,319 times and a bare string 490 times.

// This is wrong, and it fails quietly
type Record struct {
    ToolUseResult *ToolUseResult `json:"toolUseResult"`
}
Enter fullscreen mode Exit fullscreen mode

If you type it as an object only, every string-valued line fails to decode. And
if your reader skips a record on a decode error, which is a reasonable default,
you lose it silently. In my own test fixture that dropped 85 of 340 records. I
found it a week later, by accident, because a count didn't match.

The fix is a custom unmarshaller that accepts either shape:

func (t *ToolUseResult) UnmarshalJSON(b []byte) error {
    if len(b) == 0 || b[0] != '{' {
        return nil
    }
    type plain ToolUseResult
    return json.Unmarshal(b, (*plain)(t))
}
Enter fullscreen mode Exit fullscreen mode

message.content has the same problem: a list of blocks 52,641 times, a bare
string 400 times.

3. Quiet commits arrive with no hash

When the agent commits, the result carries the commit:

"toolUseResult": {
  "gitOperation": {
    "commit": { "sha": "d0a65cc", "kind": "committed", "branch": "main" }
  }
}
Enter fullscreen mode Exit fullscreen mode

Except Claude Code fills that in by reading what git printed. Silence git and
the field never appears:

commit command calls carried a hash
without -q 57 53 (93%)
with -q 88 0

Relying on that field alone found 4 of one repository's 31 commits.

The dependable signal is the shell command itself: a git commit that returned
without an error. Which leads to the trap I enjoyed least.

The bonus trap: commits that never happened

If you detect commits by matching git commit in shell commands, a command that
writes a file containing those words looks identical to one that runs them:

cat > release.sh <<'SH'
git commit -m "release $VERSION"
SH
Enter fullscreen mode Exit fullscreen mode

Seven of one project's apparent commits were heredocs like this. The fix is to
check whether a heredoc opens before the match. A real commit taking its message
on a heredoc opens it after.

Why any of this mattered to me

I was building a thing that draws your Claude Code history as a diagram: squares for days, smaller squares for tasks, a circle for every prompt, with commits marked on the work that produced them. Every number above is one I got wrong first and had to go back and fix.

It's a Go binary, runs locally, reads files already on your disk and sends
nothing anywhere. github.com/nickelsec/bough

The full format notes, including record types and the fields that carry less
than you'd hope, are at bough.run/docs/format.

If you're parsing these files yourself, the replay is the one that will get you.
It's silent, it's uneven, and every number downstream of it is wrong.

Top comments (0)