DEV Community

Cover image for How a strict allow-list silently broke token trimming in claude-mem (and the fix)
Rifat Hossain
Rifat Hossain

Posted on

How a strict allow-list silently broke token trimming in claude-mem (and the fix)

#claudecode #opensource #debugging #typescript

TL;DR: claude-mem, a Claude Code memory plugin with 100k+ GitHub stars, strips the full file contents out of Edit results before sending them to its background model. The strip only ran if every key in the result was on a fixed allow-list. Claude Code 2.1.309 added two boolean fields, so the strip silently stopped and every edit sent the whole file. I fixed it in PR #5027, which is merged upstream. On 639 real Edit results, trimmed payloads went from 29 to 168.

I did the investigation and wrote the fix with Claude Code.

The problem

My Claude Max meter showed 2% of the weekly limit used a few hours after the weekly reset, on a morning when I'd barely worked. My own Claude Code transcripts since the reset added up to about 0.26%. So most of that spend came from something that doesn't write transcripts.

That something was claude-mem's observer: a background Claude that reads your tool calls and writes memory notes. It runs as claude --no-session-persistence, so its usage never shows up in a transcript. The worker's /api/processing-status showed queueDepth: 2080, seven hours after my last session ended.

Two things made it expensive:

  1. Backlog replay. Near the limit, the quota guard paused at 93% and buffered work instead of dropping it: 3,285 tool calls. The weekly limit reset at 06:00, the plugin resumed at 06:08, and it replayed everything onto the new week. That was 677 observer turns between 06:00 and 07:00 (GMT+6).
  2. Heavy turns. Each observer turn re-reads its own conversation, which grows to about 100K tokens before it resets. On top of that, Edit results were carrying whole files. One ~210 KB file produced 7 calls in 44 seconds, each about 67.5K input tokens, about 150 output tokens, with zero cache reuse.

The backlog behavior and the conversation size are design decisions, and they're with the maintainer. The whole-file Edit payloads were a bug, and that's what this post is about.

Other users have reported heavy background usage from the observer too (#2315, #2451, #2643, #4965).

Root cause

Before 2.1.309, Claude Code's Edit tool result had these keys: filePath, oldString, newString, originalFile, structuredPatch, userModified and replaceAll. originalFile is the entire file as it was before the edit. For a big file, that's most of the payload.

claude-mem's compactEditOutput() (in src/services/worker/field-optimizer.ts) builds a smaller view without originalFile. But it bails out if the result has any key it doesn't recognize:

Object.keys(output).some(k => ![
  'filePath', 'oldString', 'newString', 'originalFile',
  'structuredPatch', 'userModified', 'replaceAll',
].includes(k))
// any unknown key → skip the trim, send the raw payload
Enter fullscreen mode Exit fullscreen mode

Claude Code 2.1.309 added two new fields to Edit results: contentNotInModelContext and staleRecovered. Both are booleans. From then on every Edit result had unknown keys, the guard returned early, and the raw payload went to the model with originalFile in it. Nothing failed and nothing was logged. The trim just stopped running.

There was a second way to end up in the same place. If the trimmed view was still over the size budget, the function fell back to the raw payload:

return JSON.stringify(view, null, 2).length <= maxChars ? view : fields.toolOutput;
Enter fullscreen mode Exit fullscreen mode

So a large oldString/newString also meant the whole file got sent.

Before changing anything, I replayed the function on real Edit results stored on my machine to make sure this was really the cause. It was.

The fix

Two small changes in that one function.

1. Unknown scalar keys no longer block the trim.

Object.entries(output).some(([k, v]) => !KNOWN_KEYS.includes(k)
  && v !== null && typeof v !== 'boolean' && typeof v !== 'number')
Enter fullscreen mode Exit fullscreen mode

(The real code keeps the key list inline. I shortened it here.)

The guard exists for a good reason: if an unknown field might hold file content, trimming around it isn't safe. But a boolean, a number or null can't hold file content, so they're kept in the view as they are. An unknown string, array or object still takes the cautious path. This also covers the next scalar flag Claude Code adds, without anyone having to update the list.

2. An over-budget view gets condensed instead of falling back to the raw payload.

// A view still over budget goes on to optimizeField, which condenses or
// truncates it; that is cheaper than sending the raw payload with the whole file.
return view;
Enter fullscreen mode Exit fullscreen mode

Verification

  • Tests: 4 passing and 2 failing before the fix, 6 out of 6 after. Typecheck and lint clean.
  • Replay: I ran 639 Edit results stored on my machine through the function. 29 were trimmed on main, 168 with the fix.

The maintainer merged it about 15 hours after my report:

"Point 2 (Edit payloads carrying the full originalFile into condense) is fixed by #5027, written by @smrifat1411, who also reported this … It's on main and ships in the next release. Thanks for such a thorough report."

Takeaways

  • Allow-lists on formats you don't own will meet a new field. Decide what an unknown field can actually carry, and only reject that.
  • Check where your fallback goes. Here the "safe" fallback was also the most expensive path.
  • Silent fallbacks are hard to catch. Nothing errored, so the only signal was the usage meter.
  • If your usage meter and your logs disagree, look for what isn't logging. Background processes started with --no-session-persistence are a good first suspect.
  • Replay real recorded data against the old and new code. Tests only cover the cases you thought of.

Links:

Top comments (0)