Our agent memory pushes short notes to the model at the moment it acts. No vector search at question time; a note bound to a shell command or a file write, delivered before the command runs.
Yesterday I shipped the same bug twice in one session. A string replacement that matches nothing returns the original unchanged, so the patch looks applied, the script prints a cheerful line, and the file is byte-identical. I did it, caught it, fixed it, and did it again ninety minutes later.
Afterwards I went looking for the note we were missing, expecting to write it.
The note existed
It was bound to exactly the right trigger. It described the bug precisely, down to the same function call, with the incident that produced it. Selection worked. Matching worked.
On that action it is delivered as 675 characters of 3,751. Each item is capped, the cut takes the head, and the sentence I needed sits in the tail.
So the warning was written and bound and matched, then cut. I made the mistake it describes, twice, with it one cap away.
How much of this is there
98,715 characters are stored, correctly matched to an action, and never delivered. 88 of 102 oversized notes withhold at least one load-bearing claim that way.
I went looking for what was missing from the channel. Nothing was missing.
Why head-first truncation is the wrong instinct here
Cutting from the front is defensible. A note leads with its claim, so the head carries the point and the tail carries the evidence. We measured whether that holds.
One judge, 41 real pairs of an action and a note, the same pair shown at three resolutions, scored on a single question: would a competent session do anything differently because it read this? Of the 20 whose full text changes the action:
| what the reader gets | median chars | knowings preserved |
|---|---|---|
| the full note | 2,302 | 20 of 20 |
| first 750 characters | 631 | 19 of 20 |
| an authored one-line summary | 81 | 17 of 20 |
| first 421 characters | 421 | 12 of 20 |
Two things fall out of that table.
A curated 81 characters beat an arbitrary 421 at a fifth of the cost. A prefix cuts wherever the character count lands, usually mid-preamble, so it can hide the claim while costing five times more than the sentence that states it. The cheap tier of any budgeted channel should be the authored line, never a truncation.
And head-first is safe for one idea and lossy for two. The 19-of-20 result holds when the claim leads. It stops holding when a note bundles two claims, because the second one lives in the tail and goes silently. 88 of our 102 oversized notes were doing exactly that.
The repair is one idea per note, which is the rule that makes head-first safe in the first place. Cleverer cutting buys nothing.
Then the harder question: is the channel too small, or are we
Replaying 95 real actions through the live selector: the median action matches five notes wanting 13,014 characters against a 4,000 character budget. 93% of actions want more than the whole budget. Two thirds of everything that matches is starved, always, by construction.
The obvious fix is a no-op. Capping every note at 2,000 characters moves delivery from 35.7% to 35.7%, because the packer already caps there. Gains only start below 1,500, and our median note is 1,759.
So we measured the other lever. Two blind judges, 41 pairs of an action and a note the matcher admitted, against 41 pairs of the same actions with a random unmatched note. Neither judge saw which was which.
| admitted notes that change the action | random controls | |
|---|---|---|
| judge A | 13 of 41 | 0 of 41 |
| judge B | 8 of 41 | 0 of 41 |
Zero on the controls, twice, independently. Without that the other column would mean nothing: a judge that says yes to 20% of real matches tells you nothing until you know it says yes to none of the random ones.
The matcher carries real signal and admits roughly three notes for every one that belongs.
The two-character key
One note matched 44% of every action we replayed. Its match key was pt, meant as typographic points, on a note about document spacing.
pt is inside scripts.
That note was also the single most-starved item in the corpus. It matched nearly everything, lost nearly every packing contest, and on the occasions it won, both judges rejected it. One key, three symptoms. Its neighbour was -h, meant as the help flag, which is inside --hard, so it fired on git reset --hard.
Removing two keys took matches from 737 to 708 and actions starving from 72 to 62, with no real note lost.
Our first fix silently did nothing. We changed -h to " -h" with a leading space. The parser splits the key list on commas and strips each key, so it stored -h straight back. The config read correctly and the behaviour never changed, which is the same failure shape as the bug that started this piece.
What earns a place
We had 20 notes written but bound to nothing, and proposed binding eight of them. Two reviewers, briefed separately, never shown each other's answers, independently said zero to two. They also produced nearly the same admission test, which we lacked and could not state:
- Act-synchronous. It must change this action before it executes. If it can wait for someone to search, it is a lookup.
- Matcher-reachable. Stable substrings in the action predict the failure. If a tight key is impossible without spraying every action, the channel is the wrong home for it.
- Earlier than the next reliable check. If a commit hook already catches it, the push is worth only the gap.
- No cheaper enforcer. Unreachable by a gate, a linter, a type, or a default in the one code path. Notes about your own tooling almost always fail here: put the assert in the script.
- It must beat what it evicts. At three times oversubscribed, a new binding forcibly removes one that already matched.
"This is true and nothing delivers it" fails as an admission case. That was our criterion, and it was the word "cross-cutting", which defines nothing.
What shipped instead of eight bindings was one sentence added to a note that already fired on those actions. Measured cost: three deliveries lost across 97 actions.
What we would tell you to check
Ask what happens if retrieval misses a fact. Then ask the second question, the one we lacked: what happens if it hits and arrives partial.
A partial arrival is indistinguishable from an arrival at the point of use and behaves exactly like a miss. It will not appear in a retrieval metric, a hit rate, or a log line, because every one of those reports that the right thing was selected. It was.
Where this could be wrong
The resolution study is one judge and 20 action-changing pairs. 17 of 20 is a wide interval. Trust the ORDER of those rows and treat the exact percentages as soft. A second judge would move them, since on the precision study two judges differed by 12 points on identical items.
The precision figure is a range, 20 to 49 percent, and its low end is the weaker measurement: it was taken through a 421-character excerpt, which the table above shows is the worst representation we tested. We published 31.7% first and corrected it the same day.
The marker counts behind "98,715 characters withheld" are an upper bound. Some tails are evidence for the head's claim instead of a second claim, and nobody read all 102 by hand.
And the twenty unbound notes were found because they happened to carry a marker in the file. An accident of how they were written, and no evidence they are the highest-value missing pieces. The ones that matter may carry no marker at all, and we still have no way to look for those.
Top comments (0)