Your coding agent shouldn't get to grade its own memory

#ai #llm #agents #python

Most memory systems for coding agents have a quiet feedback loop in them. The agent recalls a note, uses it, and the system counts that as a sign the note is good. Next time the note ranks higher, so it gets recalled more, so it ranks higher still.

If the note is right, fine. If the note is wrong, you have built a machine for making an agent more confident in its mistake.

I ran into this while building skillmem, a local memory layer for Claude Code and Codex. It stores procedures rather than facts: trigger, steps, outcome, lessons. Each one has a strength that goes up when it helps and fades when it isn't used. The first version raised strength on retrieval, because that was the obvious thing to do. It was also wrong, and this post is about what replaced it and where the replacement still falls short.

Retrieval is not evidence

A retrieval tells you the query looked like the note. It says nothing about whether following the note worked. Asking the agent doesn't help much either, because the agent that followed the note is the one least able to judge it, and models are agreeable about their own work.

So strength now moves by the kind of evidence, and the default kind is worth nothing:

STRENGTH_BOOST = 0.15

EVIDENCE_WEIGHTS: dict[str, float] = {
    "self_report": 0.0,      # the agent says it helped: recorded, not rewarded
    "test_passed": STRENGTH_BOOST,
    "diff_accepted": STRENGTH_BOOST,
    "user_confirmed": STRENGTH_BOOST,
    "failure": 0.0,          # handled separately: multiplies strength down
}

FAILURE_FACTOR = 0.7

self_report is what plain retrieval produces. It refreshes recency and the access count, so a useful note doesn't fade away, but it can't climb.

Failure is deliberately asymmetric. One failure multiplies strength by 0.7, while idleness costs a factor of 0.85 per decay run on notes left alone for 14 days. A note that was followed and then broke something should lose ground faster than one nobody needed lately.

Going unused doesn't mean useless

Decay has its own failure mode. "Never force-push to main" might go unrecalled for months, and that says nothing about whether it matters. So a note can be pinned out of decay and archiving, and rules you wrote or approved yourself are sealed, so an agent's report of failure can't push them below the point where they stop being recalled.

Archiving moves notes out of recall after 90 days without deleting them. Being wrong about what to forget should cost one command.

Keep the signal small

The next temptation is to let strength dominate ranking. On a small Russian/English bench of ten queries, a grid sweep over ranking settings favoured a gentle bonus, and the larger coefficient I started with (0.3) let strong notes outrank more relevant ones. It is now 0.05, a tiebreaker. Ten queries is not much, so treat that as a direction rather than a measurement.

Relevance decides what comes back. Strength only decides between things that are about equally relevant.

Where this still falls short

Here is the part I'd push back on if someone else had written this post.

The evidence label is still a claim. The agent calls mem_reinforce with evidence="test_passed", and nothing checks that a test ran. The design removes the accidental loop, where merely recalling a note promotes it. It doesn't stop an agent that asserts a passing test it never ran. Closing that means taking the label away from the agent and deriving it from what the harness actually saw, such as a test command's exit code.

Confirmations aren't idempotent. A retried call counts twice. Evidence ids to deduplicate them are still to do.

I can't show it improves outcomes. I have a retrieval number: hit@5 0.889 on LongMemEval. That measures finding the right memory. I don't have a controlled result showing that earned strength makes an agent finish tasks better than plain recency does, and that is the number that would settle it.

The general point

Any system where the model's output feeds the score that decides the model's next input needs an answer to one question: what in this loop comes from outside the model? If the answer is nothing, it will drift, and it will feel like learning while it does.

Code is here if you want to read the implementation or argue with it: https://github.com/liza-studio/skillmem. It's reinforce in skillmem/storage.py.

I'd especially like to hear from anyone who has wired evidence to real harness signals rather than the agent's word.

Written with AI assistance; the code, constants and numbers were checked against the repository.

DEV Community