Why I Built a Git-Anchored Memory for AI Coding Agents

#ai #opensource #git #programming

Give your AI coding agent a memory, and sooner or later it will use that memory to confidently repeat a mistake.

Here's a concrete shape of the failure: an agent tries bumping a batch size to speed up a slow export job. It works locally, so it ships the change. Weeks later, a different session — same repo, same agent, no memory of the first attempt — proposes the exact same optimization. It doesn't know that batch size was already tried and reverted, because it caused a staging timeout the original, smaller batch size had been chosen specifically to avoid. Without a memory, the agent just repeats the loop. With most "AI memory" tools, it also just repeats the loop — the abandoned attempt gets stored, then surfaced back as if it were still live advice, because nothing ever told the store that the advice had expired.

That gap is what I built robo-cortex to close.

The blind spot in "AI has memory now"

Most agent-memory projects reduce to the same recipe: embed some text, store the vectors, retrieve by similarity. It's a reasonable recipe for recall — did we ever say something like this before? It has no concept at all for validity — is this still true, given what the code looks like right now?

A vector store doesn't know the difference between a lesson that's still accurate and one the codebase quietly outgrew three refactors ago. Both come back with a high similarity score. The agent has no signal to distinguish "this is still the way things work" from "this is how things used to work, before someone fixed it." So it trusts both equally — which, for the second kind, means agents can end up more confidently wrong than if they'd had no memory at all.

The fix: anchor memory to the code, not to a vector

robo-cortex takes a deliberately unglamorous position: no embeddings, no vector database, no cloud dependency. Every memory — a lesson, a decision, an experiment — can link to the specific files it's about, captured at the git blob hash of that file at the moment the memory was recorded.

When that file's content changes, the hash no longer matches. On the next retrieval, the memory is automatically flagged needs_review — not silently trusted, not silently deleted. If a change gets reverted (committed, matching the original hash again), the memory heals back automatically. No LLM judgment call in the loop, no similarity threshold to tune — a blob hash either matches or it doesn't.

The abandoned-experiment case above is exactly the pattern this is built to catch. Here's the actual sequence, from the project's own test fixtures — an experiment gets abandoned with a reason, a lesson gets linked back to it, and the next retrieval for the same task ranks the lesson above the dead idea while the abandoned experiment itself drops out of the default result set:

$ robo-cortex status fixture-repo-a 2 abandon --reason "batch_size 500 caused the same staging timeout the original 50-item limit was chosen to avoid; reverted." --json
{"id": 2, "status": "abandoned"}

$ robo-cortex retrieve --repo fixture-repo-a --task "should I increase the scanner batch size to speed up exports" --json
{"data": [
  {"id": 3, "type": "lesson", "status": "provisional", "score": 0.599, ...}
], "meta": {"matched": 2, "returned": 2, ...}}

The lesson (id 3) comes back; the abandoned experiment (id 2) is filtered out of the default pack — not deleted, just not offered as current working knowledge.

What it costs to be honest about a small number

I ran a 30-session benchmark: 3 real historical bugs from robo-cortex's own repo, re-introduced and re-solved 10 times each by a small model (Claude Haiku 4.5), with and without a robo-cortex memory pack. The results, medians of paired per-task deltas:

Fix quality: identical in both arms — 30/30 correct, zero disqualifications
Fresh tokens: ~12% fewer, with the memory pack
API-equivalent cost: ~15% lower
Hardest task to locate unaided: up to −26% tokens / −39% cost
Break-even: a recorded memory pays for itself after ~1–6 reuse sessions (median ≈ 5)
Wall-clock: no claim made — time deltas were noise-dominated

Known limits, stated plainly rather than buried: one subject model, one well-documented codebase (which likely compresses the effect), three task types. An earlier, smaller pilot on a larger model measured a weaker 2.7% effect. Full methodology, raw per-run evidence, and the benchmark harness itself are in the repo: benchmark-results.md and EVALUATION.md. I'd rather publish a modest number I measured than a dramatic one I didn't.

What it deliberately isn't

No embeddings, no vector database, no network calls unless you opt into evidence verification. Just SQLite (with FTS5 for search), scoped to your repo — .cortex/memory.db is a plain file you can inspect with sqlite3 any time. Memories carry an audit trail: every status change requires a --reason, permanently recorded. Nothing disappears silently.

The open question

The benchmark above is honest, but it's also small and run on my own well-documented repo. The real open question is whether git-blob-hash anchoring holds up the same way on messier, less disciplined real-world code — code where files churn constantly for reasons that have nothing to do with the lessons attached to them. I don't know yet. If you try it on a real project, I'd genuinely like to hear what breaks.