I used to keep AI chat sessions open far longer than made sense, because closing one meant re-explaining my project from scratch. Rahul Garg has a name for the fix: context anchoring, "the practice of making that alignment durable." It went up on martinfowler.com and it is worth your ten minutes.
His test for whether you have got it right is the part that stuck with me:
If I can close my chat session and start a new one without anxiety, my context is properly anchored.
That reframed the problem for me. I had been treating the long session as the asset. It is the symptom.
Files, not a service
Once you accept that context should live outside the conversation, you have to decide where. There is a whole category of hosted "AI memory" products that will answer this for you, and I went and benchmarked one properly this week before deciding.
I would rather anchor to markdown files in the repo:
- Works offline. No account, no backend, no outage.
- Diffs in review. A teammate can argue with a decision record in a PR. You cannot review a vector store.
- Survives the vendor. I maintain an AI tools directory, and I have spent this month correcting listings for products that got acquired and wound down. Betting your project's memory on a seed-stage startup is a real risk, not a theoretical one.
Sync is a fine optional layer. It should not be the substrate.
The number that settled it
I measured both approaches on the same machine.
| Tokens per request | |
|---|---|
| File-based memory: a one-line index, 15 entries | 579 |
| Hosted memory MCP: 27 tool schemas, 0 entries stored | 8,866 |
The hosted service costs roughly 15x more per request, before it has remembered anything at all. The reason is architectural. MCP tool definitions ship on every single request whether you call them or not, so a broad tool surface is a permanent tax. A file-based index loads one line per memory and fetches the body only when it looks relevant.
Same idea. An order of magnitude apart in cost, purely from where you put the state.
The part I would add: anchors rot
Garg scopes his advice carefully, and the article is about capturing decisions while they are fresh. Here is what I hit after doing this for a while.
Yesterday I wrote a memory saying an automated check "does not exist yet and is worth building." Today I built it. The memory was now confidently, specifically wrong, and nothing flagged it. It would have kept telling future sessions to build a thing that already existed.
Anchored context decays exactly like documentation, except an AI reads it with more trust than a human would and acts on it. A decision record that names a function deleted last month gets asserted, not questioned.
So my working rule: every anchor carries the date and the reason, and anything naming a file, function or flag gets verified against the repo before I act on it. Writing the anchor is the easy half. Keeping it honest is the actual work, and I have not seen anyone solve it properly yet.
If you are already doing this, I would like to know how you handle staleness. That is the open problem.
Top comments (4)
The 8k token tax from MCP tool schemas alone explains why so many agent setups feel sluggish before doing any actual work. Shipping dozens of JSON schemas on every round trip eats context window fast.For context rot, the failure mode usually comes from letting anchor files track current feature status rather than invariants. Feature status belongs in tests and git history. When an anchor file documents what exists today instead of why a constraint exists, it starts drifting the moment someone pushes a commit.
The 8,866 came from my server, so I can confirm it: that is the tool-schema catalog, and it is a real architectural tax - you pay it before a single memory is stored. One correction to the framing, not the number: 579 vs 8,866 is index-versus-catalog, not file-versus-service. The catalog scales with tool surface, not with stored memories, and clients have started lazy-loading schemas on demand, which moves that cost from "every request" to "first use". A lean profile with three read tools is the configuration that should exist, and your post is a good argument for shipping it.
On staleness - that open problem at the end is the one I ended up building the most machinery for, and the core of it works in plain markdown too: a correction never edits the old anchor in place. It writes a NEW record that names the old one ("supersedes: X - reason: the check exists since today"), and the old one gets an end date. Whatever serves your anchors then refuses to serve the dead one without pointing at its successor. Your date-plus-reason rule is half of it; the missing half is that the relationship between old and new is DATA, not something the next reader has to reconstruct. Two frontmatter fields would do it in your setup.
Where I landed after measuring both directions: files are the right substrate for decisions a team reviews in PRs. The service earns its cost only where files stop - recall across machines mid-incident, and refusing to serve me my own confidently-wrong anchor.
The token number, closed out: I reproduced your measurement at the wire - 27 schemas, 32,778 bytes, ~8,859 tokens. Your 8,866 was exact to within tokenizer rounding. For context, that was already the slimmed state: the full surface is 123 tools (~27,700 tokens), cut to 27 in August behind a dispatcher that lists names without schemas.
Your post pushed it one step further: 0.10.152 adds CACHLY_PROFILE=lean - eight tools plus the dispatcher, measured 3,712 tokens per request. Everything else stays callable through the dispatcher, just without schemas riding along on every request.
Still not 579, and it never will be: the largest remaining block is the field documentation of a single write tool, and that prose is doing work - it is what teaches a model to put the key fact into the first hundred characters of a lesson instead of burying it. A one-line file index doesn't pay that tax because it doesn't buy that behavior. Your framing stands - where you put the state prices every request. Because you measured it, the price here dropped 58% in an afternoon.
Closing the loop on the autopilot claim from our other thread: verified, you were right - the file-writing code appended safely, but the printed instructions said cat > and would have flattened an existing CLAUDE.md by hand. Fixed in 0.10.152 (appends, and the same sweep caught an identical cat > in the git-hook guide). A bug report inside a comment, confirmed and shipped also same day.
Both fixes in this thread trigger on someone noticing: a supersedes record needs a person to know the old anchor died, and the invariant-versus-feature-status split assumes the drift shows up in your repo. The class that defeats both is a claim about something you do not own — "this endpoint omits field X", "that port is dead". No commit contradicts it, so git history holds no evidence either, and it is not feature status. I had one this week saying a dead debug port caused a silent fallback; re-measuring showed the dead port exits loudly with a discovery error and the missing flag was the quiet one, which is roughly the opposite instruction. Keeping the command that produced the anchor beside it is what helped, since re-running beats re-reading, though only while re-running stays cheap.