Everyone's doing it.
A MEMORY.md in the project root. A context.md the agent reads at session start. A few bullet points about past decisions. Maybe a CLAUDE.md if you watched Karpathy's video.
for one person running local agents? It works
Karpathy's setup is clever and clean.
But the moment you try to do anything more serious with agents, markdown falls apart.
Here's how:
The file has no concept of confidence: "We use Postgres" and "I think the retry logic might be buggy" look identical. Both are just sentences. Your agent treats them the same way.
Two agents writing to the same file? Last write wins: No branching, no merge, no conflict detection. You won't even know it happened until something breaks in production.
There's no history: You can't diff what the agent knew last week vs. now. You can't roll back a bad memory. You can't audit why an agent made a wrong call two deploys ago. The file just is what it is.
It also doesn't travel: Your MEMORY.md lives on your machine, for your agent. The second you add another agent, another developer, or a deployed pipeline, the memory doesn't follow. Every session burns tokens, re-establishing context that was already established last time.
If you're building pipelines with CrewAI, LangGraph, AutoGen, or others, the markdown thing isn't even on the table. Your agents don't share a file. They don't share anything. They contradict each other, duplicate work, and no one has a record of what was actually learned during a run. The orchestrator knows the output, but the reasoning that led to it is gone.
And the costs are way more concrete than people realize.
That CLAUDE.md file gets loaded into every single session. Every word, every time. If it's 500 lines, that's maybe 3,000 to 5,000 tokens of overhead on turn one. By turn 50 of the same session, you're re-sending the entire conversation history plus that file on every single message (That's before any actual work gets done. That's just the flat file being re-read over and over.)
But it gets worse as the session ages
By message 40, the agent isn't just re-reading the 4,000-token file. It's re-reading the entire history of accumulated conversations. A 40-message session where each message is ~500 tokens means the last message re-processes ~20,000 tokens of context just from history, on top of the file.
That compounds to something like $2,000–$3,500/month per 10-person team in context overhead that provides zero new value. It's just the agent re-reading stuff it has already read.
What a proper memory system changes
Instead of dumping everything into context, SenseLab compiles a ranked briefing of what's actually relevant to the current task. A 10-entry briefing might be 800 tokens instead of 4,000+. That's 80% reduction in memory related input tokens before you even account for the session history problem.
The memory compiles itself into ranked briefings so the next agent starts with what matters, not a flat file of everything that ever happened.
Evolving agents
Markdown solves "I want my agent to remember I prefer TypeScript."
It doesn't solve "I want my agents to accumulate knowledge over time, share it reliably, and not step on each other."
That problem is what most teams actually have. And there's almost nothing built for it.
That's why we built SenseLab. Git for agent memory. Versioned, branchable, with confidence scoring, causal traces, and cross-agent reads that are tracked.
Markdown will get you started, but it won't help you scale and improve agent context and experience.
Want to give it a try? Sign up for free at SenseLab
Top comments (0)