DEV Community: Jonathan Wilcox

CLAUDE.md vs. a memory MCP: what actually goes where

Jonathan Wilcox — Tue, 04 Aug 2026 06:00:58 +0000

If you use Claude Code or Cursor for real work, you already have a CLAUDE.md (or an AGENTS.md, or a .cursorrules). You've written down the build command, the "we use pnpm, not npm," the folder layout, the one weird thing about the auth flow. And it works. The agent reads it at the start of every session and stops making the same three mistakes.

So when someone shows up saying "add a memory MCP server to your agent," the honest first reaction is: why? I already have a file the agent reads. That's a fair question. This post answers it properly — including the cases where the honest answer is "you don't need one yet."

Disclosure up front: I build one of these (Kireo — more at the bottom). So read this as a builder trying not to oversell, not a neutral survey. But the framework below is the one I actually use to decide what goes in a markdown file and what goes in a memory store, and it holds no matter whose store you use.

The one distinction that matters: who writes it, and how often it changes

Here's the split that has survived contact with real projects:

	CLAUDE.md	Memory MCP
Who writes it	You, by hand	The agent, while working
What it holds	Stable facts	Accumulated decisions, dead ends, gotchas
How it changes	You edit it when it drifts	It grows every session
How the agent uses it	Loaded into context every time	Queried on demand

CLAUDE.md is for stable facts a human curates. "Run pnpm --filter api test. The API talks to Postgres and Redis. Don't touch generated/." You typed those. You re-read them when they drift. They change maybe once a month.

A memory store is for dynamic facts an agent accumulates. "Tried batching the embedding upload at 500 symbols per request — the embedding service chokes on batches that large, so I capped it at 100." Nobody hand-edits that into CLAUDE.md. The agent hit it on Tuesday, and by Thursday, in a fresh session, it's just gone — unless something wrote it down.

The test I use: would I open the file and type this by hand? If yes, it's a CLAUDE.md fact. If it's a byproduct of doing the work — a decision, a dead end, a "we tried X and it broke Y" — it wants a memory store, because you are never going back to curate it into markdown.

You don't need a memory MCP yet if…

I would rather you not bolt on a server you don't need, so here's the honest checklist. If all of these are true, a well-kept CLAUDE.md is enough, and adding infrastructure is just overhead:

One project. Your context still fits, and one file describes it.
One tool. You live in Claude Code, or only in Cursor — not both.
One machine. Same laptop, same setup, every day.
Small, curatable volume. The stable facts are few enough that you're happy maintaining them by hand, and you rarely hit "the agent solved this exact thing last week and I can't get it back."

If that's you, close this tab and go keep a good CLAUDE.md. Most solo, single-repo projects live here, and there's nothing to fix. A memory store is a tool for a problem you may not have yet.

When the file stops being enough

Four triggers move the needle. When you hit them, a static file starts losing:

1. Multiple tools. You start a feature in Claude Code, then jump to Cursor for the UI. Two tools, two separate CLAUDE.md-style files, and the context you built up in one is invisible to the other. Because MCP is an open protocol, a memory server both tools speak means a single store: written by whichever tool you were in, readable from the other.

2. Multiple machines. Laptop plus desktop plus a cloud dev box. A file per machine drifts out of sync fast. A store is one place, reachable from all of them.

3. Team-shared context. A CLAUDE.md in git is shared — but it shares stable facts you agreed to commit. The accumulated "we tried X, it broke Y" is exactly what a new teammate's agent would benefit from, and exactly what nobody writes into the committed markdown. A shared namespace in a store carries the long tail that never makes it into the file.

4. Volume past hand-curation. Beyond a certain size, CLAUDE.md becomes a wall of text that eats context on every session whether or not any line is relevant today. A store flips that: nothing loads until the agent runs a search and pulls back the top few results it actually needs. If it never searches, that costs zero context.

That last point is the real mechanical difference. A file is loaded. A store is queried. One is always in your context budget; the other is there only when the agent asks for it.

"But Claude Code already has auto memory"

It does, and it's genuinely useful — I use it. But it has a shape worth understanding before you assume it covers the same ground.

It's selective by design. The agent decides what's worth persisting. That's the right default — you don't want every stray thought in there — but it means the memory is only ever a subset of what happened. And you can't search what it chose not to write down. If it never recorded the OOM ceiling, there's nothing to recall later; you rediscover it the hard way.
It's per-tool. Claude Code's memory belongs to Claude Code. Cursor doesn't read it.
It's a file that loads into context, not a query. Mechanically it's closer to CLAUDE.md than to a searchable store: present whether relevant or not, rather than pulled on demand.

None of that makes it bad. It makes it the stable-facts-plus-a-bit layer, not the searchable-accumulation-across-tools layer. The two coexist happily: let Claude Code keep managing its file, and let a memory store hold the searchable long tail that spans your tools and machines.

If you decide you want the searchable layer

Here's what it looks like in practice, kept short and factual rather than pitched.

The agent gets a memory_save and a memory_search, plus memory_recall, memory_get, memory_update, memory_delete, memory_list_namespaces, and memory_health — eight tools total. It saves a decision or a gotcha when it judges one worth keeping; later, in any MCP client, it searches and gets the top few results back. Because it's an MCP server, "any client" includes both Claude Code and Cursor reading and writing the same store.

Two things I insisted on, because they're the parts that make me nervous as a user of other people's memory tools:

Everything is visible. There's a web dashboard where you can read, edit, and delete every memory. It is not a black box quietly accumulating who-knows-what.
Everything exports. Full JSON dump anytime — no lock-in. If the tool disappoints you, you leave with all your data.

On privacy, since it's usually the next question: the store keeps the memory text your tools explicitly send plus its metadata (namespace, tags, timestamps); the code index keeps derived embeddings and file paths, never your source. The full write-up is here: https://app.kireo.app/legal/privacy#data-storage.

Install is one line (free API key at https://app.kireo.app/api-keys):

claude mcp add kireo --scope user --env KIREO_API_KEY=ki_sk_xxx -- npx -y --package=@kireo/mcp-server kireo-mcp

The takeaway

Put stable, human-curated facts in a file the agent reads. Put agent-accumulated, searchable facts in a store the agent queries. If you only have the first kind of fact, you only need the first kind of tool — and that's a fine place to stop.

Disclosure: I build Kireo, the memory MCP server used in the examples above. It's a free beta right now — generous limits, no card required. If the honest read of your setup is "a good CLAUDE.md is enough," that's a completely legitimate place to land, and I'd rather you stop there than pay a complexity tax for a problem you don't have. The framework holds regardless of whose store you use.

*Repo: github.com/wang1051992187/kireo-mcp-server · npm: @kireo/mcp-server · site: https://kireo.app/?utm_source=devto&utm_medium=blog&utm_campaign=launch-202608

Building low-latency semantic memory for coding agents with LanceDB

Jonathan Wilcox — Tue, 04 Aug 2026 06:00:49 +0000

A coding agent starts each session cold. It has no idea what you decided last Tuesday, why you rejected the obvious approach, or which config value burned an afternoon. You can paste the context back in every time, or you can give the agent a memory it can query.

The second option sounds simple until you write it down as an engineering problem, because it has two constraints that pull against each other:

Retrieval has to be fast. Memory lookup sits inside the agent's tool loop. If a recall call adds a noticeable stall on top of everything else the agent is doing, people stop using it.
Retrieval has to be token-cheap. If pulling context back in costs more tokens than just re-explaining, the memory layer is a net negative. It has to earn its place in the prompt.

This post is about how we built that layer for Kireo, and — more usefully — the things that broke along the way. Honest framing up front: this isn't a scale-bragging post. It runs on one small VM, and the interesting part is how far a modest box goes when the retrieval path stays lean.

The token objection, head-on

The most common pushback on any agent-memory idea is: isn't this just going to bloat every prompt? Fair worry, and the answer is entirely in the design.

Memory is an MCP tool the agent calls on demand, not a blob injected into every turn. When the agent needs prior context it calls memory_search, gets back a small ranked set of hits (default 10, hard cap 50), and spends tokens only on those. Nothing is prepended to the system prompt; nothing runs on turns where the agent doesn't ask.

That distinction — pull, not push — is the whole reason the token math works. A per-turn injection scheme pays for memory on every message whether it helps or not. An on-demand top-k tool pays only when the model judges the context worth retrieving, and the budget is bounded by limit.

The server ships eight tools over MCP stdio — memory_save, memory_search, memory_recall, memory_get, memory_update, memory_delete, memory_list_namespaces, and memory_health — and every MCP client (Claude Code, Cursor, and others) sees the same set. But memory_search is the one that has to be both fast and frugal, so that's where the engineering went.

The stack: small on purpose

The whole retrieval path:

LanceDB — an embedded columnar vector store. No separate database server; it's a library that reads Lance-format files off a disk volume mounted into the API and worker containers. Memory content lives here.
Neon Postgres — holds metadata only (row bookkeeping, embedding status). It never stores the memory body. That split matters later.
Self-hosted embeddings — HuggingFace's Text Embeddings Inference (TEI) running intfloat/multilingual-e5-small, a 384-dimension model, on CPU. It speaks an OpenAI-compatible /v1/embeddings endpoint on the internal network.

Two of those choices are the load-bearing ones.

LanceDB is embedded, not a service. For a workload this size, a managed vector database is overkill. LanceDB is columnar on disk, the query path is a library call, no network hop. The vector column is an Arrow FixedSizeList(dim) — remember that, it comes back to bite us.

384 dimensions on CPU is enough, and it's cheap. Memory snippets are short — a decision, a gotcha, a config note. You don't need a 1536- or 3072-dim frontier model to separate "we chose Postgres row-level security over app-layer checks" from "the CI cache key needs the lockfile hash." Smaller vectors mean cheaper ANN and less storage, and a small e5 model does short-text semantic matching well on CPU. The one catch: e5-family models want asymmetric prefixes — passage: for stored documents, query: for search queries — supplied from config and empty for OpenAI-style models. (That innocuous trailing space caused a real bug; more below.)

A side effect of TEI's OpenAI-compatible endpoint: the same client code talks to the self-hosted model by pointing a base URL at the internal service — swapping providers was a config change, not a rewrite. The cache and rate-limit layers use the same trick: a self-hosted Redis fronted by a shim speaking the Upstash REST API, so the REST client needs no managed account.

Hybrid search, with a graceful-degradation ladder

Pure vector search misses exact-match cases (a specific error code, a function name). Pure keyword search misses paraphrase. So memory_search runs both and fuses them.

The flow:

Embed the query (cached, so repeated searches don't re-embed).
In parallel, run an ANN search over the vector column (top 50) and a native full-text search (top 50), both scoped to the caller's tenant.
Fuse the two ranked lists with reciprocal rank fusion and slice to the requested limit.

RRF is deliberately boring: it combines rankings by 1/(k + rank) without needing the two scoring systems to share a scale — cosine distance and BM25 don't. Robust and cheap, which is what you want in the hot path.

The interesting part is the degradation ladder: in a lean stack any dependency can be briefly unavailable, and search still has to return something:

Query embedding fails (provider hiccup, timeout)? Log a degraded event so it alerts, drop the vector arm, search keyword-only.
No FTS index (common in a fresh dev environment)? Fall back to a bounded case-insensitive substring scan ranked by importance.
Both arms empty? Return an empty result, not a 500.

None of that is glamorous, but it's the difference between "memory occasionally returns fewer hits" and "memory throws inside the agent's tool loop."

Namespaces and tenant isolation

Every row carries a user_id and a namespace (e.g. code-my-app for an indexed repo). Isolation is a user_id predicate pushed into every LanceDB query, plus a belt-and-suspenders assertion after results return: if any row's user_id doesn't match the caller, the code throws instead of leaking it. The filter should never be wrong — so we check anyway, on every read path. Cross-tenant leakage is the one bug you never want to ship, and a three-line assertion is cheap insurance.

War story 1: the embedding-dimension migration

The stack didn't start at 384 dimensions. It started at 1536 against a hosted model; moving to the self-hosted 384-dim e5 model meant every stored vector was now the wrong length.

Here's where LanceDB's FixedSizeList(dim) schema stops being an implementation detail. The vector column's dimension is baked into the table schema, and in the version we run there is no in-place column resize and no table rename. Once the dimension changes, every insert fails against the old table, and you can't quietly widen the column.

The migration is dump-drop-recreate:

Read every row out of the memories table.
Write a durable JSON dump to disk before anything destructive — memory content lives only in LanceDB (Postgres has metadata only), so if the recreate dies halfway, that dump is the only copy. Old vectors are dropped on reload anyway, so the dump excludes them.
Drop the table, recreate it empty at the new dimension.
Re-insert every row in batches with embedding = null and embedding_status = 'queued', then reset the Postgres status rows to queued too so the backfill job re-embeds everything with the new model.

The migration is idempotent — if the table is already at the target dimension it's a no-op — which matters when you're running it by hand on a live box, unsure whether the last attempt finished.

And there's a subtle second-order bug this exposed. The embedding cache is keyed by model + content hash — but not by dimension. So after a same-model endpoint change that alters the vector length, a stale cached vector of the old length would sail past the cache lookup and get written into the freshly recreated table, breaking the insert. The fix is a dimension guard at two layers: the embed client asserts vector.length === EMBEDDING_DIM before returning (a wrong-length embed fails and degrades to the queue instead of corrupting the table), and the cache read treats a length mismatch as a miss. The rule: never let a wrong-dimension vector reach the table, enforced at every point one could enter.

War story 2: making batch writes idempotent

Indexing a repo uploads symbols in batches — up to 100 per request. Batches time out sometimes, and the obvious retry (re-send the batch) creates duplicates unless the write path is idempotent.

The fix is content-hash dedup as a single set query, not N point lookups. Before inserting a batch, one query fetches the active (non-deleted) rows whose content_hash is in the batch's hashes — content_hash IN (...) for the whole batch — and skips them. Re-running the same upload is safe: identical content hashes to identical rows, retries don't multiply.

Two things I like here. First, one IN query keeps dedup off the per-item hot path — one query per batch, not one per symbol. Second, it's the same guarantee surfaced in the CLI docs: if a batch upload times out, re-running the same command is safe. Not an internal nicety but a documented contract — the person hitting the timeout is the one who needs to trust the retry.

War story 3: what "trash" actually means

Delete is where naive implementations quietly lose data or lie about counts. Our fixes here were all, at heart, about semantics.

Deletes are soft by default: a delete stamps deleted_at and sets a 30-day expires_at restore window. The row stays in the table, filtered out of normal reads by deleted_at IS NULL. Restore checks the window and refuses if it's expired; a TTL sweep physically removes rows past expires_at.

That created three follow-on requirements that each needed explicit handling:

A trash view needs the inverse filter. Listing deleted items isn't "include deleted" — it's "only deleted." Different queries, and "only deleted" has to win when both flags are set. Get it subtly wrong and the user sees an empty or wrong trash.
Counts have to match the view. The "trashed" count is its own query, not total − active, because off-by-a-little count math is exactly what users notice and stop trusting.
Updates are in-place, never delete-then-add. LanceDB will let you delete and re-add a row, but a crash between those calls permanently loses it while its Postgres metadata survives — a torn write. Every mutation (edit, soft-delete, restore, namespace rename) is an in-place update, so there's no window where the row doesn't exist.

One more LanceDB-shaped wrinkle: a plain scan has no ORDER BY. To list newest-first without materializing a million rows, the list path streams every matching batch, re-sorting as they arrive and truncating to limit + 1 so only the current top page stays in memory. It's more code than ORDER BY ... LIMIT, but it's what the storage engine actually supports.

The unglamorous infrastructure footnotes

Three more that belong in any honest "one VM" story:

A single log file once grew to 62 GB. A worker got stuck in an error loop against an exhausted upstream and its JSON logs ate the disk. The fix is boring and permanent: every container caps log rotation at 10 MB × 3 files. On a small box, an unbounded log is a time bomb.
The embedding image is pinned by digest. An earlier CPU tag of TEI bundled a dependency version that failed model download with a cryptic "relative URL without a base." It's now pinned to a known-good digest, with a comment saying exactly why, so an innocent :latest pull can't resurrect the bug.
Config whitespace is load-bearing. Docker's dotenv parsing trims trailing whitespace, which silently ate the trailing space in the e5 passage: prefix and quietly degraded recall until the value was quoted. The kind of bug that has no stack trace.

None of these are clever. They're the tax for running a real workload on modest hardware — and writing them down means the next person (often me, three months later) skips the debugging.

Where this runs

This is the memory layer behind Kireo, an MCP server that gives Claude Code, Cursor, and any MCP client one shared long-term memory — save decisions and gotchas as you work, recall them from any tool, and browse, edit, or delete everything in a web dashboard, with full JSON export if you ever want to walk away with your data. It's in free beta right now with generous limits and no card required.

Install is one line:

claude mcp add kireo --scope user --env KIREO_API_KEY=ki_sk_xxx -- npx -y --package=@kireo/mcp-server kireo-mcp

Grab a key and read the docs at kireo.app. On privacy: the server only sends what you explicitly pass to memory_save; it never reads your code, and code indexing stores derived embeddings and file paths, not your source — see the data-storage policy. The MCP server is on npm and GitHub.

I've kept latency claims out on purpose — I'd rather ship measured numbers than round ones, and a follow-up profiling the search path is on the list. If you build agent memory on a small stack and hit a different set of walls, I'd like to hear which ones.