Is Agent Memory Just RAG With Extra Steps? We Opened the Source Code to Find Out

Under every agent memory launch, the same comment appears: "so it's RAG with extra steps." Instead of arguing, we opened the shipping source of mem0, LangGraph, Graphiti and Generative Agents at pinned commits and read the actual read and write paths. The answer is more interesting than either side of the flame war.

Disclosure up front: I work on Mnemoverse, a memory layer for AI agents, so this is a vendor reading competitors' code, and you should weigh it accordingly. Two rules held throughout: in the full version every code claim cites file, lines and a pinned commit, and this short tour names file and commit wherever it quotes code; and there is not a single performance number, ours or anyone's. The full version with all 40 numbered claims and sources lives in our library; this is the short tour.

At read time, the skeptic is right

When a memory layer answers a query, here is mem0's shipping search path (mem0/memory/main.py, commit 001c235): embed the query, over-fetch from a vector store, run a keyword search alongside, fuse the scores, optionally rerank.

# Step 2: Embed query
embeddings = self.embedding_model.embed(query, "search")

# Step 3: Semantic search (over-fetch for scoring pool)
internal_limit = max(limit * 4, 60)
semantic_results = self.vector_store.search(
    query=query, vectors=embeddings, top_k=internal_limit, filters=filters
)

# Step 4: Keyword search (if store supports it)
keyword_results = self.vector_store.keyword_search(
    query=query_lemmatized, top_k=internal_limit, filters=filters
)

That is a textbook hybrid retrieval pipeline. If you have built retrieval, you have built this. LangGraph is even more explicit: long-term memory there is a JSON document under a namespace and a key, and semantic search is opt-in. And Anthropic ships a first-party memory tool that is six file commands with no embeddings and no scoring at all: a filesystem, from the company that makes the model.

So the concession, said plainly because our category usually does not say it: for mem0, LangGraph and Anthropic's tool, the ranking at read time is retrieval and nothing else. Generative Agents, the fourth repository we opened, is the partial exception: its read path sums cosine relevance with a recency term and an importance score the model assigned at write time, weighted 3, 0.5 and 2 in the shipping code, and that is still a scoring function over a stored index rather than a second kind of read. If somebody drew you two different diagrams for how candidates get ranked, one of them was fiction. What differs is what reading does to the record, and that comes below.

The difference lives on the write path

Not every memory layer does more than store. LangGraph's put() writes your dict as it arrived, with no model in the path, and Anthropic's tool writes a file. In the systems that do more, mem0 with inference on, Graphiti and Generative Agents, up to three things happen on write that do not happen when you index a document. A model judges whether the thing deserves to be stored at all. The new item is reconciled against what is already there, as mem0 and Graphiti do. And the stored unit can carry a validity interval separate from when the system learned it.

The third one is worth seeing in code. In Graphiti (Zep's engine, a competitor of ours), every entity edge carries five datetime fields; three of them are enough to show the idea (graphiti_core/edges.py, commit 96ef997):

expired_at: datetime | None = Field(
    default=None, description='datetime of when the node was invalidated'
)
valid_at: datetime | None = Field(
    default=None, description='datetime of when the fact became true'
)
invalid_at: datetime | None = Field(
    default=None, description='datetime of when the fact stopped being true'
)

Two axes: when the system learned something, and when it was true in the world. State "we deploy on Fridays" in March, contradict it in June, and the March record is still there, now saying "this used to be true, and here is when it stopped." Your agent can explain March's decisions after June's migration. On contradiction, nothing is deleted, so nothing has to be re-derived from a stale summary.

Can you build the same thing with timestamps in your RAG metadata? Yes, and it works. The honest difference is where the reconciliation code lives: with metadata it is query-time logic in your application, maintained by you, growing a branch per conflict type; in a memory layer it happens once at write time in code you do not own. What is amortised, not what is achievable. Anyone selling you a capability gap here is selling something that is not there.

Why agents remember the wrong things

Here is the part that made us uncomfortable. A memory layer reads from a corpus it wrote itself. Extraction writes memories, retrieval reads them, and the output feeds the next extraction. A RAG pipeline can retrieve badly, but it cannot corrupt its source: your documents were written by people, elsewhere. A memory store's extraction errors become its own retrieval corpus.

Now add the reinforcement schemes. In every reinforcement scheme we opened, retrieval is what keeps a memory alive. Generative Agents' scored retrieval rewrites last_accessed on every node it returns (retrieve.py, commit fe05a71). MemoryBank, by its own paper, increments a memory's strength and resets its clock on recall. LangGraph ships with expiry switched off, but once an item has a time-to-live, refreshing it on every read is the default (libs/checkpoint/langgraph/store/base/__init__.py, commit 644815f):

class TTLConfig(TypedDict, total=False):
    """Configuration for TTL (time-to-live) behavior in the store."""

    refresh_on_read: bool
    """Default behavior for refreshing TTLs on read operations (`GET` and `SEARCH`).

    If `True`, TTLs will be refreshed on read operations (get/search) by default.

Put together: a wrong fact that keeps getting retrieved is protected by the exact signal that was supposed to prune it. For a corpus the system did not write, "frequently used means valuable" is reasonable. For a corpus the system wrote itself, it is a feedback loop with the sign pointing the wrong way. That is the mechanism behind the complaint under every memory launch: it remembers the wrong details and applies them in the wrong places.

Where our own system fails this test

Our shipped write gate scores novelty against the nearest existing memory in the same domain. It does not judge importance or factuality, which makes the name we gave it, an importance gate, wrong: the API field is called importance, the only refusal the core emits reads "Below importance threshold", and the number behind both is novelty. The consequence, in a personal domain, which is where a write lands by default: a correction is by nature phrased almost exactly like the thing it corrects, so it scores as a near-duplicate, so it is the one input most likely to be rejected. The stale fact survives as the sole record and, having no competitor in the store, looks more authoritative than it should. That is in our 0.8.1 changelog as a user-facing known defect. We have not solved it. Since 22 August the REST write accepts a field that names the atom it corrects, and a write declared that way skips the refusal; but the old atom is only marked, not hidden, and still competes in reads, a correction phrased like the original and not declared as one is still rejected on every surface, and the MCP tools carry no such field yet. In a shared room the gate refuses only an exact duplicate, so the same correction stores there; that is a different rule, not a fix.

Three checks you can run this afternoon

On any vendor, including us. None of them is a sales-call question.

1. Open the read path. Find the search function and read it: an embed call, an over-fetch, a keyword branch, a fusion step means retrieval. That is fine. It is only a problem if the pricing claims it is something else.
2. Store a fact, contradict it, list everything. Original still there with a closed validity interval: Graphiti-style honesty. Present but hidden: demotion, not forgetting. Gone or silently rewritten: you lost the audit trail.
3. Submit a correction phrased like the original, in a personal domain. If it is not in the store afterwards, the write gate ate it and the wrong fact is now the only record. This is where we fail today in a personal domain, in public, in our changelog. Run it on us.

The whole argument, animated mechanism by mechanism:

---

Disclosure: I work on Mnemoverse, a memory engine for AI agents. This article is the short version of the original in our library, which carries all numbered claims, the paper-vs-code table, and full sources at pinned commits: RAG vs agent memory: what the source code actually shows. Code excerpts reproduced for commentary under the upstream licenses (mem0, Graphiti, Generative Agents: Apache-2.0; LangGraph: MIT).