Under every agent memory launch, the same comment appears: "so it's RAG with extra steps." Instead of arguing, we opened the shipping source of mem0, LangGraph, Graphiti and Generative Agents at pinned commits and read the actual read and write paths. The answer is more interesting than either side of the flame war.
Disclosure up front: I work on Mnemoverse, a memory layer for AI agents, so this is a vendor reading competitors' code, and you should weigh it accordingly. Two rules held throughout: in the full version every code claim cites file, lines and a pinned commit, and this short tour names file and commit wherever it quotes code; and there is not a single performance number, ours or anyone's. The full version with all 40 numbered claims and sources lives in our library; this is the short tour.
At read time, the skeptic is right
When a memory layer answers a query, here is mem0's shipping search path (mem0/memory/main.py, commit 001c235): embed the query, over-fetch from a vector store, run a keyword search alongside, fuse the scores, optionally rerank.
# Step 2: Embed query
embeddings = self.embedding_model.embed(query, "search")
# Step 3: Semantic search (over-fetch for scoring pool)
internal_limit = max(limit * 4, 60)
semantic_results = self.vector_store.search(
query=query, vectors=embeddings, top_k=internal_limit, filters=filters
)
# Step 4: Keyword search (if store supports it)
keyword_results = self.vector_store.keyword_search(
query=query_lemmatized, top_k=internal_limit, filters=filters
)
That is a textbook hybrid retrieval pipeline. If you have built retrieval, you have built this. LangGraph is even more explicit: long-term memory there is a JSON document under a namespace and a key, and semantic search is opt-in. And Anthropic ships a first-party memory tool that is six file commands with no embeddings and no scoring at all: a filesystem, from the company that makes the model.
So the concession, said plainly because our category usually does not say it: for mem0, LangGraph and Anthropic's tool, the ranking at read time is retrieval and nothing else. Generative Agents, the fourth repository we opened, is the partial exception: its read path sums cosine relevance with a recency term and an importance score the model assigned at write time, weighted 3, 0.5 and 2 in the shipping code, and that is still a scoring function over a stored index rather than a second kind of read. If somebody drew you two different diagrams for how candidates get ranked, one of them was fiction. What differs is what reading does to the record, and that comes below.
The difference lives on the write path
Not every memory layer does more than store. LangGraph's put() writes your dict as it arrived, with no model in the path, and Anthropic's tool writes a file. In the systems that do more, mem0 with inference on, Graphiti and Generative Agents, up to three things happen on write that do not happen when you index a document. A model judges whether the thing deserves to be stored at all. The new item is reconciled against what is already there, as mem0 and Graphiti do. And the stored unit can carry a validity interval separate from when the system learned it.
The third one is worth seeing in code. In Graphiti (Zep's engine, a competitor of ours), every entity edge carries five datetime fields; three of them are enough to show the idea (graphiti_core/edges.py, commit 96ef997):
expired_at: datetime | None = Field(
default=None, description='datetime of when the node was invalidated'
)
valid_at: datetime | None = Field(
default=None, description='datetime of when the fact became true'
)
invalid_at: datetime | None = Field(
default=None, description='datetime of when the fact stopped being true'
)
Two axes: when the system learned something, and when it was true in the world. State "we deploy on Fridays" in March, contradict it in June, and the March record is still there, now saying "this used to be true, and here is when it stopped." Your agent can explain March's decisions after June's migration. On contradiction, nothing is deleted, so nothing has to be re-derived from a stale summary.
Can you build the same thing with timestamps in your RAG metadata? Yes, and it works. The honest difference is where the reconciliation code lives: with metadata it is query-time logic in your application, maintained by you, growing a branch per conflict type; in a memory layer it happens once at write time in code you do not own. What is amortised, not what is achievable. Anyone selling you a capability gap here is selling something that is not there.
Why agents remember the wrong things
Here is the part that made us uncomfortable. A memory layer reads from a corpus it wrote itself. Extraction writes memories, retrieval reads them, and the output feeds the next extraction. A RAG pipeline can retrieve badly, but it cannot corrupt its source: your documents were written by people, elsewhere. A memory store's extraction errors become its own retrieval corpus.
Now add the reinforcement schemes. In every reinforcement scheme we opened, retrieval is what keeps a memory alive. Generative Agents' scored retrieval rewrites last_accessed on every node it returns (retrieve.py, commit fe05a71). MemoryBank, by its own paper, increments a memory's strength and resets its clock on recall. LangGraph ships with expiry switched off, but once an item has a time-to-live, refreshing it on every read is the default (libs/checkpoint/langgraph/store/base/__init__.py, commit 644815f):
class TTLConfig(TypedDict, total=False):
"""Configuration for TTL (time-to-live) behavior in the store."""
refresh_on_read: bool
"""Default behavior for refreshing TTLs on read operations (`GET` and `SEARCH`).
If `True`, TTLs will be refreshed on read operations (get/search) by default.
Put together: a wrong fact that keeps getting retrieved is protected by the exact signal that was supposed to prune it. For a corpus the system did not write, "frequently used means valuable" is reasonable. For a corpus the system wrote itself, it is a feedback loop with the sign pointing the wrong way. That is the mechanism behind the complaint under every memory launch: it remembers the wrong details and applies them in the wrong places.
Where our own system fails this test
Our shipped write gate scores novelty against the nearest existing memory in the same domain. It does not judge importance or factuality, which makes the name we gave it, an importance gate, wrong: the API field is called importance, the only refusal the core emits reads "Below importance threshold", and the number behind both is novelty. The consequence, in a personal domain, which is where a write lands by default: a correction is by nature phrased almost exactly like the thing it corrects, so it scores as a near-duplicate, so it is the one input most likely to be rejected. The stale fact survives as the sole record and, having no competitor in the store, looks more authoritative than it should. That is in our 0.8.1 changelog as a user-facing known defect. We have not solved it. Since 22 August the REST write accepts a field that names the atom it corrects, and a write declared that way skips the refusal; but the old atom is only marked, not hidden, and still competes in reads, a correction phrased like the original and not declared as one is still rejected on every surface, and the MCP tools carry no such field yet. In a shared room the gate refuses only an exact duplicate, so the same correction stores there; that is a different rule, not a fix.
Three checks you can run this afternoon
On any vendor, including us. None of them is a sales-call question.
- Open the read path. Find the search function and read it: an embed call, an over-fetch, a keyword branch, a fusion step means retrieval. That is fine. It is only a problem if the pricing claims it is something else.
- Store a fact, contradict it, list everything. Original still there with a closed validity interval: Graphiti-style honesty. Present but hidden: demotion, not forgetting. Gone or silently rewritten: you lost the audit trail.
- Submit a correction phrased like the original, in a personal domain. If it is not in the store afterwards, the write gate ate it and the wrong fact is now the only record. This is where we fail today in a personal domain, in public, in our changelog. Run it on us.
The whole argument, animated mechanism by mechanism:
Disclosure: I work on Mnemoverse, a memory engine for AI agents. This article is the short version of the original in our library, which carries all numbered claims, the paper-vs-code table, and full sources at pinned commits: RAG vs agent memory: what the source code actually shows. Code excerpts reproduced for commentary under the upstream licenses (mem0, Graphiti, Generative Agents: Apache-2.0; LangGraph: MIT).
Top comments (14)
Same disclosure as yours: I work on a memory layer for coding agents, so this is a competitor answering. I'm answering anyway because the third test is the best thing in the post and I owe you the result of running it.
Test 3, on us: our write gate does not eat the correction - and the reason is a trade you don't discuss.
We deduplicate by exact topic key, not by similarity. A lesson filed under the same slug updates the existing one and keeps an audit trail; a contradicting outcome raises a warning rather than a rejection. So the failure you documented cannot happen to us in that shape: a correction phrased almost identically to the original arrives under the same key and lands.
I ran it live today, not as a demo. I had stored a wrong fact this morning ("four workspaces"), found out from a human it was wrong ("there are eight"), and filed the correction under the same slug. It updated, audit entries went to 2, the old text is still recoverable. Your near-duplicate gate would have rejected it - mine took it because it never looked at the text.
But that's not a better design, it's a different one, and the bill comes due elsewhere: similarity dedup eats corrections, key dedup admits duplicates. Misspell the slug and you have created a second truth rather than fixing the first, and nothing anywhere objects. Ours has no mechanism to notice that deploy:cache and deploy:caching are the same subject. Yours would have caught that instantly and refused. Which makes me think the honest framing is not "which gate is right" but which failure you'd rather be able to see - and both are invisible by default.
Have you considered admitting the correction and letting the contradiction be the stored object? That's what Graphiti's interval buys, one level up: not "which of these two is true" at write time, but "these two disagree, here's when each was asserted." It moves the judgement to read time where the querent has context the writer didn't.
On the self-corpus point - I think it's more general than agent memory, and that makes it worse.
Your framing is that a memory store reads from a corpus it wrote itself, while RAG reads from documents people wrote elsewhere. Right, and it doesn't need a model in the loop to bite. I shipped a monitoring rule two days ago that searches a log store for a pattern. The log store writes every query it runs into its own log. The log shipper collects that log. So the rule found the record of itself asking, and counted it as evidence. Measured with a token nothing in the system emits, same query every 12 seconds:
0 → 6 → 12 → 4 → 8
Threshold was 5. Empty world, no agent, no extraction, no reinforcement - just a reader and a writer sharing one surface. Which suggests the property that matters isn't "a model wrote the corpus" but "the act of reading is itself writable to the thing being read". Extraction errors are one instance; my diagnostics were another; your reinforcement loop is a third and by far the nastiest, because it has a sign.
One addition to test 1, learned the expensive way this week. "Find the search function and read it" assumes there is one. We had scoping enforced correctly on retrieval, tested, verified - and a second read path, lookup by exact name rather than similarity, that returned full content to anyone who asked. Same store, two doors, one guarded. Nobody had counted the second as a door because it didn't look like search.
So I'd make it plural: find every function that can return stored content. Search, exact get, list, export, whatever the admin surface calls, the debug endpoint added for a demo. The tell that generalises is the signature - a read path that doesn't accept the caller identity cannot enforce anything, and that greps in a way that behaviour doesn't.
And a concession, since you made one. Our confidence decays without recall - 1.0, then 0.7 after five days, then 0.5. Which is exactly the sign error you name: a wrong lesson that keeps getting recalled sits at 1.0 forever, protected by the signal meant to prune it. Recall is not evidence of correctness, it's evidence of being findable, and we treat the two as the same number. I don't have a fix; the honest version would need recall to be distinguishable from confirmed useful recall, and nothing in our data separates them today.
Which is really your point back at you: reading is doing something to the record, in every one of these systems, and mostly in the direction nobody chose.
Your second finding makes my framing narrower than the thing it describes. I treated the model as the mechanism. Your monitoring rule has no model in it and reproduces the whole failure: a reader and a writer sharing one surface is enough, and 0, 6, 12, 4, 8 against a threshold of 5 is the cleanest statement of it I have seen. The property is that the act of reading is writable to the thing being read. Extraction is one instance, your log shipper is another, reinforcement is the worst because it carries a sign.
On the two gates I think we both framed it wrong, myself included. Dedup and contradiction are not two ends of one scale. They are two outcomes of the same job: find records that are about the same subject, then classify what relation holds. Similarity is not the verdict there, it is admission to the proceedings. Once a pair is admitted, the possible verdicts include duplicate, support, contradiction, supersession, alias, and unrelated after all. Your key gate skips the admission step and goes straight to identity; my similarity gate performs the admission and then mistakes closeness for identity. Same missing middle, two different places to fall through it.
Which is why contradiction cannot be found by distance. Opposites sit close in embedding space precisely because they share subject, predicate and context and differ in one term. A gate tuned to reject near-duplicates will reject the correction hardest when the correction is most exact.
Your read-time question has a 1969 answer that I keep forgetting is available. Fellegi and Sunter defined three outcomes for record linkage, not two: link, non-link, and possible link, where the third is a full result rather than an unresolved error. That is what a store owes the querent: both records, their assertion times, and an honest label saying the system could not decide. And entity resolution has to come first, because "he went" against "he did not go" is only a contradiction after something has established that both sentences are about the same person and the same day.
Your last paragraph is the expensive one. Recall is evidence of being findable, not of being right, and a system that treats them as one number protects its worst records with the signal meant to prune them. We have the same hole with a different curve. Two independent stores failing in the same place is worth more than either of us claiming to have closed it.
Fellegi and Sunter is the reference I should have known and didn't, and the 1969 date is the uncomfortable part - we rebuilt a worse version of a solved problem and called the missing third outcome an edge case.
Your reframing is right and it dissolves my gate too. Admission and verdict are different jobs, and I collapsed them from the other side: my key gate skips admission entirely, so it never even convenes the proceeding. Yours convenes it and then lets the strongest signal cast the vote. Same missing middle.
The consequence I hadn't drawn: an admitted pair with no verdict is a result, not a failure. We treat "couldn't decide" as an error path - logged, swallowed, invisible. As a first-class outcome it becomes something a querent can act on, and something we can count. A rising possible-link rate would be the earliest signal that a corpus has started drifting, and right now we'd read that same drift as "recall is fine."
On contradiction and distance - "a gate tuned to reject near-duplicates will reject the correction hardest when the correction is most exact" is the sentence I'll be quoting back at my own code. It also explains a result we couldn't explain: we measured whether top-k contains both sides of a contradiction and got a number low enough that we assumed a retrieval bug. It wasn't. The ranker was doing exactly what it was built to do.
And you're right about the last paragraph being the expensive one. We have a hit-rate number that we've been treating as a quality number for months, and the only honest thing I can say is that it measures findability. Two independent stores with the same hole is worth more than either of us patching it quietly - so: do you have a way to observe the possible-link band today, or would you have to build the classifier first to see it at all? That's the order-of-work question I can't answer for our side.
Heinrich, the possible-link rate as a drift signal is the part I would build first, and I think it is reachable before the classifier exists.
The band does not need a verdict function in order to be counted. It needs an admission rule and a way to see how close the contenders are. Admission you already have, and yours is stricter than anything derived: the exact topic key is a subject identity. For every key holding more than one record, take the score of the top candidate and the runner-up and look at the gap. Where that gap sits inside the noise of your own scorer, the pair is admitted and undecided. That is the band, and counting it needs nothing you do not already run.
So the order of work runs counter first, classifier second. The other way round means shipping a verdict function with no baseline to judge it against, and no way to separate a real improvement from a shift in the corpus underneath it.
On the hit rate, I would keep the two numbers apart rather than let one replace the other. Findability answers whether the record can be reached at all. The band answers how often reaching it is not enough. A system can improve on the first and get worse on the second in the same week, and that is exactly the case a single quality number cannot report.
I re-ran this before quoting any numbers. On OpenAI text-embedding-3-small, "I don't like black, I like white" vs "I like black, I don't like white" — semantic opposites — score cosine 0.955, while a genuine paraphrase of the same preference scores 0.821. Same ordering holds on BGE-M3 and text-embedding-3-large, in two languages, and "Alice loves Bob" vs "Bob loves Alice" sits at 0.85–0.97. A similarity threshold cannot tell a correction from a repeat: the correction shares more surface with the mistake than an honest restatement does — so a dedup gate eats corrections preferentially, and the more precise the correction, the more surely it dies.
None of this says write-time filtering is wrong. At unbounded-memory scale you have to filter at the door. It says cosine alone is the wrong doorman: the detector has to see polarity and argument binding, not word overlap.
The 0.955 vs 0.821 pair is the cleanest statement of the problem I have seen, and it lands on us directly: the possible-link relation you quoted from my article is computed with cosine. Two embedding models score each new lesson against the existing ones, and when both agree the entries are close, a link is recorded. So your measurement says something specific about our own door: if that link had been allowed to decide "redundant, drop it", the correction would have been the first thing to die, because it is by construction the sentence closest to the mistake it corrects.
That is why the link never deletes. A near-duplicate at write time is treated as a signal that a relation exists, not that one copy is redundant. Both stay, the newer one carries a pointer to the one it replaces, and the reader sees both with the pointer. The cost moves from the door to the read, where the model can at least look at the two texts side by side instead of at a scalar. Your numbers are the best argument for that rule I have, and I did not have them when I wrote the piece.
I should also say plainly what this thread did. Between your comments and the others here, we went back and re-measured the assumptions the article was built on, wrote down what we expected before running anything, and changed how the system behaves at write and read time. Several of the things I would have defended a month ago did not survive that. I would rather say that here than pretend the article was already right.
Where I am still unsure: for the detector you describe, one that sees polarity and argument binding, is that an extra model call per write in your setup, or a cheaper classifier over the pair? The per-write model call is the line where memory stops being cheaper than re-reading the transcript, and I have not found a way around it that I trust.
Heinrich, the honest answer to the cost question is that per write is the wrong denominator. The cosine gate you already run is an admission filter, so the pair check only needs to fire on writes that land close to an existing entry, and that is a small fraction of the stream, the exact fraction your near-duplicate rate already measures. The cost scales with how often repeats and corrections actually occur, not with how much the agent writes. And the check itself does not need a full model call. Polarity plus argument binding is what small NLI-class classifiers were built for: entailment against the nearest neighbor, contradiction as the signal that this is a correction rather than a copy. That is cheap enough to run on every admitted pair, and wrong in ways that stay visible, because in your design the pair is stored either way. Which is the part I would keep whole: both texts stay, the pointer records the relation, the reader sees the pair side by side. A detector that only decides which relation to write can afford to be cheap, since being wrong costs a mislabeled edge, not a lost correction. What runs behind my own door I will keep at the level the docs state, but the shape above is not a secret, it is arithmetic: condition the expensive check on the cheap gate you already have, and the line where memory stops being cheaper than re-reading the transcript moves out to a write rate no working agent reaches. And the paragraph about re-measuring your own assumptions and changing the system is the best thing anyone has written in this thread, mine included.
Followed up with numbers, because the arithmetic above deserved a measurement. On the same pairs where cosine folds opposites together at 0.95 plus similarity, a small multilingual NLI cross-encoder, about 280M parameters, public, labels the contradiction at 0.999, in both directions and in two languages, while genuine paraphrases come back entailment and unrelated text comes back neutral. The check costs 37 ms on CPU, no GPU involved. One honest limit: role swaps, Alice loves Bob against Bob loves Alice, read as neutral rather than contradiction, because argument binding shows up as absence of entailment, a weaker signal than polarity. This is six probe pairs, not an eval, and I would not dress it as one. But the arithmetic holds: condition this check on the cheap similarity gate you already run, and the detector costs milliseconds on the fraction of writes that actually need it. And the pairs are the ones already quoted upthread, so checking the run takes one file and a public model, nothing of mine.
I ran this at a different unit size and got a different answer, which I think is about the unit rather than the model.
Corpus: 11 real supersession pairs from a public agent-memory benchmark, where a later session genuinely retires an earlier one. Model: the multilingual NLI cross-encoder class you named, ~280M, public. Unit: the 1600-character chunks the store actually holds.
Real pairs came back at a median contradiction score of 0.966, eight of eleven above 0.9. That much matches. The control is where it came apart - the superseded session of one cell against the valid session of a different cell, no relation between them at all. Median 0.818, five of eleven above 0.9. At a 0.9 threshold that is 8 true against 5 false, on a store where almost no pair is a supersession.
I picked that control deliberately, and it is worth saying why: candidates reach the detector through the cheap similarity gate, so they are always same-domain. A looser control would have scored better and told me less.
Cost at this unit size was 9.0 seconds per pair on twelve CPU cores. Not a contradiction of your 37 ms - that is what happens when a sentence becomes a chunk.
And the caveat you put on yours applies to mine: eleven pairs is not an eval either. What I would not conclude from it is that the shape is wrong. Condition the expensive check on the cheap gate, store both texts, let the pointer carry the relation - that all still holds. It is the detector at chunk size that did not separate.
The unit is the finding, and it is the more useful of the two results. My reading of the 0.818 is that at 1600 characters a sentence-pair model is rewarding shared topic rather than one statement retiring another, and there is a checkable reason it might: models of that class read 512 tokens, and two chunks of that size do not fit in one pass, so under the model card's own truncation part of the pair is never read. Your control is the right one for exactly the reason you give: the cheap gate guarantees same-domain candidates, so any control that is not same-domain would score better and tell you less. The 9 seconds per pair fit the same reading: a few hundred positions attending to each other, spent on a prefix of the pair.
The shape survives if the detector stops seeing chunks. Split each chunk into sentences, align across the pair by the same similarity you run at the gate, keep the top match per sentence only when it clears the gate's own threshold, and run the cross-encoder on those aligned sentence pairs. The verdict for the chunk pair is then the strongest contradiction among aligned pairs, not a score over the whole passages. The expectation under test is that a pair sharing only a domain aligns on topic and yields no aligned pair that contradicts. On your unit that is about a dozen sentence pairs per chunk pair, a dozen times the per-pair figure from earlier in this thread, and every pair fits inside the window.
The test I would run is your same eleven, sentence-aligned, with the same 0.9 line, and it works only if the control's highest score falls under that line; the true-pair median already clears it at 0.966.
The fourth test is the only one that actually matters and it is also the one no memory layer vendor owns. You can build a store that keeps both versions, labels them honestly, and still have zero line of sight into which one the downstream agent used. The receipt requires instrumenting the selection step, not the storage step, and that is a different system entirely. Most of the category is building the first three tests and calling the product done.
The line you draw between the storage step and the selection step is the right one, and the code we opened puts it in a stranger place than a different system entirely. Generative Agents already writes on selection: its scored retrieval rewrites last_accessed on every node it returns, in retrieve.py at the pinned commit. That is a write executed at the moment of choosing, shipping today. It just points at the record instead of at a log the caller can read. The same holds wherever a fusion step exists: the number that decided the order is computed at selection time, and whether it reaches the caller is a return shape decision rather than a missing capability.
Where your point survives, and I think this is the real boundary, is one step further in. Knowing which records the store returned and on what basis is not knowing which one the answer rested on. The store can hand over its ranking honestly and the model can still ignore the top result and paraphrase the third. No store closes that from the inside, because the evidence lives in the generation and not in the retrieval.
So the fourth test splits in two, and the halves cost differently. Which candidates were returned and on what basis is instrumentation the read paths already half perform. Which one the agent acted on has to be observed from outside the store. The first is a logging decision that vendors could make this week and mostly have not. The second is the different system you named, and it is narrower than the whole selection step.
The correction test is especially useful because it exposes a governance problem, not just a retrieval problem. I’d add one more check: after a contradiction, ask which version is authoritative for a specific user, role, and decision time. Preserving both facts is necessary, but the system also needs provenance, permission scope, an explicit supersession rule, and a decision receipt showing why one version was used. Otherwise the audit trail exists, but the agent can still act on the wrong evidence.
Permission scope is the part that turns this from a data-model question into an operational one, and it is where a design that looks finished stops being finished. In a parallel thread someone described an append-only log where nothing is deleted and retraction is itself a decision, which removes the dangling-pointer failure entirely. It does not remove yours. The superseding record can be present in the log and unreadable by the person asking, and from their side that is the same as it not being there.
The decision receipt is the piece almost nobody ships, us included. Preserving both versions makes the contradiction inspectable afterwards. It does not tell you which one the agent acted on, or why, at the moment it acted. Without that, an audit trail proves the system could have been right rather than that it was.
So the test has a fourth step. Store a fact, contradict it, see what comes back, and then ask the system which version it used and on what basis. A store that cannot answer the fourth question is one where the first three were a rehearsal.