I graded my memory server next to Mem0's published answers. It's a tie — and here is how Mem0 gets to 92.5.

Every memory product for AI agents has a LoCoMo number now. Mem0 says 92.5, Zep says 94.7, ByteRover 92.2. My own project, total-agent-memory (TAM), scored 87 on held-out LoCoMo conversations last week, and I spent a day trying to close the gap.

The honest result: I could not beat those numbers the way they are produced. What I could do was put TAM and Mem0 on the same questions, under the same judge, and see who is actually ahead. That turned out to be possible because Mem0 does something the others don't — it publishes the per-question answers behind its figures.

The comparison

Mem0's benchmark repository (mem0ai/memory-benchmarks) ships results/platform/locomo_results.json and longmemeval_results.json: every question, the answer its platform generated with gpt-5 from the top 200 memories, and the judge's verdict.

I took those answers, TAM's answers to the same held-out questions, and graded both with one judge model and two judge prompts:

• the published judge — the LoCoMo prompt Zep and Mem0 published and everyone used until this year, and the official LongMemEval evaluator;
• Mem0's current judge prompt, straight from their repository.

Held-out questions	LoCoMo (1,144), published judge	LoCoMo, Mem0 judge	LongMemEval-S (400), official judge	LongMemEval-S, Mem0 judge
Mem0 Platform (gpt-5 answering, top 200)	88.46	94.32	91.00	91.75
TAM (gpt-5 answering)	86.54	94.23	92.25	90.75
TAM (gpt-4.1-mini answering)	88.02	94.32	87.50	88.25

No difference between the two systems at the same answering model is statistically significant (paired sign test, p ≥ 0.1; bootstrap intervals in the report). It is a tie on both benchmarks. On LongMemEval TAM is ahead on preference questions (92 vs 76) and multi-session questions (86 vs 82), behind on knowledge updates (94 vs 97).

The part I care about: TAM gets there with retrieval that runs on a laptop — SQLite, full-text search, a 384-dimension embedding model and an 80 MB cross-encoder — and no LLM call when it writes or searches. Mem0 Platform runs an LLM extraction on every write.

How the 92.5 is produced

Reading the repository to set up the comparison, I found five things that make Mem0's published figure incomparable with the numbers other projects published under the earlier protocol. All of them are in public files; commit hashes are in the report.

1. It is the managed platform. Mem0's own docs: "Scores reflect Mem0's managed platform, which includes proprietary optimizations not available in the open-source SDK."
2. gpt-5 answers, gpt-5 judges, from 200 retrieved memories per question.
3. The judge is more lenient than the one the earlier numbers used. On LoCoMo it counts an answer correct when it has one item of a list, when a date is within 14 days, when a duration is within 50%, or when it names "the same referent". The LongMemEval judge tells the grader: "You have a tendency to say 'no' too quickly … When in doubt, lean toward 'yes'." On identical answers, Mem0's LoCoMo judge scores 5–6 points higher than the published one.
4. 156 LoCoMo questions were re-run and merged into the published file. 26 of them are correct in the merged file, against 91% overall.
5. The answer prompt contains hints that match individual LoCoMo gold answers, and has since the repository's first commit, before the published run:

Line in the answer prompt	LoCoMo question → gold answer
"store with a lot of working people -> store employs a lot of people"	"Does Dave's shop employ a lot of people?" → Yes
"a game exclusive to one platform implies ownership of that platform"	"What Console does Nate own?" → A Nintendo Switch; since the game "Xenoblade 2" is made for this console
"An unnamed company deal can be linked to a previously expressed brand preference"	"Which outdoor gear company likely signed up John for an endorsement deal?" → Under Armour
"you may name it (e.g., "Eternal Sunshine of the Spotless Mind")"	"What is one of Joanna's favorite movies?" → Eternal Sunshine of the Spotless Mind
"All events occurred in 2022-2024. Never output 2025 or 2026."	the date range of the LoCoMo conversations

I am not claiming anything about intent. I am saying that a number produced this way cannot be put in a table next to numbers produced without it — including mine from last week. The per-question grading above sidesteps all five, because both systems' answers go through the same judge.

What changed in TAM to get to a tie

TAM 14.5.0, released today, has four retrieval changes that came out of the error analysis:

• The cross-encoder reads the neighbouring turns. "What was the poetry reading about?" is answered by "Self-acceptance and trans stories", which shares no word with the question; the turn before it does. The cross-encoder now scores each candidate alone and together with the turns around it.
• Relative dates are resolved against the time the message was sent. "last Thursday" comes to the reader as "last Thursday [Thu 14 December 2023]". Week-level phrases stay relative — "the week before Sun 17 December 2023" — because resolving "last week" to a calendar week cost 5.5 points on LoCoMo's temporal questions: the gold answers say "the week before".
• The context budget is shared by rank. An even split cut an assistant's list of a hundred items down to items 1–9 and 49–57 when the question asked for the 27th. Questions about something the assistant said now get the whole answer in every development case, instead of 38%.
• Two bugs: MEMORY_TEXT_EMBED_MODEL did nothing for ordinary records, and fastembed models above 2 GB could not load at all — silently leaving records without vectors.

What didn't work, for the record: an agentic reader that could call memory_recall again (worse on both benchmarks), a bigger context (worse on LoCoMo), and gpt-5 on LoCoMo (it refuses twice as often under TAM's "say Not enough information if a premise is missing" guidance, which is right for real use and wrong for a benchmark that excludes its adversarial questions).

Reproduce it

Everything — the harnesses, the cross-grader, TAM's per-question answers and the grades — is in the repository. The report has the commands: about $6 of OpenAI credit without the gpt-5 cells, $40 with them, and a few hours of CPU.

pip install total-agent-memory==14.5.0

If you run a memory product and publish per-question answers, I would like to add you to the table.