Every agent framework now ships something called "memory," and every vendor's landing page implies theirs is the only real one. They're not the same thing. Four tools currently compete for the same slot in an agent's stack — Mem0, Zep, LangChain's built-in memory classes, and Letta (formerly MemGPT) — and they solve different problems with different architectures. Pick the wrong one and you either over-engineer a simple chatbot or hit a wall the moment your agent needs to run for more than a single session.
I've run all four in production-adjacent projects over the past few months. Here's what actually differs, with the tradeoffs vendors don't put in the hero section.
Mem0: a fact store with an opinionated write path
Mem0's pitch is "just call add(), memory figures itself out." That's mostly true. Every add() runs an LLM pass that extracts discrete facts from the input, checks them against existing memories via vector similarity, and issues ADD, UPDATE, DELETE, or NONE for each one:
from mem0 import Memory
m = Memory()
m.add("I'm allergic to shellfish", user_id="alice")
m.add("Actually I can eat shrimp now, just not crab", user_id="alice")
m.search("what can alice eat?", user_id="alice")
The win is that you don't hand-write merge logic — a real time sink in any long-lived agent. The cost is that every write is itself an LLM call, so ingestion has both latency and per-write token cost, and the conflict resolver runs on text similarity, not on your app's notion of scope. It can't distinguish "the user changed their mind" from "the user has two preferences that hold in different contexts," and when it gets that wrong it silently deletes, no warning. Mem0 wins when your access pattern is "what does this user prefer/believe right now" and you're fine paying an LLM call per write for that convenience.
Zep: temporal graph, not a fact list
Zep (via its Graphiti engine) doesn't store facts as flat rows — it builds a temporal knowledge graph where every edge carries validity windows. That's the actual differentiator: you can ask not just "what does alice believe" but "what did alice believe last month, and when did that change."
from zep_cloud.client import Zep
client = Zep(api_key="...")
client.graph.add(user_id="alice", type="text",
data="Alice moved from the platform team to infra in March.")
client.graph.search(user_id="alice",
query="what team is alice on now vs six months ago")
This is the right tool when your agent reasons about change over time — support agents tracking account history, sales agents tracking deal evolution, anything where "when" matters as much as "what." The cost is real infrastructure complexity: you're running graph queries where a simpler agent just needed the last known value, and the mental model (nodes, edges, temporal invalidation) is a genuine onboarding cost for a team that only needed a preference lookup.
LangChain memory: free until it isn't
If you're already building on LangChain, ConversationSummaryBufferMemory or VectorStoreRetrieverMemory costs you nothing extra to wire up — no new service, no new API key, no new bill:
from langchain.memory import ConversationSummaryBufferMemory
memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=1000)
memory.save_context({"input": "I prefer terse code reviews"}, {"output": "Noted."})
That zero-marginal-cost is exactly why it's the wrong default for anything beyond a prototype. There's no first-party persistence layer, no multi-tenant isolation (you're rolling your own keying scheme per user), and no retrieval quality guarantees beyond whatever vector store you bolt on yourself. Worse, it couples your memory layer to your orchestration framework — if you ever want to leave LangChain, you're rewriting memory and orchestration at the same time instead of one at a time. Use it for demos and internal tools you'll throw away. Don't let it become the memory layer of something you plan to operate for a year.
Letta: the agent manages its own memory
Letta takes a completely different stance: instead of an external service the agent calls, memory is part of the agent's own runtime. The agent has editable "core memory" blocks that sit permanently in its context window, plus "archival memory" it can page in and out on its own initiative — it decides what to forget and what to promote, via tool calls it makes to itself.
from letta_client import Letta
client = Letta(base_url="http://localhost:8283")
agent = client.agents.create(
memory_blocks=[{"label": "human", "value": "Name: Alice. Role: infra."}],
model="openai/gpt-4o-mini",
)
This is the right architecture for genuinely long-horizon autonomous agents — ones that run for days, accumulate context that would blow any context window, and need to actively curate what they keep. It is the wrong architecture for a request-scoped chatbot: you're running a stateful agent process instead of making a stateless API call, and that's real operational weight (process lifecycle, storage for agent state, no simple "just query the facts" endpoint from outside the agent).
The actual decision
Stop asking "which memory tool is best" — ask which question your agent needs answered:
- "What does this user currently prefer?" → Mem0. Accept the per-write LLM cost for the convenience.
- "How did this relationship or fact change over time?" → Zep. Accept the graph-infrastructure overhead for temporal queries you can't fake with a flat store.
- "I'm prototyping inside LangChain and will decide on production memory later" → its built-in classes are fine, but budget time to rip them out before you ship.
- "My agent needs to manage its own understanding across a long-running session" → Letta. Accept the stateful-process overhead because you need the agent, not just an API, doing the curating.
These aren't mutually exclusive. A support-triage agent I built uses Mem0 for per-customer preference lookup and a Letta-style self-managed core memory for the agent's own running theory of the current incident — two different memory problems, correctly solved by two different tools instead of forcing one system to do both badly.
Top comments (2)
This is a useful split. The bit I would add is retention policy. A memory layer that never forgets becomes a second prompt nobody audits. I usually want the write path, expiry rules, and a boring inspect command before I trust the retrieval story.
Good catch—retention is where most systems fall short. Mem0 and Zep have policies on paper, but you'll end up needing to build your own inspect/audit layer because the visibility just isn't there.