Originally published at Trida AI.
Ask an agent in June who owns the Acme renewal and it may well say Priya. Priya handed the account to Daniel in March. Both facts were in the material the agent searched — the March handover note and the older account record, side by side, neither marked as finished. The model picked the wrong one, and nothing in the context it received said which statement had died.
Nothing failed in the model. Retrieval worked; both statements were genuinely relevant. What was missing was a record of the change itself: which fact is current, when the old one stopped being true, and where each came from. Keeping that record is a job of its own, and over the past year it has become a product category.
The category goes by context engineering. Zep, founded in 2023 and out of Y Combinator's Winter 2024 batch, calls itself the context engineering platform for AI agents and sells a unified context layer: company data in, governed context out. Mem0 works the narrower memory slice of the same problem and has distribution most startups would envy: AWS picked it as the memory provider for its agent SDK, and the company has raised $24 million. Letta, grown out of the MemGPT research at UC Berkeley, treats memory as something the agent maintains itself. And on November 6, Anthropic published The components of agent memory, its own architecture for what an agent keeps between sessions.
Different companies, different shapes. But the layer is distinct from the two things it sits between. It is not the model. It is not the vector database most teams reach for first, either. A vector store keeps chunks of documents, while this layer keeps facts: extracted, dated, and corrected as the company changes underneath them.
The work moves to background time
The older pattern assembled context during the run. A question arrived, the system searched raw material, and the model sorted signal from noise inside the prompt, under time pressure, at run-time prices. The new pattern does the understanding earlier — at ingestion, or in background passes between runs — so the run starts with context that has already been sorted.
Anthropic's name for the background pass is dreaming; the post credits the idea to earlier sleep-time compute work from Letta and UC Berkeley. Between sessions it rereads transcripts, tests notes against what actually happened, corrects the wrong ones, merges duplicates and builds a linked wiki over the material. On an internal corpus of about 350,000 files, Anthropic reports that answers drawn from the wiki alone met 13% of its grading criteria after two dreaming sessions and 48% after 128. Search and wiki together reached 61%, against 53% for search alone. Those are Anthropic's numbers, measured on Anthropic's corpus. Take them as a direction, not a benchmark. The direction is hard to argue with: the wiki kept getting better while nobody was querying it.
The discipline shows in the small print. Claude Code's auto memory loads only the first 200 lines or 25 KB of its index file into a session. The part of the layer that loads every time has to stay small enough to deserve it.
We've written before about what one agent should bother writing down. This layer is that instinct, industrialized: the note-keeping moves out of the agent and into infrastructure the whole company shares.
The compile-ahead pattern. Extraction, compilation and correction happen between runs — Anthropic calls the pass dreaming — so a run makes one retrieval and receives an assembled context block with validity dates and provenance already attached.
Zep's engine is Graphiti, an open-source framework (Apache 2.0) for temporal knowledge graphs. A fact there is a relationship between two entities, and it carries two timelines (when it was true in the world, and when the system learned it), four timestamps in all. When a new fact contradicts an old one, the old fact is invalidated rather than deleted: its end date is set, and it stays in the graph, queryable as history. Underneath, search mixes semantic embeddings, keyword matching and graph traversal, and what the agent receives is one assembled context block with each fact's validity dates attached, instead of a pile of chunks to adjudicate.
The same idea is arriving from the retrieval side. RAGFlow 1.0 calls it knowledge compilation: at ingestion, documents are translated into atomic propositions (one-sentence facts, each pinned to a verbatim quote from the source), so a question can later be answered from compiled evidence instead of a fresh excavation of the corpus. The team reports that a question answered from compiled evidence can save over 90% of its tokens. Compilation is the right metaphor. Source text goes in, a checked and cited artifact comes out, and the expensive thinking happens once, ahead of time.
One fact changes hands
Architectures are easiest to compare on an ordinary change. In March, a message in the account channel says Daniel takes over the Acme renewal from Priya. Three systems ingest that message. Here is what each one stores, and what an agent working from each one sees in June.
One fact changes hands in March: the Acme account's renewal owner moves from Priya to Daniel. Zep closes the old fact with a date and keeps it as history, Mem0 rewrites its extracted list in place, and Letta's agent edits its own memory file, with the edit kept as a versioned change.
All three agents answer "Daniel." The difference is what else survives. Zep's agent can also say who owned the account in February and when that changed, because the old fact was closed, not erased. Mem0's list holds the current truth with the least machinery around it; the record of the change sits behind the list, in the platform's own history, rather than in front of the agent. Letta's agent holds the file itself, and the March edit is a versioned commit, so the audit trail is the file's own past. Same answer in June. Three very different answers to the question a compliance team asks in September: show us what the agent believed, and when it stopped believing it.
Buy on correction mechanics, not benchmarks
The category's numbers should be handled with tongs. On LOCOMO, the benchmark this field argues about most, Zep has reported 84%. Mem0 re-ran the evaluation and reported 58.44% for Zep's system; Zep answered with a rebuttal saying the setup was misconfigured, and its own re-run put the figure at 75.14%. Nobody independent has settled it, and the gap between the claims is wider than most of the effects being argued over. Every vendor here, Anthropic included, grades its own homework.
So judge the layer the way you would judge a records system. When a fact changes, is the old one closed with a date, or does it linger? When the agent states a fact, can you walk it back to the source it came from? Who is allowed to see which context, and is that enforced or merely promised? And when the layer gets something wrong (it will), what does a correction look like, and does it reach the next run?
None of this makes the layer free. A compiled context is one more system to run, and background passes burn compute whether or not anyone asks a question. In one of Anthropic's tests, the wiki grew larger than the documents it summarized — a map the size of the territory, abstracting nothing. A wrong fact also behaves worse in a maintained layer than in a raw pile. A stale chunk might never be retrieved. A wrong note promoted into every session gets read every time.
That is where this stops being a tooling choice and becomes deployment work. In an enterprise rollout, the model is the part most likely to be swapped next year. The context layer — the client's facts, their corrections, their provenance — is what the next model inherits. Building it means deciding which systems count as sources of truth, which changes need a person's sign-off before they land, and which facts expire. Unglamorous maintenance, and it compounds: every correction makes the next run cheaper to trust.
Work with us
If your agents are answering from data that changed last quarter, the fix is a layer, not a longer prompt. Bring us the systems your agents read from — Trida AI's intake is where that conversation starts.
Top comments (0)