Cristián Labra

Posted on Sep 7 Originally published at 3cucharadas.cl AI-assisted

Multi-agent work in three spoonfuls III: a memory that leaves traces

#multiagent #rag #agentmemory #provenance

Status of the demo. The viewer was regenerated on August 29, 2026 from a sanitized public projection (with the non-public bits stripped out 😀): the artifact contains no mail bodies, attachments, addresses, absolute paths, tokens, credentials, or microdata.

Preamble: remembering is not enough

In the second part I went after a bounded problem: getting penta-agent's memory to retrieve evidence and to recognize when it had found none. The question in this third part is more practical, and it comes out of the system having been in use for a while: what happens to a memory as it grows and turns blurry, or even contradictory?

An index can pile up fragments without any trouble, and there are plenty of tools that already do that well. A more useful memory, in my judgment, has to carry provenance, currency, permissions, contradictions, and deletion criteria. It also has to tell finding a source apart from using it correctly. Recent literature insists on separating RAG — retrieval-augmented generation — context management, and agent memory, because they do different jobs and call for different evaluations ¹.

What follows has three movements: what changed since part II; which experiments survived a more serious evaluation; and how to show a memory without passing it off as a mind.

Spoonful 1: from retrieving fragments to governing evidence

In part II the problem was retrieving well: finding the relevant context and recognizing when there was not enough evidence. A useful memory does not only retrieve information; it also has to know where it came from, whether it still holds, where it can be used, and what is allowed to be done with it.

RAG mostly solves retrieval. The memory layer adds rules for keeping, updating, relating, or discarding evidence. None of those functions amounts, on its own, to identity. To describe provenance I use concepts compatible with PROV-O — entities, activities, and agents — while currency, sensitivity, and permissions need rules of their own ².

What changed since part II

Open item from II	What exists now	What is still open
Compare dense retrieval, lexical retrieval, and reranking.	A frozen cut of 319 contexts and 40 queries evaluated at `k=5`.	The best configuration gets 38 complete cases and 2 partials; the strict gate stays shut.
Check that an answer uses its sources and knows how to abstain.	The `context-answer-v1` contract, with 12 positive and 6 negative cases, all sanitized.	The end-to-end evaluation with a local model and the reserved human review are unfinished.
Add currency without erasing history.	The proof of concept admits `supersedes` only toward later versions and rejects competing successors.	There is still no temporal reasoner wired into live memory.
Show growth without turning the graph into truth.	A 3D viewer, regenerable from sanitized JSON and accompanied by a text alternative.	The viewer neither validates relations nor represents a mind.

The difference may look small, but it changes the question. Knowing which fragment sits close to a query is no longer enough; what matters as well is what kind of evidence it is and under what conditions it can be reused.

What memory actually exists

The PoC brings several layers together, but does not treat them as equivalent:

penta-agent experience: 16,955 indexed points and 1,432 strategies make it possible to explore actions, corrections, and failures. Semantic closeness proves neither causality nor truth.
Curated context: 319 canonical records make it possible to retrieve handoffs and decisions with their provenance. The index helps to find them, but does not replace the original record.
Research process: a projection with 796 derived artifacts and 533 links makes it possible to inspect the work trajectory without publishing documents or microdata.
Personal mail: the local inventory holds 12,072 unique messages between 2011 and 2026. It serves to test reading and deduplication within an authorized scope; it is not a public corpus and not a personality model.
Experimental personal memory: a separate prototype tests retrieval through mbox, SQLite/FTS, scopes, and purge. There is still no authorization to generate answers out of sensitive mail.

This separation heads off a frequent temptation: confusing having technical access to a source with being authorized to turn it into reusable memory.

A useful failure

The pipeline itself supplied an example of why provenance matters. The historical exporter expected a plain vector, while the current index stored the named dense vector. The first regeneration produced a degraded graph, with no semantic edges. A run against the real index caught the incompatibility and, once the exporter was fixed, 3,658 edges came back.

The lesson matters more than the number: an auditable visualization has to be able to show when its data chain degraded, why it happened, and how it was put back together. It is a real use case.

From command to task

The 1,432 recorded strategies are still the basic unit of this memory. Many of them started life as commands, so I added a deterministic classification to make them more readable: coordinating agents (399), researching and analyzing (396), tracing evidence (385), executing or automating (118), versioning changes (76), verifying (28), operating infrastructure (18), and communicating results (12).

A strategy could belong to several categories, but today it lands in the first matching rule. It is a lens for walking through the memory, not a description of the person behind it. There is plenty of room to improve this with other visualizations; if you know of one, I would be grateful for the tip.

Spoonful 2: governing sources before blending them

The architecture starts from a simple separation. The canonical record (the ledger) and the source files hold the evidence; Qdrant keeps indexes that can be destroyed and regenerated; the viewer publishes a sanitized projection; and the private adapters stay outside Git. The index is there to find things. It has no authority to turn a vector match, on its own, into a durable fact.

An architecture of permissions, not a blender

Mail, thesis material (from the master's degree), and operational memory may all be technically readable by the same system, but that does not make them interchangeable. In this PoC, only the previously curated context may feed cross-agent retrieval and the public projection. Mail stays in a private, reversible, revocable circuit; the thesis contributes authorized process metadata and nothing else.

The distinction matters: having technical access to a piece of data does not settle whether reusing it for any purpose is legitimate. Contextual integrity theory poses exactly this problem: privacy depends not only on the data, but also on the context, the actors, and the norms that govern its circulation ³.

Figure 1. The current integration is an architecture of permissions: each source determines which outputs are permitted.

The thesis is a good example. What enters the public system is a snapshot of the research index, not the corpus. The August 29 cut holds 796 derived textual artifacts — 316 audits, 244 reports, 133 contracts, 71 pseudocode files, and 32 inventory, replication, or documentation pieces — plus 533 links and 7 open leads. It leaves out the source documents, DTA, Parquet, PDF, images, paths, full texts, and microdata. The search is deliberately lexical: FTS5 and TF-IDF help to find artifacts, but they make no claim to understand the research.

There is a detail more interesting than the figures. The published snapshot is still faithful to the cut it declares: its fingerprint matches the manifest of that moment. The corpus, however, kept growing and reached 817 documents, twenty-one more. Since the index was not regenerated, the currency gate is red: check-research-index returns 2, because document_count and source_fingerprint no longer match the current state.

That does not invalidate the snapshot; it dates it. Its correct status is "valid for the August 29 cut, but not current with respect to today's corpus". A gate that had stayed green after the corpus grew would have been worse than a failed gate: it would have certified a currency that does not exist. This is the formal record of the memory's growth and potential divergence.

Reuse before reinventing

None of this requires building every memory component from scratch. In fact, a good sign of maturity would be being able to delete my own code when an existing tool solves the same problem better without degrading privacy, provenance, or reversibility.

To recover decisions scattered across coding sessions, deja-vu already indexes the local histories of many agents and exposes them through a CLI and MCP. It is a natural candidate to contrast against my own handoffs and events before writing yet another equivalent layer.

For more structured persistent memory there are more ambitious approaches. MemGPT introduced the idea of managing distinct memory tiers as a way of extending an agent's effective context ⁴; Letta's current line carries that principle into editable memory blocks and external memory. Mem0 automates the extraction, consolidation, and retrieval of information from conversations ⁵, while Hindsight distinguishes the operations of retaining, recalling, and reflecting, taking in semantic, lexical, temporal, and graph-based retrieval ⁶.

The currency problem deserves an experiment of its own. Graphiti, the open component described in Zep's architecture, explicitly models relations that change over time and keeps historical information instead of silently replacing it ⁷. That comes far closer to the problem I am trying to solve with supersedes than piling on more temporal rules of my own.

For document collections, Microsoft GraphRAG offers another useful idea: build a graph derived from the corpus and summarize communities to answer global questions that a conventional RAG handles worse ⁸. I would not use it yet as a replacement for the thesis index. First it would have to show, over a bounded scientific subset, that the added complexity and cost improve something the lexical baseline cannot resolve.

Visualization, finally, is a practically solved problem: 3d-force-graph already provides a three-dimensional layout built on Three.js and force algorithms. The relevant part of this PoC is not writing another graphics engine, but controlling which graph the browser receives and remembering that visual distance proves neither conceptual proximity, nor causality, nor human affinity.

The common rule is simple: test components, do not collect them. That a library can remember more does not mean it should receive more data.

There is also a conflict of interest worth making explicit. Mem0, Hindsight, Zep/Graphiti, and Letta are described in large part by their own teams. Their papers and repositories are appropriate sources for learning the architectures they propose, but their comparative results are not independent validation of superiority. I use them here as designs that deserve a contrast, not as winners of a competition — and my own stack is not superior either. If anything, mine is rather experimental, built for my own purposes.

A retrieval experiment that could fail

I tested Qwen3-Embedding-0.6B on an isolated copy of 319 contexts, without touching production. The evaluation used 40 queries, eight of them designed to check that the system also knew how not to retrieve evidence when it should not.

The best configuration got 38 complete cases, 2 partials, and no outright miss within the top five positions. From there I tried several possible improvements:

dropping the BM25 lexical boost held the best result;
adding a generic instruction to the embedding made both retrieval and abstention worse;
Qwen3-Reranker-0.6B ordered the results somewhat better, but pushed response time up too far;
diversifying by repository and fusing rankings did not resolve the two partial cases either.

The experiment left a conclusion more useful than finding a new model: the two open cases do not seem to need more power, but better metadata. Both of them require retrieving documents from different families within only five results.

So the next test will be to add explicit information about lineage and document intent, without using the correct answers to build those labels. Arbitrarily raising k or piling on more models would only have moved the problem elsewhere.

Until those two cases are resolved reproducibly, the collection stays in staging. In this system, remembering that an improvement has not been demonstrated yet is also part of the memory.

Spoonful 3: a map that does not pass itself off as a mind

Once what may enter the memory is sorted out, another question remains: how do you show it without mistaking a representation for reality?

The viewer answers with a precaution: it queries neither Qdrant (the database where the vectors are indexed) nor any private source when someone opens the site — it receives a sanitized JSON (a data file from which the information that must not be published has been removed), generated from a specific cut of the memory.

That way the graph can be rebuilt and audited without touching the original sources again. The public JSON export even lets you inspect directly what information the browser receives.

Seeing the network without attributing more to it than it says

The August 29 cut holds 1,432 strategies, 3,949 relations, and eight task families, derived from 16,955 indexed points.

The image below is not that network: it is an illustration, and the percentage is the joke. The real network is explored in the full viewer, which lets you select tasks, projects, and errors.

Figure 2. An illustration, not a projection of the data: the digital-clone metaphor loaded to 20%. The network hints at itself where there is density and comes apart where there is none, which is exactly what an index knows how to do. The real projection, with its nodes and edges, is in the viewer.

Open the interactive viewer in a new tab →

It is worth reading with some precautions. A node (a point in the graph) stands for a recorded strategy. An edge (a line between two nodes) stands for some derived relation: semantic similarity, a shared tool, or a correction, for instance. A community (a group of nodes that appears especially well connected) helps you get your bearings inside the network.

None of those relations proves, on its own, causality, truth, or a trait of the person who used the system. Mail and private documents do not appear as content nodes either.

The numbers belong, moreover, to a dated cut. Updating the memory does not silently change the graph: the projection has to be regenerated, sanitized again, and its differences reviewed before it is published anew.

The viewer is not the memory. It is barely a map built from one of its projections.

Retrieving a source is not enough either

Finding the right document is only half the problem. The system also has to use the right source, respect its scope, and abstain when the evidence falls short.

To test that layer I built context-answer-v1, a small set of 18 cases: 12 with sufficient evidence and 6 in which the right answer was to claim no more than what was available. The deterministic test passed all 18 cases and confirmed that the gate works; it still does not prove that a generative model answers well on its own.

The tests with external models also left a practical rule. Only the sources authorized to leave the machine should be evaluated outside it. The exportable subset passed 11/11 cases; another seven sources stayed local_only and require an evaluation inside the local environment.

The conclusion is simple: retrieving is not enough; every answer has to carry provenance, permissions, and the ability to abstain. Until that local evaluation and its human review are complete, I can talk about a governed and attributable memory, but not yet about a reliable generative memory.

Closing: a useful memory before a digital clone

The "digital clone" is still some way off; today it is a distant horizon. What is missing, at a minimum: a temporal memory able to correct itself, revocable preferences and limits, evidence of authorship, behavior evaluated in new situations, and "human" or higher control over what is kept, shared, or deleted. MemGPT⁴ and the generative agents of Park et al.⁹ offer influential architectures for handling context, memories, reflection, and planning, but they do not turn those functions into identity (Packer et al. 2023; Park et al. 2023).

In this third part, penta-agent is a local prototype of governed memory: it keeps experience, records corrections, exposes a bounded public projection, and lets its own tests fail without disturbing production.

The next step forward is not adding more nodes. It is closing the local answer evaluation and comparing alternatives — Graphiti, GraphRAG, or deja-vu — under the same cut, the same permissions, and observable output criteria. I am very much open to your comments and usage experience; I think one practical point of comparison would serve me better than going on testing the many tools that appear day after day.

References

Software and technical documentation

GetZep. n.d. Graphiti: A Framework for Building Temporal Knowledge Graphs. GitHub repository. Accessed August 31, 2026. https://github.com/getzep/graphiti.

Hugging Face. n.d. Sentence Transformers. Technical documentation. Accessed August 31, 2026. https://www.sbert.net/.

Letta AI. n.d. Letta Code. GitHub repository. Accessed August 31, 2026. https://github.com/letta-ai/letta-code.

Mem0 AI. n.d. Mem0: Universal Memory Layer for AI Agents. GitHub repository. Accessed August 31, 2026. https://github.com/mem0ai/mem0.

Microsoft. n.d. GraphRAG. GitHub repository. Accessed August 31, 2026. https://github.com/microsoft/graphrag.

Qwen. n.d. Qwen3-Embedding-0.6B. Model card, Hugging Face. Accessed August 31, 2026. https://huggingface.co/Qwen/Qwen3-Embedding-0.6B.

Qwen. n.d. Qwen3-Reranker-0.6B. Model card, Hugging Face. Accessed August 31, 2026. https://huggingface.co/Qwen/Qwen3-Reranker-0.6B.

Shulcz, Vladislav. n.d. deja-vu. GitHub repository. Accessed August 31, 2026. https://github.com/vshulcz/deja-vu.

Vasturiano. n.d. 3d-force-graph. GitHub repository. Accessed August 31, 2026. https://github.com/vasturiano/3d-force-graph.

Vectorize. n.d. Hindsight: Agent Memory That Learns. GitHub repository. Accessed August 31, 2026. https://github.com/vectorize-io/hindsight.

Conflicts of interest in the sources. The model cards, the repositories, and several of the systems papers are written by their own developers or by organizations that offer related services; LongMemEval-V2, moreover, is still declared work in progress. They are used here to document architecture, declared functions, and maintenance status, not to accept claims of superiority. penta-agent's own metrics are likewise evidence produced by the project itself, and they require reproducible artifacts and independent review before they can support general comparisons.

Hu, Yuyang, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, et al. 2025. "Memory in the Age of AI Agents". arXiv, December 15, 2025. https://arxiv.org/abs/2512.13564. ↩
Lebo, Timothy, Satya Sahoo, and Deborah McGuinness, eds. 2013. PROV-O: The PROV Ontology. W3C Recommendation, April 30, 2013. https://www.w3.org/TR/prov-o/. ↩
Nissenbaum, Helen. 2004. "Privacy as Contextual Integrity". Washington Law Review 79, no. 1: 119–58. https://digitalcommons.law.uw.edu/wlr/vol79/iss1/10/. ↩
Packer, Charles, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2023. "MemGPT: Towards LLMs as Operating Systems". arXiv, October 12, 2023; revised February 12, 2024. https://doi.org/10.48550/arXiv.2310.08560. ↩
Chhikara, Prateek, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. "Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory". arXiv, April 28, 2025. https://arxiv.org/abs/2504.19413. ↩
Latimer, Chris, Nicoló Boschi, Andrew Neeser, Chris Bartholomew, Gaurav Srivastava, Xuan Wang, and Naren Ramakrishnan. 2025. "Hindsight Is 20/20: Building Agent Memory That Retains, Recalls, and Reflects". arXiv, December 16, 2025. https://arxiv.org/abs/2512.12818. ↩
Rasmussen, Preston, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. "Zep: A Temporal Knowledge Graph Architecture for Agent Memory". arXiv, January 20, 2025. https://arxiv.org/abs/2501.13956. ↩
Edge, Darren, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2024. "From Local to Global: A GraphRAG Approach to Query-Focused Summarization". arXiv, April 24, 2024; revised February 19, 2025. https://arxiv.org/abs/2404.16130. ↩
Park, Joon Sung, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. "Generative Agents: Interactive Simulacra of Human Behavior". In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, article 2, 1–22. New York: Association for Computing Machinery. https://doi.org/10.1145/3586183.3606763. ↩

DEV Community