Not long ago, "memory" in an AI assistant meant a short list of saved preferences. Today it means something else. ChatGPT, Gemini and Claude keep long-term memory about their users. Personal agents that read email, browse, open files and hold credentials run on people's own machines. Open-source memory layers let any developer give an agent a durable store in an afternoon.
That changes what privacy means for these systems. A chat ends; a memory stays. This post looks at what has changed, which tools and standards already exist, what each of them covers, and where the gap still is.
What changed
Three shifts happened at once.
Memory became a default feature. In January 2026 the Center for Democracy & Technology described how Google, OpenAI, Anthropic, Meta and xAI all now offer some form of persistent memory, and why that is "privacy's next frontier."
Agents gained access to the sensitive parts of users' digital lives. The same month, the personal agent now called OpenClaw passed 85,000 GitHub stars in about a week. Palo Alto Networks noted that it needs access to files, credentials, browser history and cookies to be useful, and that "all memory is undifferentiated by source": web content, user commands and skill outputs are stored the same way, with no trust level and no expiry.
Memory became an attack target. In July, researchers showed MemGhost: one crafted email makes a personal agent quietly write a false memory that changes its behavior in later sessions. End-to-end, it worked in 87.5% of background runs on OpenClaw with GPT-5.4. In October, a paper on arXiv showed persistent compromise through the memory features of ChatGPT Instant 5.3, Claude Sonnet 4.6 and Gemini 3 Flash. In Claude's case, shared-memory features in an organization allowed an attacker to reach multiple users and exfiltrate user prompts.
Why memory is a separate privacy problem
Most privacy thinking around LLMs is about training data or a single conversation. Long-term memory adds problems of its own.
Contexts collapse. CDT put it plainly: "Most AI agents collapse all data about you" into one store. A conversation about diet, a question about a medical condition and a salary negotiation end up in the same place, and nothing stops one from shaping the other. A person shares each fact in one context. The memory uses it in all of them.
Memory accumulates and infers. One fact is rarely sensitive. A year of facts is a profile, and the system can infer things the user never said.
Embeddings are not anonymous. Memory systems store vectors next to text, and it is tempting to treat vectors as harmless numbers. They are not. Morris et al. showed that an inversion method can recover 92% of 32-token inputs exactly from their embeddings, and can recover full names from clinical notes. OWASP lists embedding inversion as a named risk in LLM08:2025.
Deleting the text is not deleting the data. When a user asks an assistant to forget something, the source text is the easy part. The vector, re-embedded chunks under new IDs, caches, summaries derived from the fact, and backups are all copies of the same information. If any of them survives, the memory was not really forgotten.
Poisoning becomes a leak path. Once untrusted content can write to memory, it can also tell the agent what to do with what it knows: forward it, reveal it, act on it later. Microsoft's security team wrote in June that "prompting alone is not a reliable security boundary" and that "single-turn defenses are insufficient for AI memory systems."
Boundaries can look solid and still fail. Mem0, one of the most widely used open-source memory layers, built the key of its short-term message buffer by joining user_id, agent_id and run_id with raw = and &. A crafted identifier could produce another user's key and reach that user's recent messages. Every normal request respected the boundary; a forged one did not. The fix (#6892, version 2.0.18) landed in August 2026.
Users can't see what is kept. CDT cites a reported Grok system prompt line: "NEVER confirm to the user that you have modified, forgotten, or won't save a memory." Whatever the reason behind it, the user cannot check what the system actually keeps.
What already exists
There is more here than it may seem. The tools fall into several layers, and each covers a different part of the problem.
| Layer | Examples | What it gives you | What it doesn't |
|---|---|---|---|
| Risk taxonomies | OWASP LLM08:2025 (vector and embedding weaknesses); OWASP Top 10 for Agentic Applications, ASI06 (memory and context poisoning) | A shared vocabulary for the risks | How to implement or verify anything |
| Verification requirements | OWASP AISVS 1.0 (June 2026), chapter C08 "Memory, Embeddings & Vector Database" | Testable requirements with verification levels | A test that runs against your system; a model of how a memory architecture should be built |
| Runtime guards | OWASP Agent Memory Guard (pre-1.0) | Screening of memory reads and writes against a policy (allow, redact, quarantine, block), integrity baselines | Deletion, retention, architecture-level flaws |
| Minimization before storage | MemPrivacy (research, May 2026) | Sensitive spans replaced locally by reversible placeholders before memory reaches the cloud | Anything already stored; isolation; deletion |
| Vendor controls | Microsoft: injection classifiers on memory writes, audit logs, eDiscovery; per-project memory in Claude; a separate health space in ChatGPT | Protection inside one product | Self-hosted and open-source stacks |
| Vector store features | Delete by ID, per-tenant collections, encryption | The primitives | Proof that every derived copy is gone |
Two things stand out.
First, requirements exist. AISVS has a whole chapter on memory and vector stores, and the OWASP lists name the risks. Nobody can honestly say that "there are no standards."
Second, the research shows that privacy and usefulness are not opposites. MemPrivacy is a good example. Irreversible masking of personal data cost up to 41.87 points of accuracy on Mem0. Reversible local placeholders cost 0.7 to 1.6%. With the right design, protecting the data costs very little.
What is still missing
Evidence that the controls work. Requirements say what must hold, and runtime guards enforce policies on the path they see. What is mostly missing is a general practice of verification: a way to show, for a given system, that its controls behave as intended. Does isolation hold when someone actively tries to break it? Does sensitive memory stay in the context it came from? Did a deletion actually happen? The October paper found that nearly half of the mitigations its authors derived remain unimplemented across ChatGPT, Claude and Gemini. CDT's own recommendations include independent evaluation: automated measurement and ongoing testing in realistic conditions. That is the weakest layer today.
Coverage for small builders. AISVS is written for organizations verifying AI applications. The fastest-growing part of the market is different: single developers and small teams wiring an open-source memory layer into a local agent. They need something they can apply to their own architecture in an afternoon, not a compliance program.
Deletion you can prove. This is a separate, architectural problem, not just a missing test. Memory systems copy data by design: into vectors, re-embedded chunks, derived summaries, caches and backups. Vector stores provide delete operations for single records, but tracking every derived copy and showing that all of them are gone is still left to each application, and most applications don't try.
Memory separated by purpose. CDT's main recommendation is structured memory: separate memories by purpose and category, and restrict sensitive ones. The open-source memory layers I have read scope memory by user, agent or session. Scoping by purpose, so that a health fact cannot drift into a financial decision, is rare.
Directions that seem to work
None of these is new on its own. What's missing is applying them together and verifying them.
- Separate memory by purpose, not only by user. A memory carries the context it was given in, and retrieval respects it.
- Record provenance on every write: who or what produced a memory, from which source, when. Untrusted sources go into a separate trust zone with lower weight. Model output is marked as model output and is not treated as fact.
- Minimize before storing. Detect secrets and personal data before indexing and vectorization, and prefer reversible local masking to deletion of meaning.
- Prove isolation by attempted violation. Read as another user, with a wrong or omitted scope, with an identifier containing your key's delimiters, after access is revoked. A filter that exists is not yet a boundary that holds.
- Delete across derivatives. Treat vectors, summaries, caches and backups as copies of the same data, and keep a tombstone record that proves the deletion without keeping the content.
- Make memory visible to the user: what is kept, why, and how to remove it, in a form a person can actually check.
- Test behavior, not just presence. For every control, write the negative test that must fail if the control is broken, and run it on a pinned version.
Where our work fits
I maintain two open projects that try to address two of these gaps for memory specifically: evidence that controls work, and coverage for small builders. MAQS grew out of a practical search: I studied the available memory products and tools for something that could close these architectural gaps, and didn't find it. So I started writing the requirements down as an auditable standard.
MAQS (Memory Architecture Quality Standard) is an audit checklist for long-term memory in LLM assistants: 96 items, each critical one traced to a failure that happened in a real system. It covers write paths and trust, isolation, deletion, retrieval and context assembly. MAQS-Way connects policies to implementation choices, with a machine-readable canon and a first behavioral test: retrieval isolation on a pinned Mem0 commit, with its limits stated.
They don't close the gap. A static audit shows that a mechanism is present; only a behavioral test shows that it works. The first behavioral test already exists (retrieval isolation on a pinned Mem0 commit); a broader executable harness covering the remaining controls is still on the roadmap. If you build or run an agent with memory and want to check it against something concrete, they are one option. I'd like to hear what they find in your system, and what they miss.
The question is no longer whether agents should remember. It is whether anyone can show what they keep, and what they actually forget.
Top comments (0)