Close a long coding session, open a new one tomorrow, and your AI agent greets your codebase like a stranger. The architecture decision you explained twice, the test command that only works with a certain flag, the library you told it never to touch: gone. That isn't a bug in one product. By default, a language model starts every session with a blank slate, and everything it "knows" about your project has to be put back in front of it each time.
AI agent memory is the set of techniques that fixes this. It decides what an agent writes down, where it stores it, and what it pulls back into view later. It sounds simple, but it is increasingly where the gap between a useful coding agent and a frustrating one comes from.
Quick answer: AI agent memory is any mechanism that lets an agent keep and reuse information beyond a single context window, from a plain markdown file like AGENTS.md to a dedicated memory layer built on vector search or knowledge graphs. Researchers usually split it into working memory (what is in the context window right now) and three long-term types: episodic (past experiences), semantic (facts), and procedural (how to do things). Which approach works best is still one of the most debated questions in agent design, and many of the benchmark numbers you will see published are by the vendors selling the tools.
Why AI Agents Forget Between Sessions
A model only "sees" what is inside its context window for the current session. When the session ends, so does that view. Nothing about your project carries over unless something puts it back.
The obvious fix is to paste more into the window, but that runs into a problem Chroma documented in its 2025 "context rot" research. The team tested 18 leading models and held task difficulty constant while only increasing input length. Performance became less reliable as inputs grew, even on simple tasks, and in uneven, hard-to-predict ways. In other words, a bigger pile of remembered text is not automatically a better memory.
That is the same underlying problem behind context engineering: the skill is deciding what deserves to be in the window at all. Memory is the storage side of that decision. It also complements repository intelligence, which helps an agent understand the code itself, while memory holds what the agent learned while working on it.
The Four Types of AI Agent Memory
The most common reference point is CoALA (Cognitive Architectures for Language Agents), a 2023 framework from Princeton researchers. It describes short-term working memory plus three long-term memories, borrowed loosely from cognitive science.
| Memory type | What it holds | Coding-agent example |
|---|---|---|
| Working | Whatever is active in the current context window | The files and errors in front of the agent right now |
| Episodic | Experiences from earlier runs | "Last time this migration failed because of a missing environment variable" |
| Semantic | Facts about the world and the project | "The API is versioned under /v2 and uses PostgreSQL" |
| Procedural | How to do things | Build and test commands, coding conventions, reusable skills |
The framework makes one point worth remembering. Letting an agent write new knowledge into episodic or semantic memory is relatively low-risk, but letting it rewrite its own procedural memory is significantly riskier, because that can introduce bugs or let the agent drift from what its designers intended. For coding agents, the practical version is simple: be careful about letting an agent edit its own instructions without review.
How Coding Agents Remember in Practice
Most of today's real-world agent memory is far less exotic than the research papers suggest.
Instruction files. The simplest memory is a markdown file the agent reads at the start of every session. AGENTS.md is the open version of this idea. When OpenAI contributed it to the Agentic AI Foundation under the Linux Foundation in December 2025, it said the format had been adopted by more than 60,000 open-source projects and agent frameworks since its August 2025 release, including Codex, Cursor, GitHub Copilot, and Gemini CLI. Anthropic's Model Context Protocol was contributed to the same foundation. Claude Code uses a similar convention with CLAUDE.md files. Because these are plain files in your repository, they are versioned, reviewable, and shared with your whole team.
Agent-written notes. Claude Code also has a second system called auto memory, where Claude writes its own notes as it works, such as debugging insights and corrections you gave it. Anthropic documents it as separate from CLAUDE.md, which holds instructions you write yourself. The notes are stored locally on your machine in a per-project folder rather than in your repository, only the start of the index file loads at the beginning of a session, and you can browse or switch it off with the /memory command. In CoALA terms, CLAUDE.md is roughly the procedural and semantic memory you author, while auto memory is closer to episodic notes the agent writes for itself.
Memory tools and context editing. Anthropic's memory tool lets Claude store and look up information outside the context window through a file-based system, and it runs entirely client-side through tool calls, so developers control where the data lives. It pairs with context editing, which clears stale tool calls and results as the window fills up. In Anthropic's own internal evaluations, combining the two improved performance by 39% over a baseline, and context editing alone improved it by 29%. In a 100-turn web search evaluation, context editing reduced token use by 84% and let workflows finish that would otherwise have failed from running out of context. Those are Anthropic's numbers on Anthropic's tests, so treat them as a direction rather than a guarantee for your own project.
Compaction and fresh sessions. Many tools also summarize a long conversation into a shorter version when the window gets crowded, or encourage starting a clean session and carrying forward only a short summary. It is crude compared with a dedicated memory system, but it directly attacks the context rot problem.
Memory Layers and the Benchmark Argument
Beyond files, a growing category of dedicated memory layers extracts facts from conversations, stores them in a vector database or knowledge graph, and retrieves the relevant ones for each new request. Mem0, Zep, and Letta are among the best-known names.
This is also where the numbers get contested. Mem0's 2025 research paper reported a 26% relative accuracy gain over OpenAI's memory feature on a benchmark called LoCoMo, plus a 91% lower p95 latency (about 1.4 seconds versus about 17 seconds) compared with feeding in the full conversation. Zep, a competitor, publicly disputed how its own system had been configured in that comparison. Letta, which grew out of the MemGPT research project at UC Berkeley, then reported that a plain agent with simple file tools scored 74.0% on LoCoMo using GPT-4o mini, above the 68.5% Mem0 had reported for its best graph variant. Mem0 has since rewritten its pipeline and says it now scores 92.5% on the same benchmark.
None of these figures are directly comparable, and each comes from a company with a product to sell. One independent analysis also pointed out that LoCoMo conversations are short enough to fit comfortably inside a modern context window, which limits what the benchmark can prove about long-term memory. The useful lesson is not which vendor wins. It is that simple file-based memory is a serious baseline, and that independent evaluation of agent memory is still thin.
What Can Go Wrong With Agent Memory
Memory adds new failure modes that a stateless agent does not have.
- Stale facts. Code changes faster than notes do. An agent that confidently remembers last month's architecture can be worse than one that remembers nothing.
- Bloat. An instruction file that keeps growing eventually becomes the same overstuffed context you were trying to avoid.
- Wrong conclusions that stick. If an agent saves a mistaken lesson, it can repeat that mistake in every future session.
- Sensitive data and unverified input. Memory stores can accumulate private project details. And in principle, if an agent saves notes based on content it did not verify, such as a web page or an issue comment, a bad instruction could outlive the session that introduced it. This is a design risk to plan for, not a documented incident.
How to Use Agent Memory Well Today
You do not need a memory platform to get most of the benefit.
- Start with one short instruction file. Put in build and test commands, naming conventions, and areas the agent should not touch.
- Write decisions, not diaries. Record why something was chosen, not a log of everything that happened.
- Review what the agent writes. Treat any instruction the agent adds to its own memory like a code change. In Claude Code, the /memory command shows what auto memory has saved.
- Prune on a schedule. Delete stale lines the same way you would remove dead code.
- Add a dedicated memory layer later. Reach for one only when plain files stop being enough, and test it on your own tasks instead of trusting a vendor benchmark.
FAQ
What is AI agent memory?
AI agent memory is any mechanism that lets an agent store and reuse information beyond one context window, such as instruction files, memory tools, or dedicated memory layers built on vector search or knowledge graphs.
Is a bigger context window the same as memory?
No. A larger window holds more text for one session, but it does not persist across sessions, and research on context rot shows models can become less reliable as inputs grow longer.
Do I need a vector database for agent memory?
Not necessarily. A plain markdown file such as AGENTS.md covers most coding projects, and a vendor-run test by Letta suggests simple file tools can compete with specialized memory layers on at least one benchmark.
What is the difference between RAG and agent memory?
RAG usually retrieves from an existing knowledge base such as documents or a codebase. Agent memory is information the agent itself records and reuses over time, such as decisions, preferences, and lessons from past tasks.
You Might Also Like
- What Is Repository Intelligence? How AI Coding Agents Understand Your Codebase
- AI Agent Orchestration Explained: How Multi-Agent Systems Work in 2026
Originally published on NextGen Tech.
Top comments (0)