You tell your agent, for the third time, that the project uses pnpm, not npm. It apologizes, fixes the command, and the next morning it runs npm install again.
That isn't a bug in the model. It's the default. A language model doesn't keep anything between calls. The CoALA paper (Cognitive Architectures for Language Agents) puts it in one line: "Language models are stateless: they do not persist information across calls." Anything that looks like memory is something the system around the model stores and feeds back in.
So when people say "my agent needs memory," they usually mean one of four different things. Mixing them up is why so many memory setups end up as a giant notes file that the agent half-reads and nobody trusts. This article walks through the four kinds, what each one fixes, and how to start small.
Why "just give it memory" goes wrong
The most common first attempt is a single place where everything goes: user preferences, past conversations, project facts, instructions. It works for a week. Then the file grows, old facts contradict new ones, and the agent starts acting on something that stopped being true a month ago.
The fix isn't a bigger store. It's separating information by what it's for. CoALA, which borrows its vocabulary from decades of cognitive-architecture research, splits agent memory into short-term working memory and three long-term kinds: episodic, semantic and procedural. You don't need to read the paper to use the split, but it's a useful map.
1. Working memory: what the agent is holding right now
Working memory is the information that's active for the current step: the user's request, the files it just read, the result of the last tool call, its current goal. In practice, most of it lives in the context window.
Symptom when it's the problem: the agent loses track mid-task. It forgets a constraint you gave ten messages ago, or repeats a step it already did after a long tool output pushed things out of view.
What fixes it: less noise, not more storage. Keep tool outputs short, summarize long results before they land in the conversation, and move side tasks out of the main thread. A short running task list the agent updates as it goes is a cheap way to keep the goal in view.
2. Episodic memory: what happened before
Episodic memory is a record of past experience. CoALA describes it as storing "experience from earlier decision cycles": past conversations, previous runs, what was tried and how it turned out.
Symptom when it's missing: every session starts from zero. The agent can't answer "what did we decide last Tuesday?" and repeats an approach that already failed.
What fixes it: log interactions or run summaries with a timestamp and a subject, and retrieve the relevant ones when a new task starts. The key word is relevant. Replaying entire transcripts is expensive and mostly noise. A short summary per session ("tried X, it failed because Y, switched to Z") is usually what you actually want back.
3. Semantic memory: what the agent knows to be true
Semantic memory holds facts about the world and about the agent itself, in CoALA's words "an agent's knowledge about the world and itself". For a coding agent, that means things like "this repo uses pnpm", "the staging database is read-only", or "the team prefers short commit messages".
Symptom when it's missing: the pnpm problem from the top of this article. Corrections don't stick, and you repeat the same facts every session.
What fixes it: distill facts out of episodes. One messy conversation where you corrected the package manager three times should become one clean line: "Package manager: pnpm." That's the step most setups skip. They store the conversation and hope retrieval finds the right sentence later.
Semantic memory also needs updates, not just additions. When a fact changes, the old one has to go, or the agent will eventually retrieve the stale version. A store that only ever grows will start contradicting itself.
4. Procedural memory: how the agent does things
Procedural memory is know-how: rules, workflows and the code that drives the agent. CoALA counts two forms: the implicit knowledge in the model's weights, and explicit knowledge written into the agent's code. For most builders, the part you control is the explicit one: system prompts, tool definitions, and instruction files.
If you use Claude Code, you've already worked with this. Anthropic's docs on how Claude remembers your project describe two mechanisms that carry knowledge across sessions: CLAUDE.md files, which are "instructions you write", and auto memory, "notes Claude writes itself based on your corrections and preferences". Both are loaded at the start of every conversation. The docs also note that Claude treats them as context, not enforced configuration, and that the more specific and concise your instructions are, the more consistently they're followed.
Symptom when it's the problem: the agent knows the facts but does the job the wrong way, like skipping tests before a commit or using the wrong deploy steps.
What fixes it: write the procedure down once, in the place the agent loads every time, and keep it short. Be careful about letting the agent rewrite its own procedures unsupervised. CoALA flags exactly this: writing to procedural memory is "significantly riskier" than writing to episodic or semantic memory, because it can easily introduce bugs.
A quick way to diagnose your agent
When your agent "forgets," ask which question it failed:
- Lost track during the task? Working memory. Reduce noise in the context.
- Doesn't know what happened last time? Episodic. Store session summaries and retrieve the relevant ones.
- Keeps getting a fact wrong after being corrected? Semantic. Distill corrections into clean, updatable facts.
- Knows the facts but does the job wrong? Procedural. Fix the instructions it loads every session.
Most real problems are one of these, and the fix for one rarely helps the others. A vector database won't fix a bad system prompt, and a longer system prompt won't remember last Tuesday.
Start small: a minimal memory loop
You don't need a framework to try this. Here's a deliberately tiny sketch of the loop: extract a fact, store it, retrieve it, use it, and replace it when it changes. It uses a plain dictionary keyed by subject, which handles the "update, don't append" rule for free.
facts = {} # semantic memory: one current value per subject
def remember(subject: str, value: str) -> None:
# Overwrite instead of appending, so stale facts disappear.
facts[subject] = value
def recall(subjects: list[str]) -> str:
lines = [f"- {s}: {facts[s]}" for s in subjects if s in facts]
return "Known facts:\n" + "\n".join(lines) if lines else ""
# After a session where the user corrected the agent:
remember("package_manager", "pnpm")
# At the start of the next task, inject only what's relevant:
system_prompt = "You are a coding assistant.\n" + recall(["package_manager"])
In a real agent, the extraction step would be a model call that turns a conversation into facts like package_manager: pnpm, and retrieval would match by relevance instead of exact keys. But the shape stays the same, and starting with something this simple makes it obvious what each piece is for.
Where to go next
If you want the whole mental model in one sitting, the free AI Agent Memory Quick-Start covers the four kinds of memory, the memory loop (extract, store, retrieve, use, maintain), and why memory retrieval is RAG pointed at the agent's own past, in 6 pages you can read in about fifteen minutes.
When you're ready to build it properly, AI Agent Memory: The Complete Guide goes deep on writing and reading memories, consolidation and forgetting, tiered, vector and graph architectures, and the production failure modes you'll actually hit: bloat, poisoning and contradictions.
This article was written with the assistance of AI and reviewed by the author.
Top comments (4)
I like the distinction between remembering facts and remembering how to do a task. A vector database might help an agent find past information, but it won’t fix outdated instructions or a workflow that needs updating. Separating those memory types makes it easier to identify what’s actually going wrong.
Thanks, Will. That’s exactly the trap I wanted to point at: when an agent misbehaves, the reflex is to add retrieval, but if the facts are right and the workflow is wrong, retrieval just hands it the right facts to use the wrong way. Asking “did it forget something, or does it do the job wrong?” usually tells you where to look first.
Forgetting and re-explaining are the same bill: a fact the agent loses comes back as tokens each time it is needed. A store the agent reads back cheaply beats repeating it in the prompt.
Good way to put it, Paul. A lost fact costs twice: the tokens to explain it again, and the mistakes it makes before you notice. The one thing I’d add is that a stored fact has to stay current. A memory that keeps an outdated fact can cost more than one that forgot it, because the agent acts on it with confidence.