If you've ever built an AI agent and watched it forget something you told it three steps ago, you've run into the memory problem. It's one of the most common reasons agents feel unreliable, and it's easier to understand with a kitchen than with a diagram.
The kitchen counter
Picture a chef working through a recipe. The counter in front of them holds everything for the current dish — the ingredients they just chopped, the pan that's heating up, the note they scribbled about oven temperature. It's all right there, easy to reach, but it only holds so much. And once the dish is done and the counter is wiped down, most of it is gone.
That's short-term memory in an agent. It's the current conversation, the last few tool results, the immediate context the agent is working with. Fast to access, always relevant to the task at hand, but limited in size and temporary. Once the session ends, it's cleared.
The recipe book on the shelf
Now picture the same chef's recipe book. It's not on the counter — it's on a shelf nearby, and the chef only opens it when they need to check something specific: how long to roast at this temperature, what quantity of an ingredient this recipe usually calls for. It's not something they're holding in their hands the whole time. It's something they deliberately go look up.
That's long-term memory in an agent. It's information that needs to persist across sessions — user preferences, past decisions, facts learned earlier — stored somewhere separate from the current task and retrieved on purpose when it's relevant, not carried around by default.
Where this goes wrong
A lot of agents that "forget" things aren't actually failing to store information — they're storing everything on the counter and nothing on the shelf. Every fact lives in the conversation transcript, which works until the counter gets too full or the session resets, and then it's all gone at once.
The fix isn't more counter space. It's building an actual shelf — a separate store the agent can write to and query deliberately, so a fact learned in one session is still there in the next one, instead of depending on the whole transcript being remembered word for word.
I went through this exact failure while building an agent and wrote up the fix in more detail in this piece on designing agent memory, including what the actual read-and-write pattern looks like.
Why this matters for what you're building
If your agent only ever needs to handle one task in one sitting, short-term memory alone might be fine — there's nothing worth putting on the shelf. That's often closer to a simple assistant than a full agent anyway; the line between the two usually comes down to exactly this kind of thing — how much it needs to track and decide on its own over time.
If your agent needs to remember a user's preferences across sessions, track the state of an ongoing process, or build on decisions it made earlier, you need the shelf. Trying to fake that with a longer and longer counter — stuffing more into the context window instead of building real storage — gets expensive and unreliable fast.
The short version
Short-term memory is what the agent is actively holding right now. Long-term memory is what it can go look up when it needs to. Most memory bugs come from treating the two as the same thing.
More on agent design and the mistakes that show up while building one over at Procwire.
Top comments (0)