A raw model is stateless. It only knows what's in its context window right now, and that window is both finite and volatile — it fills up on a long task, and it's wiped clean when the session ends. So the moment you come back tomorrow, the agent has forgotten your name. Memory is the machinery that fixes that.
🧩 Play with the two-tier memory demo: https://dev48v.infy.uk/ai/days/day30-agentic-memory.html
Two tiers, like a computer
Agent memory mirrors a machine. There's working memory — the live context window. It's small, fast, and thrown away at the end of the session. And there's long-term memory — an external store that's larger, durable, and searchable. The agent constantly shuttles information between them: promote a salient fact up to long-term (a write), and pull a relevant one back down into context when it's needed (a read).
The demo lets you send messages and watch both tiers react. Tell it "my name is Ada", "I prefer window seats", "always cc my manager" — then watch each fact get written, watch the context window fill up, hit New session to wipe the context, and ask "what's my name?" The answer comes back from long-term, even though the context is empty.
The context window is RAM
Working memory is the context window, and it behaves like RAM: capacity is measured in tokens, and everything the model reasons over has to fit — system prompt, tool results, the running summary, the last few turns, the new message. Go over budget and something has to give. This one hard constraint forces every other memory decision. You can't just keep appending forever.
Long-term is RAG pointed at yourself
Long-term memory is just retrieval-augmented generation aimed at the agent's own history. Every fact worth keeping gets embedded into a vector and stored. When a new turn arrives, you embed it too and fetch the most similar memories by cosine similarity. So a preference you mentioned 500 turns ago resurfaces exactly when it's relevant — without dragging all 500 turns back into context. Retrieval, not scrolling, is what makes it scale.
A write policy: remember less, remember better
Storing every message is just a transcript. It bloats the store and buries the signal. A good agent applies a write policy — keep the durable, reusable stuff (your name, preferences, standing instructions) and skip the ephemeral (chit-chat, one-off questions). In the demo, statements that carry facts get written; questions trigger a retrieval but store nothing. Less memory, but higher value, retrieves far more cleanly.
Three flavors of long-term memory
Borrowed straight from cognitive science:
- Semantic — distilled, timeless facts. "The user's name is Ada."
- Episodic — specific events that happened. "Shipped the release on Tuesday."
- Procedural — how-to rules and standing instructions. "Always cc the manager."
Tagging by type lets you retrieve, expire, and prioritise each one differently.
Reflection: summarise when it overflows
When working memory blows past its token budget, the agent doesn't just truncate — it reflects. It folds the oldest turns into a compact running summary and keeps only the most recent turns verbatim. Done well, this is lossy on phrasing, not on knowledge, because anything important was already distilled into long-term. In the demo, drop the capacity slider and watch compaction kick in live.
Forgetting is a feature
A store that only grows becomes slow and full of stale, contradictory junk. So memory has to forget. De-duplicate near-identical entries. Let a newer fact supersede a conflicting older one — say "my name is Bob" now — so the current truth wins. Decay trivia by time-to-live while key facts persist. Forgetting isn't a bug; it's how memory stays accurate.
You don't hand-roll this
In practice you reach for LangGraph (state checkpointers), MemGPT / Letta (OS-style context paging that inspired the whole pattern), or mem0 (a drop-in memory layer). And because memory is user data, privacy is first-class: redact PII before storing, scope memory per user, and support "forget me".
Every write, embed, cosine-retrieval and compaction in the demo is real code running in your browser — send it a fact, clear the session, and watch it still remember: https://dev48v.infy.uk/ai/days/day30-agentic-memory.html
Part of AIFromZero. 🌐 https://dev48v.infy.uk
Top comments (0)