Your agent "forgets" between sessions, drifts on stale exports, or eats the context window on raw API payloads. Often the fix is memory and context design rather than a model upgrade. Below is one practical takeaway per episode from Chain of Thought's collection on how agents remember, in the order I'd revisit them when hardening production memory.
Why treat memory as its own discipline?
In Agent Memory: The Last Battleground in the AI Stack, Richmond Alake makes memory engineering a named practice next to prompt engineering and context engineering, with its own failure modes. Two he returns to often: the wrong mental model, and deleting records when you mean to forget them. His line for production is simple: don't delete, forget. Controlled decay, using the Generative Agents paper's mix of relevance, recency, and importance, is how he talks about implementing that.
He maps working, episodic, semantic, and procedural memory onto agent architecture, and segments what enters the window by type instead of one flat transcript. He demos converged search (vector, graph, spatial, relational) in a memory-aware financial services agent. Files work for prototypes; databases are what he points to when memory has to carry load. If you take one rule from the episode, retire or down-rank memories rather than hard-deleting because a screen got cluttered.
A sketch of the scoring idea (illustration only):
def memory_score(relevance: float, recency: float, importance: float) -> float:
return relevance + recency + importance
def should_surface(score: float, threshold: float) -> bool:
return score >= threshold # forget = below threshold, not DELETE
What breaks when the context feeding the agent is wrong?
Michel Tricot's thesis in Context Poisoning is Killing Your AI Agents: How to Stop it is that agents fail because the data reaching them is wrong, rather than because the model is bad. He demos a Gong query through raw API calls that burned 30,000 extra tokens and took about three minutes. The same query through a context store ran in about a minute and used a fraction of the context window.
Treat that as a pre-flight check before you add retrieval tricks: log what actually lands in context. Poisoned or bloated context reads as bad reasoning. Put a curated context layer in front of chatty integrations, especially when you join entities across Salesforce, Zendesk, and Gong. He also describes memory as another form of context; I would fix upstream context before anything fancier in memory.
What compounds if agent loops get commoditized?
Jerry Liu's The AI Framework Era Is Over: Why Context Is the Moat is a wager on what survives when harnesses absorb yesterday's scaffolding: context quality. He built LlamaIndex into widely installed plumbing, then argued the framework era is over while the context layer compounds.
On agent memory, he frames it as persistent context: what should still be available after the agent loop ends. If you are allocating engineering time, I'd overweight document and context fidelity (he names 95%+ accuracy as the bar for legal, insurance, and financial document work) over another orchestration wrapper. Persistent memory is only as trustworthy as the context you stored and can rehydrate.
What has to change when state outlives one conversation?
Mikiko Chandrasekhar on mastering multi-agent systems lays out MongoDB's pitch to be the memory store for agents once their state outlives the conversation. The data layer takes on vector search, durable storage, and debugging chat logs as multiple agents share state.
Her builder takeaway is to treat agents like software products. Reliability for non-deterministic systems still means observability, evaluations, and guardrails. When memory is shared across agents, the store is part of the product: version writes, log retrievals, and give operators a path from a bad answer to a bad memory write.
How do you stop bolting memory onto the side?
Aishwarya Srinivasan's Architecting AI Agents: The Shift from Models to Systems argues most agents are built backwards, starting with models instead of system architecture. She walks prompt engineering into context engineering, and says production agents need memory orchestrated alongside models and tool calls rather than bolted on later.
Benchmarks alone will not carry you; she pushes evaluation-driven development and human-in-the-loop limits on autonomy. On a whiteboard, put memory on the same diagram as tools and model routes. If memory is a late add-on box, new tools risk fighting your orchestration layer.
Checklist
- Name memory engineering explicitly: segment by memory type, and prefer forget-with-decay over delete in production (Richmond Alake, episode 53).
- Measure what enters the window before blaming the model; cut context poisoning and token waste with a store in front of noisy APIs (Michel Tricot, episode 52).
- Treat persistent agent memory as high-quality context that compounds, and hold document pipelines to the accuracy bar your vertical needs (Jerry Liu, episode 61).
- Design the data layer for shared, long-lived agent state, retrieval, and traceable chat history when chats end but work continues (Mikiko Chandrasekhar, episode 33).
- Orchestrate memory with models and tools in one architecture, and ship evals with the agent instead of after it (Aishwarya Srinivasan, episode 42).
More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.
Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.
Drafted with AI assistance from the episode transcripts.
Top comments (0)