DEV Community

Anindya Mukherjee
Anindya Mukherjee

Posted on

Why Does Your AI Agent Forget What It Did Five Minutes Ago?

You deploy your agent on Monday. It handles a support ticket beautifully — figures out the customer's billing issue, updates the record, sends a polite follow-up. Chef's kiss.

Tuesday, the same customer writes back with a follow-up question. Your agent responds like it's never seen this person before in its life. No memory of the ticket. No context. Just a cheerful "Hi there, how can I help you today?" like a golden retriever meeting its owner for the fifteenth time.

Your agent doesn't have amnesia. It was never designed to remember.

The Dirty Secret About Agent "Memory"

Most agent tutorials show you how to stuff conversation history into a context window. That works great for a single session. But the moment that session ends — poof. Gone. Your agent's "memory" was just a local variable that got garbage collected.

Think of it like this: conversation history is a Post-it note on your monitor. Useful right now, useless tomorrow. What you actually need is a filing cabinet.

# What most tutorials teach you (session memory)
messages = [{"role": "user", "content": "Fix my billing"}]
response = agent.run(messages)  # Works great... until the session ends

# What you actually need (persistent memory)
from datetime import datetime

class AgentMemory:
    def __init__(self, store):
        self.store = store  # Database, Redis, vector DB — anything persistent

    def remember(self, user_id: str, fact: str, category: str = "general"):
        self.store.save({
            "user_id": user_id,
            "fact": fact,
            "category": category,
            "timestamp": datetime.utcnow().isoformat(),
            "ttl_days": 90  # Don't hoard stale facts forever
        })

    def recall(self, user_id: str, query: str, limit: int = 5):
        return self.store.search(user_id=user_id, query=query, top_k=limit)
Enter fullscreen mode Exit fullscreen mode

The difference? One treats every interaction as a first date. The other actually learns who you are.

The Three Layers of Agent Memory You're Missing

Here's what production agents need — and what 90% of tutorials skip entirely.

Layer 1: Working Memory (the Post-it)

This is your context window. It holds the current conversation, recent tool results, and whatever the agent is actively thinking about. You already have this. It's not enough.

Layer 2: Episodic Memory (the Journal)

What happened in past interactions? Not the raw transcripts — nobody needs 50,000 tokens of old chat logs. You need distilled facts: "This user prefers email over Slack," "Last ticket was about a double charge on Oct 3," "Customer escalated twice — handle with care."

def distill_episode(transcript: str, llm) -> list[str]:
    """Extract durable facts from a conversation transcript."""
    prompt = f"""Extract 3-5 key facts from this support interaction 
    that would help a future agent serve this customer better.
    Focus on: preferences, unresolved issues, sentiment, decisions made.

    Transcript:
    {transcript}

    Return as a JSON list of strings."""

    facts = llm.generate(prompt)
    return json.loads(facts)

# After every session, distill and store
facts = distill_episode(session_transcript, llm)
for fact in facts:
    memory.remember(user_id=customer.id, fact=fact, category="episode")
Enter fullscreen mode Exit fullscreen mode

This is the layer that turns your agent from a stranger into someone who actually knows things. And it's embarrassingly simple to add.

Layer 3: Semantic Memory (the Knowledge Base)

General knowledge about your domain that isn't tied to one user. Pricing rules. Policy changes. Product quirks. The stuff that lives in your docs but your agent can never find when it needs to.

def build_semantic_memory(docs: list[str], vector_db):
    """Index domain knowledge for agent retrieval."""
    for doc in docs:
        chunks = split_into_chunks(doc, max_tokens=500)
        for chunk in chunks:
            embedding = embed(chunk)
            vector_db.upsert(
                text=chunk,
                vector=embedding,
                metadata={"source": doc.source, "updated": doc.last_modified}
            )

# At query time
relevant = vector_db.search(query=user_question, top_k=3)
context = "\n".join([r.text for r in relevant])
Enter fullscreen mode Exit fullscreen mode

Without this layer, your agent re-discovers the same information from scratch every single time. It's like a new hire who refuses to read the wiki.

The Mistake That'll Bite You: Hoarding Everything

The instinct is to save everything. Every message, every tool call, every intermediate thought. Don't.

An agent with infinite memory is an agent that drowns in noise. Last month's shipping policy doesn't help when the policy changed yesterday. A customer's 2023 preferences are probably wrong by now.

The fix: TTLs (time-to-live) on episodic facts, freshness scores on semantic chunks, and a simple rule — if the agent retrieves a fact and it contradicts current context, flag it for review instead of trusting it blindly.

The Two-Line Test

Want to know if your agent actually has memory? Ask it these two questions across separate sessions:

  1. "What did we talk about last time?"
  2. "What do you know about me?"

If the answer to both is some version of "I don't have that information," congratulations — you've built a very expensive stateless function.

The good news: adding persistent memory is a weekend project, not a rewrite. A vector database, a distillation prompt, and a retrieval step before each response. Three components. That's it.


What's the weirdest thing your agent has "forgotten" that cost you time or trust? Drop it in the comments — I'm collecting war stories for a follow-up.

Top comments (0)