The biggest limitation of AI agents isn't intelligence. It's memory.
Your agent forgets the conversation from yesterday. It forgets the decisions you made last week. It starts fresh every time, like a goldfish with a PhD.
Here's how to fix it.
Why Memory Matters
Without memory, your agent:
- Repeats mistakes you've already corrected
- Asks questions you've already answered
- Forgets your preferences
- Can't learn from experience
- Treats every session like meeting you for the first time
With memory, it becomes a colleague who knows your work.
Three Types of Memory
1. Conversation Memory (Short-term)
What was said in the current session.
Most AI tools handle this automatically. The context window holds recent messages. But there's a limit — usually 100K-200K tokens for Claude.
Problem: Long conversations push out older context. The agent "forgets" the beginning.
Solution: Summarize periodically.
def maybe_summarize(messages, max_tokens=50000):
if count_tokens(messages) > max_tokens:
summary = summarize_conversation(messages[:-10])
return [{"role": "system", "content": summary}] + messages[-10:]
return messages
Keep recent messages raw. Compress old ones into summaries.
2. Session Memory (Medium-term)
What happened in recent sessions. Today, yesterday, this week.
Implementation: Save key information after each session.
def save_session_memory(session_id, content):
memory_file = f"memory/sessions/{session_id}.json"
data = {
"date": datetime.now().isoformat(),
"summary": summarize(content),
"decisions": extract_decisions(content),
"action_items": extract_actions(content),
"key_facts": extract_facts(content)
}
save_json(memory_file, data)
At session start, load recent session memories:
def load_recent_context():
sessions = get_recent_sessions(days=7)
context = "Recent activity:\n"
for s in sessions:
context += f"- {s['date']}: {s['summary']}\n"
return context
3. Long-term Memory (Permanent)
Facts that should persist forever. User preferences. Project context. Learned patterns.
Implementation: Structured knowledge base.
# memory/long_term/user_preferences.md
"""
- Prefers concise responses
- Uses Python, not JavaScript
- Works in PST timezone
- Hates em dashes
"""
# memory/long_term/project_context.md
"""
- Main project: AutoMate AI
- Stack: Python + n8n + Claude
- Goal: $100 -> $10,000
"""
Load relevant long-term memory based on current task:
def get_relevant_memory(current_task):
# Simple: load everything
# Better: embed and retrieve similar memories
memories = load_all_memories()
return filter_relevant(memories, current_task)
The CLAUDE.md Pattern
For Claude Code specifically, CLAUDE.md acts as permanent memory.
# Project Memory
## My Preferences
- Always use type hints in Python
- Commit messages in conventional commit format
- Don't add comments unless non-obvious
## Recent Decisions
- 2024-01-15: Chose n8n over Make for automation
- 2024-01-12: Using Claude API instead of OpenAI
## Current Context
- Working on content automation
- 15/122 articles written
- Gumroad product live at [link]
This file loads automatically. The agent "remembers" across sessions.
Memory Retrieval Strategies
Naive: Load Everything
context = load_all_memories()
prompt = f"{context}\n\nUser: {user_message}"
Works for small memory stores. Breaks when you have megabytes of history.
Better: Keyword Matching
def get_relevant(query, memories):
keywords = extract_keywords(query)
return [m for m in memories if any(k in m for k in keywords)]
Fast but misses semantic relationships.
Best: Embedding Search
from anthropic import Anthropic
import numpy as np
def get_relevant_memories(query, k=5):
query_embedding = embed(query)
scores = []
for memory in all_memories:
score = cosine_similarity(query_embedding, memory.embedding)
scores.append((score, memory))
scores.sort(reverse=True)
return [m for _, m in scores[:k]]
Find semantically similar memories even if words don't match.
Memory Decay
Not all memories should live forever.
def apply_decay(memories):
now = datetime.now()
for memory in memories:
age_days = (now - memory.created).days
# Session memories decay after 30 days
if memory.type == "session" and age_days > 30:
if memory.importance < 0.5:
memory.delete()
# Long-term memories persist but can be archived
if memory.type == "long_term" and age_days > 180:
if not recently_accessed(memory):
memory.archive()
Keep relevant memories fresh. Archive the rest.
Practical Example
My writing agent:
Start of session:
- Load CLAUDE.md (permanent preferences)
- Load session memory from yesterday
- Load relevant long-term facts about current project
During session:
- Maintain conversation context
- Periodically summarize if context gets long
- Extract and save important decisions
End of session:
- Save session summary
- Update long-term memory with new learnings
- Clean up temporary working files
Result: The agent "remembers" that I'm working on 122 articles, what the Gumroad product is, my writing style preferences, and what we decided about the content strategy.
Memory Limits
Even with good memory systems, there are constraints:
- Context window: Can't load all memories at once
- Relevance: More memories = more noise
- Staleness: Old memories can mislead
Balance comprehensiveness with focus. Load what's needed for the current task, not everything ever recorded.
Build vs Buy
Build your own: Full control. Fits your workflow. More work.
Use existing tools:
- mem0: Memory layer for AI agents
- LangChain memory modules
- Pinecone/Weaviate for vector storage
For most use cases, start simple (file-based), add sophistication when needed.
Complete memory system implementation — storage patterns, retrieval strategies, CLAUDE.md templates — is in AI Automation Blueprint 2026. $29 for the full architecture.
Top comments (0)