The agent we built in Post #4 has one big problem — the moment the script ends, it forgets everything. Next time you run it, it starts from zero. No memory of past conversations, no retained facts, nothing.
For a quick experiment that's fine. For anything you'd actually use, it's a dealbreaker. Memory is what makes an assistant feel like it knows you over time.
Three kinds of memory
There are three different ways to give an agent memory, and they serve different purposes.
Short-term memory is what we've been using all along — the messages list. It's fast and immediate, but it only lasts for the current session and is limited by the context window. When the conversation grows too long to fit, you have to make a choice: drop old messages, compress them, or archive them somewhere.
Long-term memory is persistent storage outside the model. Facts, past conversations, anything you want to survive across sessions — stored as embeddings in a vector database and retrieved when relevant. This is exactly what we did in Post #3 with documents, just applied to memories instead.
Episodic memory is structured records of what happened in past sessions — not raw facts, but events and context. Think of it as a journal: "On Monday, the user asked about X, we went through Y, and they got stuck at Z." Keyed by session or date, easy to look up chronologically.
Production systems use all three together. We'll build each one in the exercises below.
When context gets too long
The simplest approach to a growing context is a sliding window — keep only the last N messages and drop everything older. It's easy to implement but brutal: information falls off a cliff the moment it slides out of the window.
A better approach is summarisation. When the history gets too long, you ask the model to compress it into a few sentences, keep that summary, and start fresh. You lose some nuance but preserve the important thread of the conversation.
Setup
pip install ollama chromadb
ChromaDB has two modes. We've been using the in-memory one so far:
# In-memory — lost on restart (what we used in Posts #3 and #4)
client = chromadb.Client()
# Persistent — survives restarts (what we need for memory)
client = chromadb.PersistentClient(path="./memory_store")
The persistent client just writes to a folder on disk. No extra infrastructure needed.
Exercise 1 — Sliding window
Keep only the last N messages in context:
import ollama
def chat_with_sliding_window(max_history: int = 6):
history = []
system = "You are a helpful assistant."
print(f"Sliding window chat (last {max_history} messages kept). Type 'quit' to exit.\n")
while True:
user_input = input("You: ").strip()
if user_input.lower() == "quit":
break
history.append({"role": "user", "content": user_input})
windowed = history[-max_history:]
response = ollama.chat(
model="llama3.2",
messages=[{"role": "system", "content": system}] + windowed
)
reply = response.message.content
history.append({"role": "assistant", "content": reply})
print(f"AI: {reply}")
print(f"[Sending {len(windowed)} of {len(history)} total messages]\n")
chat_with_sliding_window()
Tell it your name early, have 8+ turns of unrelated chat, then ask "what's my name?" — it forgets. That's the window cutting off old context. Simple and easy to implement, but you can feel the hard cutoff.
Exercise 2 — Conversation summarisation
Instead of dropping old messages, compress them:
import ollama
def summarise_history(history: list[dict]) -> str:
conversation = "\n".join(
f"{m['role'].upper()}: {m['content']}" for m in history
)
response = ollama.chat(
model="llama3.2",
options={"temperature": 0},
messages=[{
"role": "user",
"content": f"""Summarise this conversation in 3–4 sentences.
Capture key facts, decisions, and current context.
Write as a neutral summary, not as a participant.
Conversation:
{conversation}
Summary:"""
}]
)
return response.message.content
def chat_with_summarisation(compress_after: int = 8):
history = []
summary = ""
system = "You are a helpful assistant."
print(f"Summarisation chat (compresses after {compress_after} messages). Type 'quit' to exit.\n")
while True:
user_input = input("You: ").strip()
if user_input.lower() == "quit":
break
history.append({"role": "user", "content": user_input})
if len(history) > compress_after:
print("\n[Compressing history...]\n")
summary = summarise_history(history[:-2])
history = history[-2:]
print(f"[Summary: {summary[:100]}...]\n")
messages = [{"role": "system", "content": system}]
if summary:
messages.append({
"role": "system",
"content": f"Summary of earlier conversation:\n{summary}"
})
messages.extend(history)
response = ollama.chat(model="llama3.2", messages=messages)
reply = response.message.content
history.append({"role": "assistant", "content": reply})
print(f"AI: {reply}")
print(f"[History: {len(history)} msgs | Summary: {'yes' if summary else 'no'}]\n")
chat_with_summarisation()
Run the same long conversation as Exercise 1 and compare. Summarisation preserves the thread much better — but notice how specific details sometimes get dropped in the compression. That's the tradeoff.
Exercise 3 — Long-term vector memory
Now let's build memory that persists across script restarts:
import ollama
import chromadb
import uuid
import datetime
client = chromadb.PersistentClient(path="./memory_store")
memory = client.get_or_create_collection("long_term_memory")
def store_memory(content: str, source: str = "conversation"):
emb = ollama.embeddings(model="nomic-embed-text", prompt=content).embedding
memory.add(
ids=[str(uuid.uuid4())],
embeddings=[emb],
documents=[content],
metadatas=[{"source": source, "timestamp": datetime.datetime.now().isoformat()}]
)
print(f" [Stored: {content[:60]}]")
def retrieve_memories(query: str, top_k: int = 3) -> list[str]:
if memory.count() == 0:
return []
emb = ollama.embeddings(model="nomic-embed-text", prompt=query).embedding
results = memory.query(
query_embeddings=[emb],
n_results=min(top_k, memory.count())
)
return results["documents"][0]
def chat_with_memory():
system = "You are a helpful assistant with access to memories from past conversations."
print(f"Memory chat. Existing memories: {memory.count()}")
print("Commands: 'remember: <fact>' | 'recall: <topic>' | 'quit'\n")
while True:
user_input = input("You: ").strip()
if user_input.lower() == "quit":
break
if user_input.lower().startswith("remember:"):
fact = user_input[9:].strip()
store_memory(fact, source="manual")
print("AI: Got it, I'll remember that.\n")
continue
if user_input.lower().startswith("recall:"):
topic = user_input[7:].strip()
memories = retrieve_memories(topic)
if memories:
print("AI: Here's what I remember:")
for m in memories:
print(f" - {m}")
else:
print("AI: Nothing stored about that yet.")
print()
continue
relevant = retrieve_memories(user_input)
messages = [{"role": "system", "content": system}]
if relevant:
memory_text = "\n".join(f"- {m}" for m in relevant)
messages.append({
"role": "system",
"content": f"Relevant memories from past conversations:\n{memory_text}"
})
messages.append({"role": "user", "content": user_input})
response = ollama.chat(model="llama3.2", messages=messages)
reply = response.message.content
store_memory(f"User said: {user_input}")
print(f"AI: {reply}")
print(f"[{len(relevant)} memories retrieved | {memory.count()} total]\n")
chat_with_memory()
Tell it a few facts, quit, run it again. It remembers. After running, check the ./memory_store/ folder — it's just files on disk. The memory grows with every run, and retrieval is semantic, so it finds relevant memories even when the wording doesn't match exactly.
Exercise 4 — Agent with memory tools
Finally, let's give the ReAct agent from Post #4 the ability to store and recall memories as tools:
import ollama
import chromadb
import uuid
import datetime
client = chromadb.PersistentClient(path="./agent_memory")
memory = client.get_or_create_collection("agent_memory")
def store_memory(content: str) -> str:
emb = ollama.embeddings(model="nomic-embed-text", prompt=content).embedding
memory.add(
ids=[str(uuid.uuid4())],
embeddings=[emb],
documents=[content],
metadatas=[{"timestamp": datetime.datetime.now().isoformat()}]
)
return "Memory stored."
def recall_memory(query: str) -> str:
if memory.count() == 0:
return "No memories stored yet."
emb = ollama.embeddings(model="nomic-embed-text", prompt=query).embedding
results = memory.query(query_embeddings=[emb], n_results=min(3, memory.count()))
docs = results["documents"][0]
return "\n".join(f"- {d}" for d in docs) if docs else "Nothing relevant found."
def calculate(expression: str) -> str:
try:
return str(eval(expression))
except Exception as e:
return f"Error: {e}"
TOOLS = {"recall_memory": recall_memory, "store_memory": store_memory, "calculate": calculate}
tools = [
{
"type": "function",
"function": {
"name": "recall_memory",
"description": "Search past memories for relevant information from previous conversations.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "What to search for in memory"}
},
"required": ["query"]
}
}
},
{
"type": "function",
"function": {
"name": "store_memory",
"description": "Save an important fact to memory for future conversations.",
"parameters": {
"type": "object",
"properties": {
"content": {"type": "string", "description": "The fact to remember"}
},
"required": ["content"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate",
"description": "Perform arithmetic calculations.",
"parameters": {
"type": "object",
"properties": {
"expression": {"type": "string", "description": "Python math expression"}
},
"required": ["expression"]
}
}
}
]
def run_agent(task: str, max_iterations: int = 8):
print(f"\nTask: {task}")
print("=" * 60)
messages = [
{
"role": "system",
"content": (
"You are a helpful assistant with memory tools. "
"Use recall_memory to check if you have seen relevant information before. "
"Use store_memory to save important facts for future conversations. "
"Use calculate for math. Answer clearly when done."
)
},
{"role": "user", "content": task}
]
for i in range(max_iterations):
response = ollama.chat(model="qwen2.5", messages=messages, tools=tools)
if response.message.tool_calls:
messages.append(response.message)
for tool_call in response.message.tool_calls:
name = tool_call.function.name
args = tool_call.function.arguments
print(f" → {name}({args})")
result = TOOLS[name](**args)
print(f" ← {str(result)[:120]}")
messages.append({"role": "tool", "content": str(result)})
else:
print(f"\nAnswer: {response.message.content}")
return
print("[Max iterations reached]")
# First run
run_agent("My name is Alex and I prefer Python over JavaScript. Remember this.")
# Quit and restart the script, then run this:
run_agent("What do you know about my programming preferences?")
Quit after the first run and restart the script before running the second task. The agent should recall what was stored in the previous session. Watch whether it proactively uses recall_memory at the start — that's the behaviour you want from a memory-aware agent.
Wrapping up
Memory turns a stateless tool into something that feels like it actually knows you. The sliding window is the simplest approach but the most limited. Summarisation is a good middle ground for long conversations. Persistent vector memory is what you need for anything that spans multiple sessions.
In practice, most real systems combine all three — recent messages in context, long-term facts retrieved from a vector store, and session summaries for episodic context. We've now built each piece separately, which means you can mix and match them as needed.
In Post #6, we take a different direction — instead of one agent doing everything, we build multiple specialised agents that work together. See you there. 🚀
Top comments (0)