How do you make an AI agent actually remember?
For detailed breakdown read at sistava.com/en/insights
It is the question that inevitably surfaces once an AI system moves out of prototyping and into long-running production. Why does it forget a core constraint after a week? Why does it re-introduce itself every morning? Why does it pick the wrong tool even though it was corrected three days ago?
At Sistava, where you can hire autonomous AI employees, we had to solve this problem to survive. We run a workforce of around 1,000 AI employees in production, operating continuously across live environments for over two months. At this scale, standard context strategies fail. These systems don't get a polite session reset; they face a massive real-world hurdle: facts change over time.
If a user utilizes Gmail today and switches to Outlook next month, an agent needs to track both. It has to know which one is current, exactly when the switch happened, and it cannot act like the old truth is still valid. Standard vector database similarity scores do not understand chronological decay or truth overrides. Mix old and new context, and the agent confidently fabricates or forgets the one detail that mattered.
After extensive runtime experience scaling this workforce, the obvious answer - pick a vector store, dump text chunks in, and hope for the best - completely broke. Memory in a long-running agent isn't a single database. It requires at least seven distinct layers running in parallel across multiple database types.
The Architectural Split (The CoALA Framework)
The academic literature has already recognized these limitations. The seminal CoALA paper (Princeton, 2023) formalized the episodic, semantic, and procedural split from cognitive science for language model agents. It outlines modular components: working memory as a short-term scratchpad, plus long-term episodic for experiences, semantic for facts, and procedural for skills.
In a production environment, each of these layers requires its own write rules, its own lifecycle, and its own read path. They cannot run as a loose stack; they must be isolated so they do not contaminate one another.
Working Memory
This is the active, per-turn scratchpad holding the immediate plan-so-far, the raw tool output that just came back, or transient chain-of-thought reasoning. It lives entirely within the LLM's native context window or as an in-memory variable in the runtime environment.
The Production Lesson: Do not let working memory leak. Transient scratchwork must never accidentally flush into long-term storage, or the agent will begin writing unverified thoughts into its historical knowledge base. Enforce a hard wall - working memory has no persistent backing store. It lives, it dies, it is gone.Conversation Memory
This tracks the immediate message history so the agent doesn't have to re-derive the active thread context on every turn. Most modern agent frameworks ship a checkpointer that auto-loads thread history from a Postgres backend on invocation.
The Production Lesson: Run a summarizer middleware that triggers when the live conversation crosses a strict token threshold. It compresses older turns into a single structural system message while keeping the recent tail intact, maintaining a dense, cost-efficient context window.Episodic Memory
A time-indexed log of past execution loops, historical runs, and specifically, the failures ("Last Tuesday the webhook timed out, so I routed through the fallback queue"). It provides chronological continuity.
The Production Lesson: A vector store alone fails here because similarity scoring doesn't understand time. Store raw transcripts alongside LLM-generated execution summaries, keyed explicitly by thread_id and timestamp. Use a background cron job to truncate older episodes to summaries only, rather than forcing the agent to handle eviction at runtime.
sistava.com memory inspection4. Semantic Memory
This stores slow-changing, deterministic facts about the user, the business, or integrated tools ("The core platform is called Atlas", "The manager prefers brief markdown reports"). It is edited in place, never blindly appended.
The Production Lesson: Split this layer into two distinct substrates: a human-editable markdown file (the "Sovereign Notebook") and an LLM-extracted graph. If they disagree, the notebook explicitly wins. This gives operators a clear vector to intervene; if an extracted fact is noisy, a manual entry in the notebook out-votes the graph noise on equal footing.
- Knowledge Graph While semantic memory holds raw facts, the knowledge graph maps the structural edges between entities - who did what, which event caused what, or which entity is a duplicate of another. A vector store treats text chunks like isolated islands; a graph database (such as Neo4j, Memgraph, or KuzuDB) connects them. It allows an agent to walk contextually from a specific customer entity straight to the exact email thread where a pricing tier was modified without re-reading thousands of irrelevant chunks. AI Employee knowledge graph at sistava.com
Handling Changing Realities: Temporal Edges
The non-obvious requirement of the graph layer is temporal awareness. To handle shifting user preferences or infrastructure changes over months of runtime, you must stop deleting or overwriting data when state changes.
Instead, every extracted fact in the semantic and graph layers needs a valid_at and invalid_at timestamp:
(User) -[USES_TOOL {valid_at: "2024-01-01", invalid_at: "2026-02-15"}]-> (Gmail)
(User) -[USES_TOOL {valid_at: "2026-02-16", invalid_at: null}]--> (Outlook)
When today's session contradicts yesterday's state, the ingestion pipeline invalidates the old edge instead of erasing it. This preserves a clean, immutable audit trail, allowing the LLM to logically reason about when a preference shifted or an infrastructure stack was updated.
The Build vs. Buy Lesson: Do not write this temporal logic yourself. Utilize open-source libraries that sit on top of your graph DB to handle the LLM-driven extraction, deduplication, and contradiction detection. Writing relationship-inference engines from scratch can easily burn six months of development time.
Procedural Memory
Procedural memory stores execution mechanics and behavioral habits, not world facts. It dictates how an agent performs tasks ("When checking a raw CSV dataset, first validate header consistency").
This data lives in structured skill files (typically markdown documents) that the agent loads on demand based on task routing. Some are explicitly authored by engineers; others are written by the agent itself during asynchronous self-reflection steps.
The Production Lesson: Keep semantic and procedural data separated. A fact like "The client uses Slack" is semantic and belongs in the notebook. A rule like "When notifying via a webhook, format payload fields as snake_case" is procedural and belongs in a skill file.Checkpoints
Operating underneath all other layers is a highly serializable, low-latency snapshot of the exact execution state of an agent workflow. This is not thread history; it is the active node in the graph, the pending tool payloads, and the unwritten output stream.
It is the difference between a background container crashing and losing a forty-minute execution loop, or surviving a pod restart and picking up cleanly at minute thirty-three. Utilizing a durable execution engine like Temporal gives you deterministic checkpointing at every activity boundary out of the box.
Infrastructure Matrix & Preventing Contamination
To maintain performance, these layers require separate storage shapes, read patterns, and write triggers:
LayerStorage ShapeWrite TriggerRead PatternWorkingIn-memory scratchpadPer-turn executionNative context window injectionConversationAppend-only log + summarizerEvery incoming messageAuto-loaded on invocationEpisodicTime-indexed transcript + JSON summariesPost-message background workerRecency-weighted semantic retrievalSemantic (Notebook)Single editable Markdown fileExplicit agent tool writesFull text injected to promptSemantic (Facts)Graph DB (Neo4j class)Auto-extracted post-messageEntity-anchored sub-graph matchingKnowledge GraphGraph DB with temporal propertiesUnified extraction loop with factsContextual edge-walking between nodesProceduralMarkdown skill filesHuman authorship or reflectionDynamically loaded based on taskCheckpointsKV Store / Postgres / Workflow engineEvery single execution stepInstantly restored on worker restart
Preventing Contamination
Naming the layers is straightforward; wiring them without cross-contamination is where production pipelines fail.
Episodic leaking into Semantic: If every line of a historical brainstorming session gets extracted as a hard "fact," the agent will interpret a transient hypothetical idea as absolute truth. Enforce strict LLM confidence thresholds or run your fact extraction pipelines on summarized episodes rather than raw chat logs.
Conversation leaking into the Graph: Active conversation is full of throwaway syntax and short pleasantries. Ingesting every message verbatim fills a graph database with garbage nodes. Enforce length-gated ingestion filters to skip processing short, transactional messages.
Managing Upstream LLM Costs
An advanced knowledge graph ingestion pipeline requires between five to nine discrete LLM calls per message (handling entity extraction, graph deduplication, relationship inference, contradiction testing, and entity summary updates), alongside multiple embedding calls. Multiplied across thousands of active conversations running concurrently, background memory costs can quickly eclipse primary agent execution costs.
To keep this sustainable at scale, bake in kill switches and per-tenant gates from day one. Every layer running unattended in the background must have a configuration-level flag or a feature toggle. When an upstream model update or unexpected schema change causes an extraction loop to degrade or spin out of control, you need a way to stop the financial bleeding instantly without triggering an emergency production redeployment.
The Rebuild Blueprint
If you are starting over building an agent memory infrastructure today, this is the recommended development order:
Map the Concerns First: Do not select an orchestration framework based on hype. Map how your system will handle these seven distinct concerns before writing application logic.
Postgres for Foundations: Use Postgres for conversation history and step-level checkpointing. Boring, ACID-compliant storage is exactly what you want here.
Path-Routed KV for Filesystems: Implement a simple key-value store for notebooks and skill files, allowing the agent to interact with its procedural knowledge using clean, standard filesystem tool calls.
Native Graph + Temporal Constraints: Deploy a native graph database (Neo4j, Memgraph, or KuzuDB) paired with an off-the-shelf library that manages temporal constraints natively.
Tight Vector Tooling: Use a highly optimized vector store (pgvector, Qdrant, or Weaviate) specifically to index static external knowledge documents like Notion workspaces, Slack history, or uploaded manuals.
Ultimately, separating transient reasoning from immutable history and structured relational facts is what transforms a fragile chatbot into a reliable system. By treating memory as a multi-layered infrastructure concern, you build an environment where an agent's capability doesn't degrade over time , it compounds.
sistava.com knowledge ingestionBuilding Agent Memory Yourself?
The seven layers, the wiring, and the cost ceilings are a lot to get right on the first run.
If you want this exact architecture adapted to your tech stack, check out our support options at Sista AI. If you would rather talk engineer-to-engineer, I take a few of these architectural deep dives personally. You can reach me directly at zalt.me.
Top comments (0)