Building Persistent Memory for AI Agents: A 4-Layer File-Based Architecture

The Problem: AI Agents Forget Everything

When I first started building AI agents, I quickly hit a frustrating wall: they forget everything between sessions. Whether it was a ChatGPT plugin, a local LLM running on my machine, or Agent Zero orchestrating tasks, each interaction felt like starting from scratch. This wasn't just inconvenient—it broke workflows that required continuity.

I needed a way to give AI agents persistent memory—a way to store knowledge, context, and state that persists across sessions. After experimenting with various approaches (vector databases, key-value stores, etc.), I landed on a 4-layer file-based architecture that works with any AI agent, from cloud-based models to local LLMs.

Here's how it works, and how you can implement it too.

---

The 4-Layer Memory Architecture

This architecture is inspired by how humans (and some animals) remember things: short-term memory, long-term memory, episodic memory, and procedural memory. Translating that into a file-based system, we get:

1. Short-Term Memory (Working Memory)
2. Long-Term Memory (Knowledge Base)
3. Episodic Memory (Session Logs)
4. Procedural Memory (Action Logs)

Each layer serves a distinct purpose, and together they create a robust, persistent memory system.

---

Layer 1: Short-Term Memory (Working Memory)

Purpose: Temporary storage for the current session. This is where the AI agent keeps track of ongoing tasks, intermediate results, and context.

Implementation:
A JSON file (working_memory.json) that gets reset at the start of each session but persists during the session.

{
  "current_task": "analyze user preferences",
  "context": {
    "user_id": "user123",
    "preferences": {
      "theme": "dark",
      "notifications": true
    }
  },
  "intermediate_results": {
    "sentiment_score": 0.85,
    "topics": ["AI", "productivity"]
  }
}

Why JSON?

• Easy to read/write in most programming languages.
• Supports nested structures for complex context.
• Can be invalidated (reset) at the start of a new session.

Code Example (Python):

import json
import os

WORKING_MEMORY_FILE = "working_memory.json"

def load_working_memory():
    if os.path.exists(WORKING_MEMORY_FILE):
        with open(WORKING_MEMORY_FILE, "r") as f:
            return json.load(f)
    return {}

def save_working_memory(data):
    with open(WORKING_MEMORY_FILE, "w") as f:
        json.dump(data, f)

def reset_working_memory():
    if os.path.exists(WORKING_MEMORY_FILE):
        os.remove(WORKING_MEMORY_FILE)

---

Layer 2: Long-Term Memory (Knowledge Base)

Purpose: Permanent storage for facts, rules, and general knowledge. This is the AI agent's "brain" for things it should never forget.

Implementation:
A directory (knowledge_base) containing text files, Markdown files, or structured data (like JSON) organized by topic.

knowledge_base/
├── facts.json
├── rules.md
├── user_profiles/
│   ├── user123.json
│   └── user456.json
└── tools/
    ├── available_tools.json
    └── tool_descriptions.md

Example (facts.json):