Building Persistent Memory for AI Agents: A 4-Layer File-Based Architecture

As AI agents become more integrated into our workflows, one persistent challenge remains: memory. How do we give these agents the ability to recall past interactions, maintain context across sessions, and build on previous knowledge? I've been experimenting with different approaches to solve this, and today I'm sharing a practical 4-layer file-based memory architecture that works with any AI agent—whether you're using ChatGPT, Claude, Agent Zero, or local LLMs.

The Problem: Stateless AI Agents

By default, most AI agents are stateless. Each interaction starts fresh, with no recollection of past conversations. While this is great for privacy, it's terrible for productivity. Imagine having to re-explain your project structure every time you ask for help, or losing the context of a multi-step debugging session. This is where persistent memory comes in.

The Solution: A 4-Layer File-Based Architecture

After testing various approaches, I settled on a 4-layer file-based memory system. This architecture is simple, flexible, and works across different AI agents. Here's how it breaks down:

Layer 1: Session Logs

The foundation of our memory system is the session log. This is a plain text file that records every interaction between the user and the AI agent. Each entry includes:

• Timestamp
• User input
• AI response
• Metadata (e.g., session ID, tags)

Example structure:

2023-10-15 14:30:22
User: Can you explain how to implement OAuth2 in a Node.js app?
AI: Certainly! OAuth2 is an authorization framework that enables applications to obtain limited access to user accounts on an HTTP service...
Metadata: { session_id: "abc123", tags: ["oauth2", "nodejs", "authentication"] }

Layer 2: Context Summaries

While session logs are great for raw data, they're not efficient for quick lookups. That's where context summaries come in. These are concise, human-readable summaries of past interactions, stored in a separate file.

Example structure:

2023-10-15
- Discussed OAuth2 implementation in Node.js
- Covered client credentials flow, authorization code flow
- Mentioned popular libraries like passport-oauth2

Layer 3: Knowledge Graph

For more complex scenarios, we need a way to extract relationships and entities from our interactions. This is where the knowledge graph comes in. It's a JSON file that stores entities (e.g., people, projects, concepts) and their relationships.

Example structure:

{
  "entities": {
    "OAuth2": {
      "type": "concept",
      "related_to": ["Node.js", "passport-oauth2", "authorization code flow"]
    },
    "Node.js": {
      "type": "technology",
      "related_to": ["OAuth2", "Express.js", "JavaScript"]
    }
  },
  "relationships": [
    {
      "source": "OAuth2",
      "target": "Node.js",
      "type": "used_in",
      "description": "OAuth2 can be implemented in Node.js applications"
    }
  ]
}

Layer 4: User Preferences

Finally, we have a layer for user preferences. This JSON file stores user-specific settings, such as preferred response style, favorite tools, or common parameters.

Example structure:

json
{
  "user_id": "user123",