From Hot to Cold Memory: Building a Persistent Knowledge Base for Your AI Agents

A few months ago, I hit a wall with my AI agent. It could handle real-time chat and simple RAG on a static prompt, but it had no memory of the 10,000+ articles, youtube lectures, and PDFs I’d consumed. I was essentially building an agent with amnesia.

I tried note-taking apps, read-it-later tools, and even custom scrapers, but nothing gave me a clean pipeline that fed directly into my agent’s memory. That’s why I started the Knowledge and Memory Management (KMM) plugin. It’s an open‑source extension for hermes-memory-installer that turns content consumption into a structured, queryable knowledge base.

What KMM Does

KMM is a full-stack pipeline: collect → process → store → sync. Under the hood it’s a Python plugin with 40+ collection tools, AI-powered note generation, three-tier memory, and rclone-backed cloud sync.

Collection Layer (40+ Tools)

Instead of siloed scrapers, KMM provides a unified interface for:

• Web scraping (9 engines: newspaper3k, trafilatura, readability, etc.)
• Video/audio (12 engines via yt-dlp, whisper transcription, frame extraction)
• Articles & RSS (10 engines: feedparser, mercury, etc.)
• Documents & OCR (10 engines: MarkItDown, pypdf, tesseract, etc.)

Each tool outputs a standardized document (title, content, metadata) that feeds directly into the analysis layer.

Analysis Layer

Once collected, content is processed with optional AI:

• Automatic note generation (summaries, key points)
• Knowledge graph extraction (entity/relation discovery)
• NLI fact-checking (contradiction detection)
• Book summarization with chapter breakdowns

Storage Layer: Three-Tier Memory

Your agent gets three levels of recall:
| Tier | Backend | Capacity | Use Case |
|------|---------|----------|----------|
| Hot | memory tool in-memory | N/A | Recent/frequent context |
| Warm | Hindsight (vector store) | ~10K nodes | Medium-term semantic search |
| Cold | gbrain (SQLite + embeddings) | ~11K pages | Long-term archival |

The notes_rag module handles semantic search across all tiers, returning the most relevant snippets to your agent.

Cloud Sync (12+ Drivers)

KMM uses rclone to sync your knowledge base to any cloud:

# Example: sync cold memory to OneDrive
kmm sync --from local --to onedrive:knowledge-base

Supported: OneDrive, Google Drive, Dropbox, Mega, pCloud, WebDAV, S3, 阿里云盘, 百度网盘, and more.

Getting Started in 5 Minutes

# Install the memory sidecar first
pip install hermes-memory-installer

# Clone KMM
 git clone https://github.com/mage0535/Knowledge-and-Memory-Management.git
 cd Knowledge-and-Memory-Management

# Create environment and install
python -m venv venv
source venv/bin/activate
pip install -e .

# Configure the paths (replace with your agent home)
export AGENT_HOME=/path/to/agent/data

# Collect a webpage
kmm collect https://example.com/interesting-article

# Now query your agent about it

The collected content is automatically processed, stored in the three-tier memory, and ready for semantic search.

When to Use This (and When Not To)

Use it if:

• You’re building a personal knowledge base for an AI agent or copilot
• You consume a lot of web/video/document content and want it searchable
• You need a portable, syncable memory across machines

Skip it if:

• Your agent already has a bespoke memory pipeline (this is an extension, not a standalone agent)
• You only need ephemeral, single-session memory
• You want a SaaS product with a managed UI (this is a CLI/plugin tool)

Why Open Source?

I built KMM because existing solutions were fragmented. The scrapers only collect, the note apps only store, the RAG tools don’t sync. This plugin ties it all together in a modular, MIT‑licensed way. It’s v0.0.2 so contributions are very welcome—especially new collectors, sync drivers, or embedding strategies.

If your agent should remember what you read, give Knowledge and Memory Management a star and see how it fits your stack. I’d love to hear your feedback.