DEV Community

mage0535
mage0535

Posted on • Originally published at hermes-agent.nousresearch.com

Knowledge-and-Memory-Management v0.0.2: Portable Knowledge Collection and Memory Management

#ai #automation #opensource

Knowledge-and-Memory-Management v0.0.2 is out, delivering a clean release that prioritizes portability and modularity. This version shifts from hardcoded personal paths to $AGENT_HOME, making your knowledge pipelines reproducible across environments. If you’re building autonomous systems that need to ingest web content, video transcripts, or articles, this is the update you’ve been waiting for.

The core design separates collection from memory management. The knowledge_collector module handles ingestion, while memory_manager handles storage, retrieval, and decay. The $AGENT_HOME environment variable anchors all runtime paths—no more hardcoded /home/user strings. Set it once, and your agents can carry their knowledge base anywhere.

Knowledge Collection: Web, Video, Articles

The collector supports three primary sources:

  • Web: Scrapes and parses HTML, extracting body text and metadata. Handles rate limiting and retry logic.
  • Video: Takes a YouTube URL, downloads captions (if available) or generates transcripts via Whisper integration.
  • Articles: Parses RSS feeds or direct PDF links, chunking content by sections.

All sources normalize into a KnowledgeEntry dict: {source, timestamp, content, embeddings}. The collector writes raw entries to $AGENT_HOME/knowledge/raw/ and passes them to the memory manager for processing.

Memory Management with $AGENT_HOME

The memory manager is where the clean release shines. Previous versions used os.path.expanduser("~/knowledge"), which broke across systems. v0.0.2 requires $AGENT_HOME to be set, then constructs all paths relative to it:

  • $AGENT_HOME/memory/ stores persistent memories.
  • $AGENT_HOME/knowledge/ holds raw and processed collections.
  • $AGENT_HOME/config/ contains source definitions and memory decay rules.

This design lets you ship a single agent.env file with AGENT_HOME=/opt/myagent or %AGENT_HOME%\data—no platform-specific configuration.

The memory manager indexes entries by semantic embeddings (via a pluggable model provider). It supports TTL-based decay and relevance pruning. Knowledge entries with low access frequency are archived after configurable intervals.

Code Example: Collecting and Storing a Web Article

Here’s a minimal example using the knowledge_collector and memory_manager jointly. Set $AGENT_HOME first, then run:

import os
from knowledge_collector import scrape_web
from memory_manager import index_entry

# Set portable home
os.environ.setdefault("AGENT_HOME", "/tmp/myagent")

# Collect from a URL
entry = scrape_web("https://example.com/technical-overview")
# entry is a dict: {
#   "source": "web",
#   "url": "...",
#   "content": "parsed text...",
#   "timestamp": 1728000000
# }

# Index into memory with automatic path resolution
memory_id = index_entry(entry)
print(f"Memory stored at {os.path.join(os.environ['AGENT_HOME'], 'memory', memory_id)}")
Enter fullscreen mode Exit fullscreen mode

The index_entry function calculates embeddings, writes the entry to $AGENT_HOME/memory/, and updates the retrieval index. Clean, testable, and environment-agnostic.

Why This Matters

For experienced developers building layered agents, path portability removes a major friction point. You can now containerize your knowledge pipeline and deploy without rewriting paths. The explicit separation between collection and memory also lets you swap storage backends (SQLite, Postgres, or plain files) by changing the memory manager provider—all without touching collection logic.

The $AGENT_HOME convention aligns with the broader trend toward self-contained agent runtimes. Pair it with environment-specific configs, and you get consistent behavior from local dev to production.

Get Started

Download the v0.0.2 release, set $AGENT_HOME, and point the collector at your first source. The changelog details the path migration and new memory decay parameters. Community contributions are welcome—especially for additional source parsers and backend connectors. Let’s make knowledge management as portable as the agents we build.

Top comments (0)