The Knowledge-and-Memory-Management project provides a streamlined system for aggregating, storing, and querying information from web, video, and article sources. Version 0.0.2 is a clean release that replaces all hardcoded personal paths with the $AGENT_HOME environment variable. This change is critical for developers deploying agents across diverse environments, eliminating configuration drift and simplifying version control.
Knowledge Collection Architecture
The collection module is modular, with a dedicated extractor for each source type. The web extractor uses HTTP sessions with retry logic and can handle JavaScript-heavy pages via a headless browser fallback. Extracted content is cleaned and saved with metadata such as URL and timestamp. The video extractor transcribes audio to text using a speech-to-engine, supporting batch processing and outputting timestamps. The article extractor parses PDF and EPUB files via libraries like PyMuPDF, extracting text and metadata Author, publication date.
All extractors normalize content into a uniform schema: {source, content, metadata, timestamp}. This data is stored in JSON format under a structured directory tree. With v0.0.2, the base directory for knowledge is $AGENT_HOME/knowledge, ensuring consistent access across environments.
Memory Management Implementation
The memory manager indexes knowledge entries using embeddings from a sentence transformer model. The vector index is persisted with FAISS for efficient similarity search. When a query is received, it is embedded and compared against the index, returning the most relevant
Top comments (0)