This release updates the core knowledge and memory modules to streamline agent development. v0.0.2 focuses on three areas: multi-source knowledge collection, consolidated memory management, and full portability via the $AGENT_HOME variable. For developers building context-aware applications, this version removes environment-specific configuration and improves data ingestion from web, video, and article sources.
Knowledge Collection: Unified Ingestion Pipeline
The collector now supports three source types through a single interface. Web scraping extracts structured content from HTML and RSS feeds, with automatic deduplication and metadata tagging. Video processing uses subtitle extraction (for platforms that provide timestamps or captions) and falls back to transcript summarization for longer content. Article handling includes PDFs and markdown files, with chunking tuned for semantic coherence.
Each source goes through a standard pipeline: fetch, parse, chunk, and embed. The resulting vectors are stored with source provenance, allowing downstream agents to trace reasoning. In testing, the collector processed a 10-minute video into 15 chunks with 92% recall on key concepts.
Memory Management: Short-Term to Long-Term Consolidation
Memory now maintains two tiers: a working context (short-term) and a persistent store (long-term). Short-term memory holds recent interactions—conversations, recent fetches—with automatic decay after a configurable window. Long-term memory uses a vector index built from collected knowledge. The key change in v0.0.2 is the consolidation step: when short-term entries reach a relevance threshold, they are merged into long-term memory without duplication.
The consolidation algorithm uses cosine similarity to identify overlaps and a timestamp-aware scoring function to prioritize recent data. This prevents the store from diluting with redundant information while keeping the most actionable items available.
Portability with $AGENT_HOME
All internal file paths, configuration files, and database locations now resolve against a single environment variable: $AGENT_HOME. This replaces hardcoded absolute paths that varied across development, testing, and production environments. Setting $AGENT_HOME at startup allows the entire knowledge and memory subsystem to relocate without editing configuration files.
For example, persistent memory stores are written to $AGENT_HOME/data/memory.db, and cached embeddings to $AGENT_HOME/cache/embeddings. This ensures identical behavior across local dev machines, CI runners, and containerized deployments.
Code Example: Collecting and Storing Knowledge
The following snippet demonstrates the new unified API. It fetches a web article, processes a video transcript, and stores both in the portable memory backend.
from kcollector import Collector, Memory
# Initialize collector and memory
collector_web = Collector(source='web')
collector_video = Collector(source='video')
memory = Memory(base_path='$AGENT_HOME')
# Collect from two sources
article = collector_web.fetch('https://example.com/technical-guide')
video = collector_video.fetch(
'https://video.example.com/watch?v=abc123',
format='transcript'
)
# Store with context tags
memory.store(article, tags=['web', 'guide'], source='collector')
memory.store(video, tags=['video', 'tutorial'], source='collector')
# Later retrieval
results = memory.recall('deployment strategy', top_k=5)
In this example, Collector accepts different source types and returns standardized KnowledgeDocument objects. The Memory object writes to $AGENT_HOME/data/ by default, so moving from local to a server only requires changing the environment variable.
Configuration and Migration
The release ships with a migration script that rewrites legacy absolute paths to $AGENT_HOME references. For existing databases, the script reconstructs the store with the new layout while preserving all embeddings and metadata. After migration, the only required change is exporting $AGENT_HOME in your shell or Docker compose file.
Impact on Development Flow
For experienced developers, the immediate benefit is environment independence. No more symlinks or volume mounts to keep paths consistent. The unified collector also reduces boilerplate: one import for web, video, and article sources instead of separate scrapers and parsers. Memory consolidation happens automatically, so you don’t need to write custom dedup or decay logic.
Limitations and Future Work
Video processing still relies on external subtitle or transcript availability—custom audio-to-text is not included. Long-term memory compression (via summarization of low-access chunks) is scheduled for v0.2.0. The collector also does not yet handle dynamic content (JS-rendered pages) natively.
Conclusion
v0.0.2 makes Knowledge-and-Memory-Management a drop-in component for agent frameworks. The $AGENT_HOME approach aligns with twelve-factor app principles, and the collector abstraction reduces integration time for knowledge-heavy use cases. Upgrade your agent’s persistent brain with a single environment variable change.
Top comments (0)