Knowledge-and-Memory-Management v0.0.2: Clean Release with Portable $AGENT_HOME

v0.0.2 of Knowledge-and-Memory-Management is now available. This release scrubs every hardcoded absolute path from the codebase and replaces it with the portable $AGENT_HOME environment variable. If you tried v0.0.1, you had to edit a config to set your data directory. Now you just export AGENT_HOME, run a collector, and the system handles the rest. The entire knowledge collection pipeline—web scraping, video transcript extraction, and article parsing—remains intact, but the storage layer is finally clean.

What Changed

Previously, config.toml referenced /home/user/knowledge-collection and similar absolute paths. That broke deployments, containers, and team setups. v0.0.2 eliminates those by reading $AGENT_HOME at runtime. If the variable is unset, it falls back to $HOME/.knowledge. All collectors now store output under $AGENT_HOME/collections/<type>/, and the SQLite memory database lives at $AGENT_HOME/memory.db. This single change makes migrations, backups, and multi-machine workflows trivial.

Knowledge Collection Pipeline

• Web: Uses readability-based HTML-to-Markdown extraction with fallback to raw text.
• Video: Pulls YouTube transcripts via the Data API or performs local speech recognition for downloaded files.
• Articles: Parses PDF and EPUB using a metadata-aware pipeline that retains authors, DOI, and publication date.

Each collector normalises output into a unified markdown block with a YAML frontmatter containing source URL, collected timestamp, and a content hash for deduplication. The memory index (tf-idf with optional BM25) supports both exact and fuzzy queries.

Portable by Default

The main selling point of this release is operational portability. You can now define AGENT_HOME per session, per container, or in your shell profile. The same binary works across Linux, macOS, and WSL without editing a single configuration line.

export AGENT_HOME=/mnt/nas/knowledge
knowledge-collect --type web --url https://example.com/api-docs
knowledge-query --keywords "REST endpoint" --max-results 3

The first command stores the parsed page under /mnt/nas/knowledge/collections/web/api-docs.md. The second invokes the memory index and returns the top three matches. All internal path resolution uses $AGENT_HOME, so you never see a stale absolute path.

Memory Management

v0.0.2 introduces automatic archival. By default, entries older than 90 days are moved to $AGENT_HOME/archive/, keeping the active index lean. You can override the retention period in $AGENT_HOME/memory.toml:

[retention]
article = 180
video = 30

Garbage collection runs after each knowledge-collect call if the index size exceeds a configurable threshold. This prevents unbounded growth without manual cleanup.

Migration from v0.0.1

Users upgrading from v0.0.1 should move their data to the new location:

mv ~/.knowledge-collection $AGENT_HOME
knowledge-reindex --from $AGENT_HOME

The reindex command updates the SQLite database to use relative path references. After that, all future operations are fully portable.

Why This Matters

This release makes the project suitable for CI/CD pipelines, server deployments, and shared multi-user setups. The codebase now contains zero local references (paths, usernames, or machine names). Every storage decision defers to the environment variable, so you can version control your configuration without exposing personal details. For experienced developers, this means one less barrier when integrating the toolkit into automated workflows.

Get v0.0.2, set $AGENT_HOME, and start collecting. The release notes detail additional minor fixes and the updated SQLite schema.