Why I Taught My AI Agent to Forget

I've spent about 700 hours in Claude Code over the last year, building a personal AI assistant that runs my calendar, watches my crypto positions, and generally does the boring parts of my job so I don't have to. Somewhere around hour 400 I noticed a pattern that was quietly wasting a big chunk of that time: every single session started from zero.

Not "from zero" in the sense that the agent forgot my name. It has a project CLAUDE.md, it can grep old transcripts, it's not stupid. What it forgot was procedure — the stuff you only learn by doing something wrong once. It would confidently re-propose an approach we'd already tried and abandoned three weeks earlier, for exactly the reason it was about to run into again. It would re-discover the same gotcha in the same deploy script.

The fix people reach for is "give the agent memory." I tried the existing options first. They didn't fit, for a reason that turned out to be more interesting than the tools themselves: they treat every memory as equally worth keeping. I ended up building something different — skillmem, a local, self-improving skill memory for Claude Code (and any MCP client) — and the thing that makes it work isn't that it remembers. It's that it's allowed to forget.

Facts vs. procedures

Most "AI memory" tools are fact stores. You tell them "the user's favorite database is Postgres" and they retrieve that fact later. That's useful, but it's not what was costing me time. What was costing me time was procedural knowledge: when you hit X, do Y, because Z burned us last time. Trigger, steps, outcome, lessons.

skillmem stores exactly that shape. A skill looks like this:

skillmem learn skill-deploy-drain \
  -t "Zero-downtime deploy needs a drain, not a bare restart" \
  --trigger "deploying a change to a long-running service with in-flight requests" \
  --steps "1) stop accepting new requests 2) wait for in-flight to finish 3) restart 4) health-check" \
  --outcome success \
  --lessons "a bare systemctl restart drops in-flight responses to users"

Before the next task, the agent calls mem_recall (or a hook does it automatically) and gets the skills relevant to what it's about to do, not everything it's ever learned. That distinction — relevant, not everything — is where forgetting comes in.

Forgetting is the feature, not the bug

If you never forget anything, your memory store degrades into a haystack. Six months in, a naive "just keep appending" system has hundreds of skills, most of them one-off dead ends, half of them contradicting each other, and the signal-to-noise ratio of recall drops through the floor. I've watched this happen to a plain markdown "lessons learned" file — it becomes something nobody reads, including the agent.

So skillmem models skill strength the way spaced-repetition systems model memory strength, on an Ebbinghaus-style decay curve:

• A skill starts with some strength when it's learned.
• Every time recall actually helps — the agent used it and it worked — strength gets +0.15.
• On a schedule, unused skills decay by a factor of 0.85 per idle period.
• Strength has a floor of 0.05 — it never hits zero, it just fades toward irrelevance.
• Lifecycle follows the decay: active → stale after 30 days untouched → archived after 90 days at the floor.

Archived doesn't mean deleted. Every archived skill is snapshotted to a JSONL backup first, and skillmem restore brings it back. The design goal was "nothing is destroyed, but recall only sees what's currently earning its keep." That's a meaningfully different guarantee than either "keep everything forever" or "prune and lose it."

skillmem skills                  # list skills with strength bars
skillmem decay --days 14         # manual decay + lifecycle sweep

The write path has to be free, or it won't happen

Here's the part I think is actually the interesting engineering decision, and it's not the decay curve — it's the cost of writing.

If recording a skill costs an LLM call, you will not record most skills. You'll record the dramatic ones, the ones worth the token spend, and skip the small ones — which, empirically, are most of the useful ones. This is the actual difference between skillmem and tools like claude-mem (which spawns a second Claude session to process and write each memory) or mem0 (which pipes writes through a model to extract and structure them). Both are reasonable designs. Both mean every write has latency and a token cost, which means the agent's incentive is to write rarely.

skillmem's write path is a SQLite INSERT plus Snowball stemming for the search index. No LLM in the loop. It's milliseconds, and it's free. That changes the calculus: the agent can afford to learn from every non-trivial task, not just the memorable ones, because the marginal cost of being wrong about "was this worth remembering" is close to zero. Worst case, an unhelpful skill just decays away on its own in a month.

Retrieval: hybrid, local, and it doesn't care what language you ask in

Recall needed to solve a specific annoyance for me: I switch between English and Russian mid-session depending on who I'm talking to about the agent's work, and I wanted a skill written in one language to surface for a query in the other, without paying for embeddings from a hosted API on every keystroke.

skillmem's retrieval is a hybrid of two fully local signals, fused with Reciprocal Rank Fusion (K=60):

1. Lexical — SQLite FTS5 with BM25 ranking, using separate Snowball stemmers for English and Russian, so "deploying" and "deploy" (or their Russian equivalents) match.
2. Semantic — ONNX-exported paraphrase-multilingual-MiniLM-L12-v2, 384-dim embeddings, running on CPU, fully offline. This is what makes a Russian query find an English-language skill and vice versa — the embedding space is shared across languages even though the lexical index is per-language.

skillmem recall "deploy the bot to prod"
skillmem search "hash chain" --kind feedback

On why this isn't backed by a vector database: at personal-agent memory scale — thousands, not millions, of rows — a brute-force numpy matmul over the whole embedding column is sub-millisecond. Adding a vector index at that scale adds a dependency, a service to keep running, and a new way for queries to fail, in exchange for speed you don't need. The "boring" implementation is the correct one here; I'd revisit it if this were storing memories for a fleet of agents rather than one person's assistant, but that's not the problem it's solving.

Does it actually retrieve well?

I didn't want to ship "trust me" numbers, so skillmem's retrieval is benchmarked against LongMemEval (Wu et al., ICLR 2025), on the full oracle set, n=479:

Question type	n	hit@5	MRR
Overall	479	0.871	0.622
single-session-assistant	56	0.982	0.746
knowledge-update	72	0.944	0.676
single-session-user	64	0.938	0.719
multi-session	125	0.848	0.568
single-session-preference	30	0.833	0.465
temporal-reasoning	132	0.780	0.579

The weakest category is temporal reasoning — questions that hinge on when something was true, not just whether it was said. That's a known, open gap; it's tracked as issue #2 and I'd genuinely like help on it (more below).

The number I actually care about more than the headline hit-rate is that this pipeline has zero LLM calls in the retrieval loop, which means it's deterministic — run the benchmark twice, get the same numbers — and fast: median 0.76 seconds per query on a laptop CPU. You can reproduce it yourself:

python bench/longmemeval.py --sample 0 -k 5

(see bench/README.md for the oracle file and our reporting rules — I don't want to publish a bare percentage without saying what retrieval mode and embedding model produced it, and I'd like that to be a norm other memory tools adopt too.)

Memory as an attack surface

One thing that doesn't get discussed enough with agent memory: if an agent's memory is writable by anything the agent reads — a webpage, a file, a tool's output — then memory is a prompt-injection vector. Poison a skill today, and the agent quietly follows bad instructions weeks later, long after anyone's reviewing the conversation where it happened.

skillmem appends every edit to a SHA256 hash chain. The record is Unicode-normalized to NFC before hashing specifically so the same logical edit produces the same hash whether it happened on macOS (which likes NFD) or Linux (which defaults to NFC) — a detail that actually bit me in testing before I added it.

skillmem verify --strict         # walks the whole chain, fails loud on any break

This doesn't stop an injection from proposing a bad skill. It stops a bad skill from being silently rewritten after the fact without leaving a trace. That's a narrower guarantee than "immune to prompt injection," and I want to be honest about the boundary: it's tamper-evidence, not tamper-prevention.

Wiring into Claude Code

The whole point was to make this invisible in day-to-day use, not another tool I have to remember to call. skillmem init --claude-code installs:

• 6 hooks — auto-recall on every prompt, a session recap on Stop, an MCP-config guard on session start (so a broken config doesn't silently disable memory), and three more covering tool use and history.
• 8 mem_* MCP tools — mem_search, mem_get, mem_list, mem_write, mem_update, mem_learn, mem_recall, mem_reinforce.

uv venv && uv pip install -e '.[semantic]'
skillmem init --claude-code
skillmem doctor                  # health check: DB, schema, semantic status

Every hook is best-effort — a broken database or a missing embedding model never blocks Claude Code from responding, it just quietly skips the memory step.

It also works in the Claude Desktop chat app, as a plain MCP server: you get the 8 mem_* tools on demand, but the automatic hooks (auto-recall, session recap) are a Claude Code mechanism and don't run there. That's a real limitation, not an oversight — hooks need a place to attach in the host application's lifecycle, and Desktop doesn't currently expose one the same way.

Cross-platform scheduling for decay and export uses whatever the OS actually gives you — launchd on macOS, Windows Task Scheduler (schtasks) on Windows, systemd user timers with a cron fallback on Linux — set up with:

skillmem schedule install        # decay daily 04:15, export weekly Sun 04:30

130 tests run in CI across all three OSes, because "works on my Mac" is not a real cross-platform claim.

No lock-in

I did not want to build something where your accumulated skill history is trapped in a proprietary SQLite schema. export-all dumps every memory to plain markdown with YAML frontmatter:

skillmem export-all ./vault
skillmem import-vault ~/Obsidian/Notes

The round-trip is exact — export, then re-import, and you get the same records back. If skillmem stops being the right tool for you, or you want to inspect your skills in Obsidian, your data isn't hostage to the tool.

Limitations, honestly

• Temporal reasoning is the weak point. 0.780 hit@5 against >0.83 everywhere else. Questions like "what did I decide before I changed my mind about X" are harder for a retrieval system that isn't explicitly modeling time as a first-class axis. This is open — see issue #2.
• It's single-user, single-machine by design. The brute-force cosine search and SQLite backend are the right call at the scale of one person's assistant. They are the wrong call if you're trying to share a memory store across a team or a fleet of agents — don't reach for this expecting that.
• Tamper-evidence, not tamper-prevention. The hash chain tells you that something was altered after the fact; it doesn't stop a bad skill from being written in the first place if something upstream is compromised.
• Hooks are a Claude Code feature. In any other MCP host, you get the tools but you're calling mem_recall and mem_learn yourself instead of getting them for free on every turn.
• English/Russian only, for now, for the stemming side of hybrid search — the semantic model is multilingual, but I've only tuned and tested the lexical half for the languages I actually use.

Where it stands

skillmem is Apache-2.0, pip install skillmem, and the repo is at github.com/liza-studio/skillmem. It came directly out of a real annoyance rather than a plan to build a product — I wanted my own agent to stop wasting my time re-learning things, and building it that way (an actual daily-use dependency, not a demo) is why the write path had to be free and the retrieval had to work in two languages: those weren't feature-planning decisions, they were requirements from the thing I was actually using it for every day.

If you want to help, the most useful thing right now is issue #2 — improving temporal-reasoning retrieval without reintroducing an LLM into the query path. Bug reports, benchmark reproductions that disagree with mine, and "this assumption about your use case is wrong" are all welcome too.