DEV Community

Cover image for I Built a Memory Layer for LLM Agents That Knows Which Facts Go Stale
Richard Emate
Richard Emate

Posted on • Originally published at github.com

I Built a Memory Layer for LLM Agents That Knows Which Facts Go Stale

#ai #python #opensource #machinelearning

VoltMem didn't start because I kept hitting bugs in production agents.

It started with a conversation about how memory actually works — why some beliefs stick for decades while others evaporate in hours, and what triggers the audit when an old calibration stops matching present reality. That led to continual-learning research on the stability–plasticity tradeoff, and then to a structural parallel in agent memory: most layers treat every fact the same at write and search time.


The Berlin → Paris Problem
Most memory systems treat "I live in Berlin" (volatile) with the same protection as "I prefer concise answers" (stable). VoltMem differentiates them by domain.

Your AI assistant knows you live in Berlin. You moved to Paris three months ago. It still thinks you live in Berlin. Meanwhile, the fact that you prefer concise answers — stable for years — gets the same grip as "currently working on a database migration," which you finished last week.

Everything is stored the same way. Everything decays (or doesn't) at the same rate. There's no concept of how volatile a piece of knowledge actually is.

Mem0 remembers relevant facts. VoltMem remembers current truth.

Table of Contents

The core insight

Think about how different types of facts actually behave over time:

  • Your personality traits — barely change over decades
  • Core preferences (communication style, aesthetic sensibilities) — stable for years
  • Your job — changes every few years
  • What you're currently working on — changes weekly
  • Your mood right now — changes hourly

An LLM memory system that treats all of these with the same protection strength makes systematic errors in a predictable direction: it holds volatile facts too long (stale knowledge), or overwrites stable facts on thin evidence (corrupted knowledge). You can't fix both with one dial.

What you need is domain-aware protection — and, at search time, down-ranking of stale volatile memories even when they're semantically close.

The math (stay with me, it's not that bad)

Protection weight (per domain):

wd=1Vdγ w_d = \frac{1}{V_d^{\gamma}}

Write / audit decision — escalate (audit + update) when Et>θtE_t > \theta_t :

Et=[Mt⋅RtCα]⋅Vd⋅Gt θt=θ0Vd⋅Lt \begin{aligned} E_t &= \left[\frac{M_t \cdot R_t}{C^{\alpha}}\right] \cdot V_d \cdot G_t \ \theta_t &= \frac{\theta_0}{V_d} \cdot L_t \end{aligned}

Where:

  • MtM_t — how strongly the new observation contradicts what's stored (0 to 1)
  • RtR_t — how reliable the source is (explicit user statement vs. weak inference)
  • CC — how many times this memory has been confirmed (repetition count)
  • α\alpha — entrenchment exponent (how hard history fights back)
  • VdV_d — domain volatility prior (how fast does this type of fact change?)
  • GtG_t — goal-attainment delta (is updating this memory actually useful?)
  • LtL_t — cognitive load

High-volatility domain → low threshold → easy to update.
Low-volatility domain → high threshold → hard to update.

Retrieval — down-rank stale volatile memories:

score=similarity⋅(1−Vd⋅staleness) \text{score} = \text{similarity} \cdot \left(1 - V_d \cdot \text{staleness}\right)

The practical effect: a confident blip like "the user seemed extroverted today" won't overwrite a deeply confirmed "user is introverted" — but "user moved to Paris" will cleanly supersede "user lives in Berlin" because location is a volatile domain where a single explicit statement clears the update bar easily.

On Split-MNIST, this isn't a free-lunch accuracy booster. It's a validated control knob: run the same pipeline with volatility priors shuffled or inverted, and the ordering breaks (REAL > SHUFFLE > SWAP). Pre-arXiv draft: volatility_ewc_portfolio.pdf. Full reproduction: docs/RESEARCH.md.

What it looks like in practice

I compared VoltMem against Mem0 (open-source LLM memory) on three concrete scenarios — a case study, not a leaderboard claim:

Scenario 1: Location update
User says they moved from Berlin to Paris.

Mem0 VoltMem
Result Stale "Berlin" stored, 2 conflicting facts Updated to "Paris", 1 clean fact

Scenario 2: Stable preference blip
User says they "really like short replies" in one session, contradicting an established preference for thorough explanations.

Mem0 VoltMem
Result Adopts the blip Keeps original (resists weak contradicting evidence)

Scenario 3: Volatile mood
User's mood shifts from "great" to "stressed".

Mem0 VoltMem
Result Stale "great" persists Updates to "stressed"

VoltMem: 3/3 current top answer on these scripted scenarios. Challenge the scripts:

python experiments/mem0_side_by_side.py
Enter fullscreen mode Exit fullscreen mode

Retrieval haystack (same chunks, different ranker): cosine-only returns the stale fact first 20% of the time; VoltMem 0% stale@1.

python experiments/retrieval_haystack_bench.py
Enter fullscreen mode Exit fullscreen mode

LongMemEval-S (n=60): 70% answer@5 — ties cosine, does not beat it. If your only metric is public benchmark SOTA, this isn't the pitch. The pitch is update policy + retrieval freshness on mixed-volatility personal memory.

python experiments/longmemeval.py --split s --per-type 10
Enter fullscreen mode Exit fullscreen mode

Using it 

pip install voltmem[embeddings]
Enter fullscreen mode Exit fullscreen mode

Core library: zero required dependencies. Embeddings optional (sentence-transformers).

from voltmem import create_memory

mem = create_memory("app.db", user_id="alice")

mem.add("I live in Berlin")
mem.add("I prefer concise, direct answers")
mem.add("Actually I moved to Paris last month")   # updates location, not prefs

hits = mem.search("where does the user live?", limit=3)
print(hits[0]["memory"])   # "Actually I moved to Paris last month"
Enter fullscreen mode Exit fullscreen mode

Inject into any LLM system:

memories = mem.search(user_message, limit=5)
context = "\n".join(f"- {m['memory']}" for m in memories)
system = f"What you know about this user:\n{context}"
Enter fullscreen mode Exit fullscreen mode

Built-in domain volatility priors

VoltMem ships with sensible defaults you can override:

Domain Volatility Behavior
personality_trait 0.05 Strongly protected
core_preference 0.08 Strongly protected
biographical 0.10 High protection
professional_context 0.30 Medium — changes every few years
current_project 0.55 Updates readily
emotional_context 0.80 Fast-moving
current_task 0.90 Minimal protection

Custom domains:

from voltmem import create_memory, DomainRegistry

domains = DomainRegistry()
domains.register("client_relationship", 0.35)
domains.register("active_deal_stage", 0.70)

mem = create_memory("crm.db", user_id="rep_01", domains=domains)
Enter fullscreen mode Exit fullscreen mode

The priors are hand-tuned today — that's an open gap, and one of the places I'd most like real-world feedback.

LangChain integration 

pip install voltmem[langchain]
Enter fullscreen mode Exit fullscreen mode 
from voltmem.integrations.langchain import VoltMemMemory

memory = VoltMemMemory(session_id="user-42", db_path="app.db")
memory.load_memory_variables({"input": "Where do I live?"})
memory.save_context({"input": "I moved to Paris"}, {"output": "Noted."})
Enter fullscreen mode Exit fullscreen mode

Where this came from

This grew out of a philosophical conversation about how human minds handle stale beliefs — when to trust an old habit and when to question it.

The observation: animals mostly rely on impulses and simple reinforcement to build routines. Human minds add a monitoring layer on top — but that layer can go wrong when calibrated by social contexts that no longer apply. An old rule, reinforced enough times in a specific environment, can feel like an unquestionable fact even when the environment has fundamentally changed.

That maps almost exactly onto the LLM memory problem. A memory system calibrated by early conversation data can become rigid in the same way — protecting old "truths" that are now stale because they were confirmed enough times in the past.

The escalation equations above formalize the same idea: use historical reinforcement as one input, but also factor in domain volatility, source reliability, and actual mismatch — rather than letting any one factor dominate. The continual-learning experiments validated that as a causal control knob; VoltMem is the engineering artifact applied to agent context memory.

What's next

  • More benchmark scenarios — expanding beyond the 3-scenario Mem0 comparison, including cases where VoltMem loses
  • Smarter domain classification — heuristics and optional Ollama LLM work today, but priors are hand-tuned and new domains still need manual registration; next up is better defaults, cloud LLM support, and inferring domain + volatility from context
  • Async support — sync store today; most production LLM apps are async
  • arXiv preprint — theoretical foundations written up; submission in progress

Try it 

pip install voltmem[embeddings]
python examples/contradiction_demo.py
python -m examples.chat_app
Enter fullscreen mode Exit fullscreen mode

If you're building anything with persistent LLM memory, I'd genuinely like to hear how the stale-knowledge problem shows up in practice — the use cases I haven't thought of are usually the most interesting ones.

Check out VoltMem on GitHub and leave a Star!

Top comments (1)

Collapse
 
rouche01 profile image
Richard Emate •

Independent researcher here — if you're an arXiv endorser in cs.LG/cs.AI and willing to review the draft, I'd appreciate a DM. PDF + full repro in the repo.