DEV Community

Yuchen Lin
Yuchen Lin

Posted on

Designing a Drift-Resistant Memory System for LLMs

#ai #opensource #systemdesign #webdev

I recently built an open-source long-term memory engine called ProjectMemory.

While working on it, I realized that most LLM memory systems fail for the same reason: they treat memory as accumulated text instead of managed state.

This article explains that idea from a systems design perspective.

I recently built an open-source long-term memory engine called ProjectMemory.

While working on it, I kept running into the same problem:
LLM systems with “memory” work well at first, but drift over time.

They:

  • Forget earlier decisions
  • Contradict previous summaries
  • Change goals without explicit updates
  • Start giving generic next steps

The common reaction is to improve prompts, increase context windows, or add more summarization layers.

But after building and benchmarking a real system, I came to a different conclusion:

Long-term memory for LLMs is not a prompting problem.
It’s a state management problem.

The core mistake: memory as accumulated text

Most LLM memory systems look like this:

memory = previous_summary + new_events
summary = LLM(memory)
Enter fullscreen mode Exit fullscreen mode

Over time, this causes:

  • Information loss
  • Subtle goal drift
  • Conflicting summaries
  • Irreversible errors

Because the system treats memory as just text, the model is free to reinterpret or rewrite facts.

There is no notion of:

  • Protected state
  • Valid transitions
  • Consistency constraints
  • Rebuildability In other words: no state model.

*A different approach: memory as evolving state
*
In ProjectMemory, memory is modeled as a sequence of state transitions.

Each digest is not “a new summary,” but:

next_state = transition(previous_state, recent_events)
Enter fullscreen mode Exit fullscreen mode

Where:

previous_state = last digest

recent_events = new logs, notes, or decisions

transition = LLM-proposed update + deterministic checks

So the LLM does not directly control memory.
It only proposes a new state.

The system decides whether that state is allowed.

Consistency gates

To prevent drift, each proposed digest passes through consistency gates.

These enforce rules like:

  • Core goals cannot change unless explicitly updated.
  • Stable facts must not disappear.
  • Decisions cannot contradict previous accepted decisions.

If a digest violates these constraints:

  • It is rejected.
  • The previous state remains valid.
  • The system can retry or escalate.

This turns memory into something closer to:

  • A state machine
  • A versioned configuration
  • Or a database with constraints

Not just a growing text blob.

Correctness over latency

One key design decision in the project was:

A wrong memory is worse than no memory.

So the system intentionally trades:

  • Higher digest latency for
  • Stronger consistency guarantees

Benchmarks showed that enabling consistency checks:

  • Increased digest latency
  • But significantly improved state stability
  • And eliminated certain classes of drift

For long-running systems, this trade-off is worth it.

Rebuildability as a first-class property

Another key idea is that memory must be rebuildable.

Because digests are:

  • Stored as versioned states
  • Derived from event streams

The system can:

  • Recompute memory from raw events
  • Detect drift
  • Compare online vs rebuilt states
  • Repair corrupted summaries

This is much closer to:

  • Event sourcing
  • Or stateful system design

Than to prompt engineering.

Why this matters

Many people think long-term memory is mainly about:

  • Better prompts
  • Better summaries
  • Bigger context windows

But those only delay drift.

If memory is treated as unstructured text,
the system has no way to enforce consistency over time.

Reliable memory requires:

  • State models
  • Transition rules
  • Consistency checks
  • Rebuildable history

That’s a systems problem, not a prompting problem.

ProjectMemory (v0.1.0)

This article is based on building ProjectMemory, an open-source, developer-first long-term memory engine.

It provides:

  • Event-based memory ingestion
  • Layered digest pipelines
  • Consistency gates
  • Retrieval over structured memory
  • Benchmark tooling

GitHub:
https://github.com/yul761/ProjectMemory

Closing thought

If an LLM system is expected to run for weeks or months, memory is no longer just context.

It becomes state.

And once it is state, it needs:

  • Rules
  • Constraints
  • And a way to rebuild the truth.

Top comments (0)