We've spent two years making models smarter and context windows bigger. But here's the dirty secret of production agents: most of them "remember" by overwriting a variable. That's not memory. That's a bug wearing a trench coat.
Your agent remembered the user "doesn't like X" yesterday. Today it hands them a plan built on X. You check the log and find one line: pref.x = like. Who changed it? When? On what basis? Nobody knows. Not you, not the agent.
This isn't a model failure. It's an architecture failure — and it's the single most common one in agent memory today.
The core idea in one sentence
Most agent "long-term memory" fails not because software didn't store data, but because memory was wrongly modeled as an overwritable state variable. The correct architecture is memory = an append-only event log (WORM) + a materialized view projected on demand. The write side is forever traceable and never overwrites; the read side forever projects "the current single source of truth."
"Being able to store" and "being able to store trustworthily" are two different things. A paid decision-maker buys the second. An overwriteable state store is engineering's default laziness; append-only + materialized view is the step that turns memory from "data" into "evidence."
Why this matters now
Three structural shifts are happening in production, not in theory:
- Agents went from single-turn assistants to autonomous systems that rewrite their own config. The moment the writer of memory stops being "one trusted, reviewed programmer" and becomes "a possibly-concurrent, possibly-wrong, non-deterministic model," writing stops being an operation and becomes a source of incidents.
- Tool-call protocols (MCP-style) give the agent a peripheral bus — but memory sits outside that bus. MCP solved "how does the agent call tools," not "how does context survive reliably across sessions." Tools are legs; memory is the brain.
- Evidence / audit / rollback went from a compliance checkbox to a debugging necessity. A production agent corrupts its own memory at 3 a.m. and breaks everything downstream. The next morning you don't need "it's broken now" — you need the replay: which steps, what did it write, why.
The fix: append-only log (WORM) + materialized view (MV)
Two iron rules decide everything:
-
Write side: only append, never update. Every fact / preference / conclusion lands as an immutable event carrying timestamp, actor, and rationale. Even a correction is a new
retract/supersedeevent — express invalidation by appending, never by overwriting history away. - Read side: only read from the materialized view, never do mutable aggregation on the evidence layer. The projector folds the log into a "current state snapshot" (latest-wins, or business-rule merge). The snapshot can be cached, overwritten, even lost and rebuilt from the log — the view is expendable; the log is not.
Your 30-minute architecture health-check
-
Is your memory layer a "state store" or "log + view"? Open the schema; any
UPDATE/UPSERT/overwriteon a memory entity ⇒ you carry the three risks above. AllAPPEND/INSERT+ reads via projection ⇒ you pass. -
Turn correction into an appended
retractevent. Addsupersede {target_event_id, new_value, actor, rationale}. Test: any row must answer which supersede/retract still reference it. -
Force three required metadata on every write:
ts,actor,rationale. Test: no new write point may skip these three. -
Make the read-side "current state" a swappable projection:
current_snapshot := project(event_log, merge_strategy). Start withlatest_wins. - Split evidence vs view consistency: log = append-only, permanent, strongly consistent; view = rebuildable, disposable, eventually consistent.
- Add a guardrail on autonomous edits: before an agent appends, require "rationale must have a source". Else route to a "pending adjudication" queue.
Honest objections
- "Human memory is overwrite/forgettable; why not copy that?" Human forgetting is a passive failure to separate evidence from noise — not a feature. Copy the auditable replay side of human memory.
- "Append-only will blow up storage." View caching + periodic archival absorbs it. Events are mostly light JSON; trust beats storage cost.
- "We're single-session, state store is fine." Today single-session, tomorrow multi-session multi-agent. Pay the migration cost early.
If you're building an agent memory layer, run the six-point check above today — 30 minutes tells you whether your memory layer can ship to production.
I'm writing a "deep AI systems" series, one piece a week. Next: Agent task-queue state-machine design — why pending/running/done isn't enough.
If this was useful, an emoji react helps more people stuck in the same hole see it. Disagreements welcome in the comments — I'll reply to each.
Top comments (0)