*Part 2 of "Multi-Agent Systems in Production: What They Don't Tell You", a four-part series following the saga of Horcrux Hunt, a multi-agent Harr...
For further actions, you may consider blocking this person and/or reporting abuse
The really interesting research direction you have pointed out is the selective forgetting. An agent that remembers everything indefinitely isn’t necessarily better than one that remembers selectively. When this happens, so the challenge is designing policies for what to retain, summarize, decay, retrieve, or discard while preserving the information required for long-horizon tasks. That feels like a fundamental problem for reliable agentic systems. Thanks for covering in detail.
Thanks for mentioning. Yes, the agents need to pass right context to the LLMs, not all context.
The three-layer model makes the failure modes much easier to reason about: working context, retrieval, and durable event history each need different policies. I especially like the fail-fast angle—when retrieval confidence is low, surfacing uncertainty is safer than silently filling the context with loosely related memories. Measuring retrieval quality separately from answer quality would make this design easier to tune.
Yes. That separation is one of the key ideas I was trying to establish with the three-layer model and measuring retrieval separately from answer quality. If the agent gives a bad answer, we should first be able to ask this- did it reason badly, or did we give it the wrong evidence?
Exactly—the authority model is what makes memory operationally safe. If the agent can distinguish user-confirmed facts, tool observations, and its own inferences, then conflict resolution becomes an explicit policy decision rather than a hidden ranking heuristic. I also like the idea of exposing uncertainty to the caller instead of forcing every retrieval into a single definitive answer.
Glad you liked the solution Max! Do follow for the next part to see how coordination becomes a challenge
I would want the compressed state to preserve more than the current belief and its confidence.
Suppose an accepted requirement changes. The important question is not only what the agent should now believe, but which derived specifications, tests, and implementation decisions must be reviewed.
Does your state model retain authority, provenance, and dependencies, or are those reconstructed from the event store when a decision changes?
ok so in the current design, the compressed state primarily preserves the current belief + confidence, while the durable event store retains the underlying history from which provenance and dependencies can be reconstructed. I wouldn't say the 55-token state is sufficient for dependency-aware change propagation. If an accepted requirement changes, you need more than “what do I believe now?”, you need "why it was accepted, what it depends on, and what downstream artifacts were derived from it".
I think that points to a natural next evolution of the model while keeping only the decision-relevant state in working context.
That separation makes sense: keep the compact belief in working context and preserve the causal graph outside it. The missing operation is invalidation.
When an accepted requirement changes, the system should be able to ask which beliefs, plans, tests, and artifacts were derived from it, mark them as needing review, and bring only that affected slice back into context. Otherwise the durable event store contains provenance but the agent cannot use it at the moment a decision changes.
A compact state does not need to carry the full history. It needs a stable pointer into a dependency structure that can reconstruct the relevant history on demand.
Right, I think "invalidation is the missing operation" in the model as I’ve described it.
And I really like the "stable pointer" framing. It preserves the compact-state principle while giving the agent a way to reconstruct why a belief exists when it actually matters.
That stable pointer is also where change propagation becomes testable. If a requirement changes, the system should be able to follow the pointer and identify which beliefs, specifications, tests, and implementation decisions may now be stale.
I would keep the compact state small, but make invalidation produce an explicit impact set rather than silently rebuilding context. Then you can evaluate that set for missed dependencies and false positives.
Would you invalidate downstream beliefs immediately, or mark them as review-required until the affected evidence is rechecked?
the 'perfect reasoning, wrong memory' diagnosis is the one that took us longest to internalize. we kept tuning the model and adding more context and the win rate wouldn't move.
turned out the issue was stale state poisoning the context. an earlier tool call had fetched user preferences, but by step 7 the user had updated them and the agent was acting on the cached version. we added
valid_untiltimestamps to every memory entry and an eviction pass before context assembly. dropped our 'agent did the wrong thing' error class by about 60%.does your solution handle mid run state invalidation, or does it assume the world is static between turns?
Hi. Thanks for your query. Yes, This is exactly the kind of failure mode I was trying to capture with the Horcrux Hunt game. So, my current architecture accounts for mid-run state changes where Voldemort relocates Horcruxes specifically to invalidate previously correct signals. The belief layer applies temporal decay, while the persistent event store remains the source of truth; the LLM only receives the recomputed current state. The closest section is ‘The Relocation Trap’, where I explicitly introduce mid-run state changes and stale beliefs. The belief decay/current-state computation section then describes how I prevent those stale beliefs from continuing to drive decisions. But your
valid_untilapproach highlights an important extension: validity should probably be an explicit property of memory, not just an implicit consequence of decay. That also fits the larger “Fail Fast, Fail Free” principle I’m exploring: before spending tokens reasoning over a memory, first ask whether that memory is still valid.In other words: don't just retrieve the right memory, first validate that it's still true before you reason over it. Your 60% reduction is a great example of why that distinction matters.
Anannya, the split between decay by turn count and revocation by authority matters more than it looks, and the game hid the difference because only one thing was ever corrupting the belief state. A probability alone cannot tell apart nobody touched this for ten turns from somebody explicitly overturned it, and those two call for different responses, one a soft discount and the other a hard overwrite. Practitioners commenting on posts of mine about belief tracking split on exactly this, and six independently asked for a field naming who revoked a claim or what replaced it, not a decay constant applied uniformly. Two of those six went further and said their own systems do not model authority at all: they log who closed a decision, never whether that person had standing to close it.
The other place I would push is the entropy gate. Three branches, heuristic, compressed call, full call, and none of them is allowed to say that uncertainty is high so I decline rather than guess. That is a real option in human decision making and it is missing here, which means the architecture cannot separately measure how often the agent correctly recognised its own ignorance. That number is not the same as win rate. An agent can win more while still guessing confidently in exactly the states where it should have refused, and 52 percent would not show that failure mode at all.
Actually, this is a really good push. I agree that the decay and revocation should be modelled as different events because one says "this belief is getting stale", while the other says "this belief is explicitly no longer valid". Authority/provenance would make that distinction much more actionable than a uniform decay constant. And the "decline" branch is an important omission. My entropy gate currently decides how much context/retrieval to use, but not whether the agent should say "I don't have enough evidence to act". That’s very aligned with the Fail Fast, Fail Free theme, recognizing uncertainty early should itself be treated as a successful outcome, not a failure.
The next iteration definitely needs to measure calibrated uncertainty/refusal quality separately from win rate. Great catch.
I ship a retrieval layer for coding agents, and I ran the paired experiment your Layer 1 argument predicts. It agreed with you in one place and disagreed in another.
Where it agreed: Your entropy gate decides how much to retrieve. The dominant factor in my system turned out to be what text gets handed to the retriever at all. Searching with the draft the agent is about to write surfaced the governing memo in 11 of 11 sessions. Searching with a statement of the goal surfaced it in 1 of 14. Same corpus, same retriever, same k. The draft carries the hazard's own vocabulary; a goal statement does not.
Where I was wrong, I predicted the memory tax at plus 40 to plus 120 percent input tokens and measured minus 1 percent. The hook fired a median of 3 times per session, not the 10 my design assumed. I had priced a tax I never paid.
Where it disagreed. One task family failed 6 of 6 in both arms, with its governing memo injected on every single write. Retrieval was solved and the win did not arrive. So I would add a fourth bucket beside your four failure modes: the information was correct, it was present, and the loss happened downstream of its arrival. No memory architecture reaches that one.
6 rescues against 1 regression over 34 pairs of a registered 48, McNemar p equal to 0.125. I shipped it on that, knowingly, and I record it as non-significant rather than dressing it up.
Which makes me curious about the 23 to 52 percent. Over 100 games, what is the interval on that, and did you run an A/A arm to see how far two identical configurations drift apart on their own? My noise floor was the single most useful thing I measured, and it was the thing I nearly skipped.
This is a really useful data, esp. the “correct information, present in context, failure downstream” bucket. I’d absolutely separate that from memory failures; otherwise we risk blaming retrieval for problems that actually belong to reasoning, planning, or execution.
And I agree on the A/A point. The "noise floor" is critical when the agent itself is stochastic. My 23–52% result was based on the observed game outcomes, but I didn’t run a formal A/A baseline, so I’d be cautious about interpreting that range as purely attributable to the memory architecture. That’s definitely something I’d add to the next iteration.
The RAG overload point is something I’ve encountered repeatedly. Retrieval quality isn’t simply about maximizing recall. Once you start injecting too many semantically related chunks into the context, you increase interference and make it harder for the model to identify the information that actually matters for the current decision. Context selection and compression are becoming just as important as retrieval itself. Nice one!
Yes, the classic problem. Setting threshold alone doesn't help. Dynamic compression, shared context are techniques that will help along with this
Agent memory probably needs its own observability layer. Beyond token usage and latency i would want to track things like memory retrieval precision, stale memory rate, retrieval diversity, context utilization, forget rate, and whether retrieved memories actually influenced the final action. Without those signals, memory failures are incredibly difficult to distinguish from model failures.
Absolutely agree.
Token usage and latency tell us how much the agent consumed, but not whether it retrieved the "right memory at the right time". Retrieval precision, stale-memory rate, forget rate, and memory-to-action influence would make it much easier to answer the key debugging question: "did the model fail, or did we fail the model before reasoning even started"? That’s a metric layer I’d definitely add to the next iteration of this experiment.
When two high-p locations got there different ways and one is a decoy. Your 55-token map can't tell them apart
Yes I agree, and this is a limitation of aggressive memory compression.
If two high-p candidates have different ways and one is deliberately a decoy, a 55-token state representation can preserve what the agent believes without preserving enough of why it believes it. At that point, the sufficient statistic assumption breaks down.
That’s a useful distinction for the architecture. For me-compress the state, but preserve the confidence signals needed to distinguish competing hypotheses is the goal. Otherwise, we may save tokens only to lose the information required for the next decision.