*Part 2 of "Multi-Agent Systems in Production: What They Don't Tell You", a four-part series following the saga of Horcrux Hunt, a multi-agent Harry Potter game that taught me everything about production AI the expensive way.
Harry Had Perfect Reasoning and a 77% Failure Rate
After fixing the cost problem (Part 1), I still had a mystery: Harry kept losing.
Not because he couldn't reason but when given correct, relevant context, Harry made optimal decisions over 95% of the time. I tested this by hand-crafting perfect context and feeding it to the model. Brilliant moves every time.
But his in-game win rate was only 23%. Something between "perfect reasoning ability" and "actual gameplay" was destroying his performance.
I ran 100 games and tracked every failure. Every single one traced back to the same root cause: Harry was remembering the wrong things.
Not a reasoning problem. A memory architecture problem.
The Adversarial Memory Stress Test
Horcrux Hunt isn't just a game, it's a memory stress test designed by accident and refined by design. It exposes every weakness in how agents manage context because it has a thinking adversary actively corrupting memory.
Harry's challenge:
- Track probabilities across 15 locations over 50 turns
- Each observation provides a signal (positive, negative, or "destroyed, but it was a decoy!")
- Distinguish real Horcruxes from decoys planted to waste his moves
- Reduce uncertainty with each turn while Voldemort fights back
Voldemort's anti-memory strategy:
- Relocate Horcruxes mid-game → Harry's old signals become lies
- Plant decoys at high-probability locations → inject false confidence
- Time relocations after Harry commits → maximize wasted actions
- Increase uncertainty every turn Harry gets closer
The game is fundamentally an entropy race Harry reducing uncertainty, Voldemort increasing it.
This is the same problem your customer service agent faces (user changes intent), your code agent faces (repo state changes), or your research agent faces (new evidence contradicts old conclusions). Horcrux Hunt just makes the adversary explicit.
The Four Memory Failure Modes
Every failure I observed fell into one of four categories.
1. Stale Memory (The Relocation Trap)
What happened: Harry attacked Hogwarts with high confidence based on a positive signal from turn 12. But Voldemort relocated the Horcrux on turn 24. Harry attacked on turn 25, found nothing, wasted a precious action.
Root cause: The positive signal from turn 12 was 13 turns stale. Voldemort's relocation invalidated it. Harry's reasoning was perfect ("attack highest probability location"). His memory was lying to him.
2. Retrieval Failure (Buried Signal)
What happened: Harry re-searched a location he'd already confirmed empty. A negative signal from turn 8 existed in his context, but buried under 4,800 tokens of subsequent history.
Root cause: Relevant information existed but was functionally invisible. In 6,000 tokens of context, the model's attention couldn't find a critical 20-token signal from 40 turns ago. The signal was there. Harry couldn't find it in the noise.
3. Memory Overload (The Timeout)
What happened: By turn 40, Harry's context was 6,000 tokens. Generation took 25 seconds. Lambda timed out at 30 seconds. The game crashed. Harry lost by default.
Root cause: Too much memory. The context window became a liability, not because the information was wrong, but because carrying all of it was physically too slow.
4. Decay Problem (Lost in the Noise)
What happened: A critical positive signal from turn 3 was still valid at turn 40 (Voldemort hadn't relocated that particular Horcrux). But Harry couldn't find it among 37 turns of subsequent noise. He searched elsewhere and missed an easy win.
Root cause: Important early observations drowned out by recent but less important ones. No temporal weighting. No relevance scoring. Just raw chronological context.
The pattern: Every failure was memory architecture disguised as reasoning failure. The LLM's reasoning was fine. The information it received was wrong, stale, incomplete, or overwhelming.
The Memory Tax: Voldemort's Best Weapon
Context windows aren't a flat fee. They're a compounding tax and Voldemort exploits this ruthlessly:
| Turn | Harry's Context | Cost Multiplier | Voldemort's Advantage |
|---|---|---|---|
| 1 | ~800 tokens | 1× | None (equal information) |
| 10 | ~1,800 tokens | 2.25× | Harry carries stale signals |
| 25 | ~3,500 tokens | 4.4× | Multiple relocations buried in context |
| 50 | ~6,000 tokens | 7.5× | Harry can't find relevant signals in noise |
Every turn Voldemort corrupts (relocate, decoy) adds tokens to Harry's context that are actively misleading. Harry pays to carry lies. The longer the game goes, the more Harry pays to process increasingly corrupted memory.
This is the "Fail Fast, Fail Free" lens on memory: every token Harry doesn't need is a micro-failure, wasted money processing information that doesn't help or actively hurts his decisions.
Memory as Entropy Management
The insight that changed everything: memory architecture IS entropy management.
Shannon entropy measures uncertainty. With 15 equally-likely Horcrux locations:
H = log₂(15) = 3.9 bits (maximum uncertainty)
Harry's job: each observation should reduce entropy (concentrate probability on fewer locations). Voldemort's job: each action should inject entropy (spread probability back out).
Turn 1: H = 3.9 bits (uniform, knows nothing)
Turn 10: H ≈ 2.8 bits (narrowing, eliminated some locations)
Turn 30: H ≈ 1.4 bits (confident, 2-3 likely locations)
Turn 50: H ≈ 0.6 bits (near-certain, ready to attack)
But Voldemort fights back:
Turn 25: Harry at H = 1.8 bits (getting close!)
Turn 26: Voldemort relocates → Harry at H = 2.5 bits (setback!)
Turn 27: Voldemort plants decoy → Harry at H = 2.9 bits (further back!)
The memory question becomes: what should Harry carry in context to minimize entropy as fast as possible? The answer: not 50 turns of narrative. Just the current probability distribution.
The Three-Layer Memory Framework
Three layers, each with a specific job and dramatically different cost.
Layer 1: Working Memory (Context Window)-
What Harry sees RIGHT NOW. Most expensive real estate where every token costs money every single turn.
# What Harry's LLM actually receives (55 tokens):
AgentContext(
turn=25,
budget_remaining=3,
available_allies=["ron"],
cooldowns={"dumbledore": 2},
belief_map={"Hogwarts": 0.34, "Azkaban": 0.22, "Ministry": 0.18},
entropy=1.4,
last_signal="negative @ Godrics Hollow"
)
Rule: Only what's needed for THIS decision. Nothing historical. Nothing derivable. Nothing redundant.
Layer 2: Retrieval Memory (Computation Layer)-
Where probabilities are computed, entropy is tracked, and signals are processed. This layer does heavy work OUTSIDE the LLM.
class HorcruxBeliefMap:
def update(self, location, signal, turn):
if signal == "positive":
self.beliefs[location] *= 3.0
elif signal == "negative":
self.beliefs[location] *= 0.1
elif signal == "destroyed":
self.beliefs[location] = 0.0
# Decay old beliefs (Voldemort may have relocated)
for loc in self.beliefs:
if self.last_updated[loc] < turn - 10:
self.beliefs[loc] *= 0.7 # uncertainty grows with time
self.normalize()
Cost: 0 tokens. A few microseconds of Python. Infinitely cheaper than asking Claude to reason over 50 turns of narrative.
Layer 3: Persistent Memory (Event Store)-
Complete game record. Every signal, every action, every state change. Lives in DynamoDB. Never enters the context window directly.
# DynamoDB item- cheap, complete, permanent
{
"game_id": "horcrux_42",
"turn_log": [...every event...], # 5-10 KB
"current_state": {...}, # latest snapshot
"belief_history": [...], # probability over time
"ttl": epoch + 30_days # auto-cleanup
}
This is the "source of truth", if Layer 2 needs to recompute, it pulls from Layer 3. But Layer 3 never, ever enters the LLM's context.
Architecture:
Layer 3 stores EVERYTHING (¢)
▼ computed into
Layer 2 produces COMPRESSED STATE ($)
▼ injected as
Layer 1 receives ONLY WHAT'S NEEDED ($$$)
The Key Fix: Dynamic Context Compression
Before (naive, dump everything into context):
Turn 1: Harry searched Hogwarts → negative
Turn 2: Harry searched Diagon Alley → positive
Turn 3: Harry attacked Diagon → decoy!
Turn 4: Voldemort relocated...
Turn 5: Harry used Ron at Azkaban → positive
... (50 turns = 2,000+ tokens of narrative)
Harry's LLM reads all of this. Costs $0.015. Takes 8 seconds. And half the signals are stale (Voldemort relocated since then).
After (compressed, only current beliefs):
# 55 tokens. $0.0002. <1 second.
"Horcrux likely at: Hogwarts (34%), Azkaban (22%), Ministry (18%).
Entropy: 1.4 bits (medium confidence). Budget: 3 actions remaining.
Ron available in 2 turns. Last signal: negative @ Godric's Hollow."
97% compression: 2,000+ tokens → 55 tokens.
Why lossy works: The belief map is a sufficient statistic. If Hogwarts has p=0.34, it doesn't matter WHETHER that came from a positive signal on turn 3 or the absence of negative signals over 20 turns. The probability encodes all the decision-relevant information.
Harry makes better decisions with 55 focused tokens than with 6,000 noisy ones. Less context = less noise = less distraction = better signal-to-noise ratio.
Entropy-Gated Retrieval: Fail Fast, Fail Free for Memory
Not every Harry decision deserves the same memory investment. Entropy tells you which decisions are hard:
def harry_decide(game_state, belief_map):
entropy = calculate_entropy(belief_map)
if entropy < 1.0: # LOW uncertainty, Harry is confident
# No LLM needed. Heuristic: attack top target.
return HeuristicDecision(belief_map.top_target()) # 0 tokens, $0
elif entropy < 2.5: # MEDIUM uncertainty, some ambiguity
# Compressed belief map is enough context
context = compress_to_55_tokens(belief_map) # 55 tokens
return llm_decide(context)
else: # HIGH uncertainty, genuinely hard
# Worth the investment: richer retrieval
context = build_full_context(belief_map, recent_signals) # 200-500 tokens
return llm_decide(context)
This is "Fail Fast, Fail Free" for memory retrieval. When entropy is low, Harry already knows what to do and spending 200 tokens to confirm an obvious decision is waste. The entropy check catches that waste before it costs anything:
| Entropy | Strategy | Tokens | Cost | % of Decisions |
|---|---|---|---|---|
| H < 1.0 | Heuristic (attack top target) | 0 | $0 | ~35% |
| 1.0 ≤ H < 2.5 | Compressed belief map | 55 | $0.0002 | ~45% |
| H ≥ 2.5 | Full retrieval | 200-500 | $0.002 | ~20% |
35% of Harry's decisions cost zero tokens. The entropy gate catches "I don't need to think about this" before the meter starts running.
ε-greedy exploration: Even when entropy is low, 10% of the time Harry explores a non-top location. This prevents tunnel vision whichis critical against Voldemort, who can exploit predictable behavior.
The Results
Same Harry. Same Voldemort. Same Claude 3 Sonnet. Same 50-turn game. Radically different memory architecture:
| Metric | Naive (all-in-context) | Memory-Optimized | Change |
|---|---|---|---|
| Tokens per decision | 5,000 | 55 | -97% |
| Context at Turn 50 | 6,000 tokens | ~200 tokens | -97% |
| Cost per game | $1.95 | $0.35 | -82% |
| Latency per turn | 12s | 3s | -75% |
| Harry win rate | 23% | 52% | +29pp |
| Stale memory failures | ~20% of turns | <5% | -75% |
| Hallucination rate | ~15% | ~3% | -80% |
| Annual savings at scale | — | — | $576,000 |
The hallucination reduction is the most satisfying: with 5,000 tokens, Harry had more material to confabulate from ("I think I remember a positive signal at..."). With 55 tokens of clean probabilities, there's nothing to hallucinate about.
And Harry wins more because Voldemort's memory corruption strategy stops working. Relocating a Horcrux only hurts Harry if Harry is carrying stale signals. With a Bayesian map that decays old beliefs automatically, relocations get priced in mathematically. Voldemort can't corrupt math.
Five Memory Design Principles (Learned from Horcrux Hunt)
The LLM should consume memory, not produce it. Bayesian maps compute beliefs. The LLM receives results. Voldemort can't corrupt math the way he corrupts narrative.
Match memory depth to decision difficulty. Entropy-gate your retrieval. Easy decisions don't deserve expensive context. Fail fast, fail free.
Compress at boundaries, not inside the LLM. The compression from 2,000 tokens to 55 happens in deterministic Python and not via LLM summarization (which costs tokens and can hallucinate).
Persist abundantly, retrieve selectively, present minimally. DynamoDB stores everything. Layer 2 computes what matters. Layer 1 shows only what's needed.
Less context = better decisions. Harry performs BETTER with 55 focused tokens than 6,000 noisy ones. Signal-to-noise ratio > signal quantity. Always.
The Principle
Harry doesn't lose because he can't reason. He loses because he remembers wrong.
Fix the memory. The reasoning follows.
Every token of unnecessary context is a failure to apply "Fail Fast, Fail Free" at the memory layer. Check whether you need to think before you start thinking. Gate before you retrieve. The cheapest context is the context you never load.
💬 Quick memory test for your agent:
Run your agent through the same multi-turn task 5–10 times.
Then deliberately introduce something it was told several turns earlier.
🧠 Remembers it correctly? → Your memory architecture might be doing its job.
🔄 Remembers sometimes? → You probably have a retrieval or context-selection problem.
❌ Forgets, contradicts, or repeats itself? → Your agent may not have a reasoning problem at all. It may have a memory problem.
🤷 Never tested it? → Now you know what to test.
Drop your agent’s weirdest “I definitely told you that already” or "Never said that" moment in the comments. 👇
I’ll tell you which memory failure mode you're probably dealing with.
And if you want to follow the Horcrux Hunt production saga,
→ Read this blog to know more about the "Fail Fast, Fail Free" process: Part 0: Fail Fast, Fail Free
→ Next up: Part 3: When Harry's Tools Betray Him. We saw Memory is fixed. Cost is fixed. But when I added tools (search spells, ally abilities, divination artifacts), Harry started calling the wrong ones. 60% of tool interactions failed and not because Harry couldn't reason about WHICH tool to use, but because the tools themselves were having ambiguous descriptions.*
I am a Gen AI Developer Advocate & Architect. I built a multi-agent AI game to entertain a conference audience and accidentally created the most expensive stress test for multi-agent systems I'd ever seen. The game taught me that the hardest problem in AI isn't reasoning. So, I adapted the classic "Fail Safe" and came up with "Fail Fast, Fail Free" because it's all about remembering the right things at the right time.
Top comments (24)
The really interesting research direction you have pointed out is the selective forgetting. An agent that remembers everything indefinitely isn’t necessarily better than one that remembers selectively. When this happens, so the challenge is designing policies for what to retain, summarize, decay, retrieve, or discard while preserving the information required for long-horizon tasks. That feels like a fundamental problem for reliable agentic systems. Thanks for covering in detail.
Thanks for mentioning. Yes, the agents need to pass right context to the LLMs, not all context.
The three-layer model makes the failure modes much easier to reason about: working context, retrieval, and durable event history each need different policies. I especially like the fail-fast angle—when retrieval confidence is low, surfacing uncertainty is safer than silently filling the context with loosely related memories. Measuring retrieval quality separately from answer quality would make this design easier to tune.
Yes. That separation is one of the key ideas I was trying to establish with the three-layer model and measuring retrieval separately from answer quality. If the agent gives a bad answer, we should first be able to ask this- did it reason badly, or did we give it the wrong evidence?
Exactly—the authority model is what makes memory operationally safe. If the agent can distinguish user-confirmed facts, tool observations, and its own inferences, then conflict resolution becomes an explicit policy decision rather than a hidden ranking heuristic. I also like the idea of exposing uncertainty to the caller instead of forcing every retrieval into a single definitive answer.
Glad you liked the solution Max! Do follow for the next part to see how coordination becomes a challenge
I would want the compressed state to preserve more than the current belief and its confidence.
Suppose an accepted requirement changes. The important question is not only what the agent should now believe, but which derived specifications, tests, and implementation decisions must be reviewed.
Does your state model retain authority, provenance, and dependencies, or are those reconstructed from the event store when a decision changes?
ok so in the current design, the compressed state primarily preserves the current belief + confidence, while the durable event store retains the underlying history from which provenance and dependencies can be reconstructed. I wouldn't say the 55-token state is sufficient for dependency-aware change propagation. If an accepted requirement changes, you need more than “what do I believe now?”, you need "why it was accepted, what it depends on, and what downstream artifacts were derived from it".
I think that points to a natural next evolution of the model while keeping only the decision-relevant state in working context.
That separation makes sense: keep the compact belief in working context and preserve the causal graph outside it. The missing operation is invalidation.
When an accepted requirement changes, the system should be able to ask which beliefs, plans, tests, and artifacts were derived from it, mark them as needing review, and bring only that affected slice back into context. Otherwise the durable event store contains provenance but the agent cannot use it at the moment a decision changes.
A compact state does not need to carry the full history. It needs a stable pointer into a dependency structure that can reconstruct the relevant history on demand.
Right, I think "invalidation is the missing operation" in the model as I’ve described it.
And I really like the "stable pointer" framing. It preserves the compact-state principle while giving the agent a way to reconstruct why a belief exists when it actually matters.
That stable pointer is also where change propagation becomes testable. If a requirement changes, the system should be able to follow the pointer and identify which beliefs, specifications, tests, and implementation decisions may now be stale.
I would keep the compact state small, but make invalidation produce an explicit impact set rather than silently rebuilding context. Then you can evaluate that set for missed dependencies and false positives.
Would you invalidate downstream beliefs immediately, or mark them as review-required until the affected evidence is rechecked?
the 'perfect reasoning, wrong memory' diagnosis is the one that took us longest to internalize. we kept tuning the model and adding more context and the win rate wouldn't move.
turned out the issue was stale state poisoning the context. an earlier tool call had fetched user preferences, but by step 7 the user had updated them and the agent was acting on the cached version. we added
valid_untiltimestamps to every memory entry and an eviction pass before context assembly. dropped our 'agent did the wrong thing' error class by about 60%.does your solution handle mid run state invalidation, or does it assume the world is static between turns?
Hi. Thanks for your query. Yes, This is exactly the kind of failure mode I was trying to capture with the Horcrux Hunt game. So, my current architecture accounts for mid-run state changes where Voldemort relocates Horcruxes specifically to invalidate previously correct signals. The belief layer applies temporal decay, while the persistent event store remains the source of truth; the LLM only receives the recomputed current state. The closest section is ‘The Relocation Trap’, where I explicitly introduce mid-run state changes and stale beliefs. The belief decay/current-state computation section then describes how I prevent those stale beliefs from continuing to drive decisions. But your
valid_untilapproach highlights an important extension: validity should probably be an explicit property of memory, not just an implicit consequence of decay. That also fits the larger “Fail Fast, Fail Free” principle I’m exploring: before spending tokens reasoning over a memory, first ask whether that memory is still valid.In other words: don't just retrieve the right memory, first validate that it's still true before you reason over it. Your 60% reduction is a great example of why that distinction matters.
Anannya, the split between decay by turn count and revocation by authority matters more than it looks, and the game hid the difference because only one thing was ever corrupting the belief state. A probability alone cannot tell apart nobody touched this for ten turns from somebody explicitly overturned it, and those two call for different responses, one a soft discount and the other a hard overwrite. Practitioners commenting on posts of mine about belief tracking split on exactly this, and six independently asked for a field naming who revoked a claim or what replaced it, not a decay constant applied uniformly. Two of those six went further and said their own systems do not model authority at all: they log who closed a decision, never whether that person had standing to close it.
The other place I would push is the entropy gate. Three branches, heuristic, compressed call, full call, and none of them is allowed to say that uncertainty is high so I decline rather than guess. That is a real option in human decision making and it is missing here, which means the architecture cannot separately measure how often the agent correctly recognised its own ignorance. That number is not the same as win rate. An agent can win more while still guessing confidently in exactly the states where it should have refused, and 52 percent would not show that failure mode at all.
Actually, this is a really good push. I agree that the decay and revocation should be modelled as different events because one says "this belief is getting stale", while the other says "this belief is explicitly no longer valid". Authority/provenance would make that distinction much more actionable than a uniform decay constant. And the "decline" branch is an important omission. My entropy gate currently decides how much context/retrieval to use, but not whether the agent should say "I don't have enough evidence to act". That’s very aligned with the Fail Fast, Fail Free theme, recognizing uncertainty early should itself be treated as a successful outcome, not a failure.
The next iteration definitely needs to measure calibrated uncertainty/refusal quality separately from win rate. Great catch.
I ship a retrieval layer for coding agents, and I ran the paired experiment your Layer 1 argument predicts. It agreed with you in one place and disagreed in another.
Where it agreed: Your entropy gate decides how much to retrieve. The dominant factor in my system turned out to be what text gets handed to the retriever at all. Searching with the draft the agent is about to write surfaced the governing memo in 11 of 11 sessions. Searching with a statement of the goal surfaced it in 1 of 14. Same corpus, same retriever, same k. The draft carries the hazard's own vocabulary; a goal statement does not.
Where I was wrong, I predicted the memory tax at plus 40 to plus 120 percent input tokens and measured minus 1 percent. The hook fired a median of 3 times per session, not the 10 my design assumed. I had priced a tax I never paid.
Where it disagreed. One task family failed 6 of 6 in both arms, with its governing memo injected on every single write. Retrieval was solved and the win did not arrive. So I would add a fourth bucket beside your four failure modes: the information was correct, it was present, and the loss happened downstream of its arrival. No memory architecture reaches that one.
6 rescues against 1 regression over 34 pairs of a registered 48, McNemar p equal to 0.125. I shipped it on that, knowingly, and I record it as non-significant rather than dressing it up.
Which makes me curious about the 23 to 52 percent. Over 100 games, what is the interval on that, and did you run an A/A arm to see how far two identical configurations drift apart on their own? My noise floor was the single most useful thing I measured, and it was the thing I nearly skipped.
This is a really useful data, esp. the “correct information, present in context, failure downstream” bucket. I’d absolutely separate that from memory failures; otherwise we risk blaming retrieval for problems that actually belong to reasoning, planning, or execution.
And I agree on the A/A point. The "noise floor" is critical when the agent itself is stochastic. My 23–52% result was based on the observed game outcomes, but I didn’t run a formal A/A baseline, so I’d be cautious about interpreting that range as purely attributable to the memory architecture. That’s definitely something I’d add to the next iteration.
The RAG overload point is something I’ve encountered repeatedly. Retrieval quality isn’t simply about maximizing recall. Once you start injecting too many semantically related chunks into the context, you increase interference and make it harder for the model to identify the information that actually matters for the current decision. Context selection and compression are becoming just as important as retrieval itself. Nice one!
Yes, the classic problem. Setting threshold alone doesn't help. Dynamic compression, shared context are techniques that will help along with this
Agent memory probably needs its own observability layer. Beyond token usage and latency i would want to track things like memory retrieval precision, stale memory rate, retrieval diversity, context utilization, forget rate, and whether retrieved memories actually influenced the final action. Without those signals, memory failures are incredibly difficult to distinguish from model failures.
Absolutely agree.
Token usage and latency tell us how much the agent consumed, but not whether it retrieved the "right memory at the right time". Retrieval precision, stale-memory rate, forget rate, and memory-to-action influence would make it much easier to answer the key debugging question: "did the model fail, or did we fail the model before reasoning even started"? That’s a metric layer I’d definitely add to the next iteration of this experiment.
When two high-p locations got there different ways and one is a decoy. Your 55-token map can't tell them apart
Yes I agree, and this is a limitation of aggressive memory compression.
If two high-p candidates have different ways and one is deliberately a decoy, a 55-token state representation can preserve what the agent believes without preserving enough of why it believes it. At that point, the sufficient statistic assumption breaks down.
That’s a useful distinction for the architecture. For me-compress the state, but preserve the confidence signals needed to distinguish competing hypotheses is the goal. Otherwise, we may save tokens only to lose the information required for the next decision.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.