I once watched a fan-out of 40 parallel agents finish in under three minutes and produce a result so contradictory that the downstream synthesis agent hallucinated a reconciliation just to make the pieces fit. Every individual agent had done its job well. The chaos wasn’t in any single worker. It was in the space between them.
That was the day I stopped thinking of fan-out as "throughput" and started thinking of it as "concurrency control with an LLM on top."
The Efficiency That Bites Back
The fan-out pattern is simple to describe. Take a complex task, split it into independent subtasks, spawn an agent per subtask, run them concurrently, then fan-in to aggregate. Microsoft's Agent Framework documentation describes it exactly this way—fan-out and fan-in edges allow multiple branches to run simultaneously, aggregate their results, and scale workflows efficiently.
The appeal is obvious. Instead of one agent sequentially reading ten documents, ten agents read one document each. Wall-clock time approaches the slowest branch, not the sum. Resonate's durable fan-in documentation puts the benefit plainly: total wall time approaches the slowest channel, not the sum.
I built my first serious fan-out for a research pipeline: one question, six sources, six subagents, one synthesis. It was beautiful in the demo. Then production happened.
The Chaos I Didn't See Coming
The first sign was subtle. Two agents returned conflicting numbers for the same metric. Not wrong numbers—conflicting. One had read a cached page, the other a live one. Both were "correct" against their sources. The synthesis agent picked one arbitrarily and moved on.
The second sign was louder. Three agents wrote to the same shared state object during the fan-out. One wrote a validated result, another overwrote it with a stale one. The fan-in consumed the stale value. No error. No exception. Just a quietly corrupted output.
I went looking for a framework bug. What I found was a research literature that had already named my problem.
A 2026 position paper at ICML argues that many multi-agent failures are fundamentally concurrency control problems. Agents concurrently read and write shared state, and long LLM inference windows amplify the risk of stale reads, lost updates, and inconsistent outcomes. The paper maps failure modes commonly attributed to "coordination" or "communication" breakdowns directly onto classical concurrency anomalies.
That hit hard. I had been thinking about my fan-out as a workflow problem. It was a database problem.
The Anomalies That Kill Fan-Out
The arXiv paper "Verified Detection and Prevention of Concurrency Anomalies in Multi-Agent LLM Systems" formalized four specific anomalies that every fan-out engineer should know by heart:
- Stale-generation: An agent reads shared state, begins a long generation, and commits a result based on a state that has since changed. The paper's travel-booking example: a flight agent reads "trip date = June 14," drafts a reservation for several seconds, and during that window the user updates the date to June 21. The agent commits the original date. No fault occurred at any layer. Yet the system produced an external effect no current state justifies.
- Phantom-tool: Two agents discover and register the same tool concurrently. One overwrites the other's registration.
- Causal-cascade: Agent A reads, Agent B writes a correction, Agent A proceeds on the stale read. In message-based multi-agent systems, this maps directly to the write-after-read pattern.
- Tool-effect reordering: Concurrent agents invoke tools in an order that produces effects inconsistent with the intended sequence.
The paper also found a silent lost update in ByteDance's deer-flow and tool-effect reordering in LangGraph's ToolNode on unmodified output. These aren't exotic. They're in the tools we're using today.
The Semantic Problem Nobody Warns You About
Structural concurrency anomalies are the first layer. The second layer is worse: semantic conflicts.
Two agents can both be structurally correct and semantically incompatible. One returns "revenue grew 12% year over year." Another returns "revenue declined 3% quarter-over-quarter." Both are true. Both are correct extractions from valid sources. The synthesis agent has no protocol for deciding which claim should currently be trusted.
A 2026 paper on Semantic Consensus identifies this as Semantic Intent Divergence—cooperating LLM agents develop inconsistent interpretations of shared objectives due to siloed context. The paper's framework achieved 100% workflow completion by detecting and resolving these conflicts before actions were committed, compared to 25.1% for the next-best baseline.
I didn't have that framework. I had a synthesis prompt that said "reconcile any conflicts" and hoped for the best.
What Actually Works
I rebuilt the fan-out around three structural changes that map directly onto the research.
Shared state needs isolation levels, not hope. The ICML position paper's argument is that concurrency control is the missing discipline. Snapshot isolation adds roughly 8% tokens on one workload; pessimistic locking costs 1.6-2.3×, not the order-of-magnitude penalty commonly assumed. I moved to a model where each fan-out branch writes to its own scoped state, and the fan-in merges explicitly. No branch reads another branch's in-flight writes.
Fan-in needs adversarial verification, not passive aggregation. A July 2026 paper on verify-gated fan-out found that the empirical Pareto frontier is dominated by the cheapest tier at minimum breadth—and that production harnesses increasingly close every fan-out with adversarial verification. The fan-in isn't a summarizer. It's a gate. If two branches disagree, the disagreement is surfaced, not smoothed over.
Durable promises, not best-effort parallelism. Resonate's fan-out implementation makes every spawn and every await a durable promise. If one branch fails and retries, the others stay checkpointed and don't re-execute. Crash recovery only re-runs whatever was in flight when the worker died. I had been using Promise.all, which collapses latency but gives up checkpointing: a crash mid-batch re-runs everything, and a retry blasts the same side effect twice.
The paper's line is the one I keep coming back to: the fact that each is durable is what makes this safe—every other implementation has subtle re-execution bugs.
Who's Actually Shipping This
Lyft runs a parallel safety fan-out in production. Before any LLM reasoning, malicious-intent and safety-issue detectors run concurrently. The fan-out adds 90ms to each request but lifts block recall to 99.2% on red-team probes. Their meta-agent router holds conversation state and re-routes mid-chat when intent shifts, while safety checks fan out in parallel rather than blocking on a single sequential gate.
Amazon's Expansion-Contraction pattern, published at ACM CAIS 2026, uses concurrent path analysis over a domain graph. The results: 98.2% accuracy on a production supply chain, 100% on public benchmarks, outperforming single agent baselines by 14+ percentage points, with concurrent path analysis yielding up to 1.43× speedup and investigation caching reducing token usage by up to 93.9%.
lionagi, a governed multi-agent orchestration framework, exposes fan-out as a first-class CLI operation: three workers in parallel, then a synthesis pass. Its architecture keeps branches, sessions, and flows as ordinary Python objects with typed, inspectable state—no opaque blobs, no hidden prompt assembly.
When to Fan Out (and When Not To)
The Bright Data engineering team's writeup is the most honest guidance I've found. Fan-out is well suited to reading tasks. Actions that write should not be delegated to subagents in the same way. Their rule: fan out when the question has independent parts and the subagents only read. Anything that writes stays on the root agent behind an approval gate.
Three 2026 papers report limits on when multi-agent systems help. Silo-Bench found that as a system grows, the cost of coordinating agents cancels the gains from running them in parallel. A Nature Machine Intelligence paper reported that the more capable models tested gained less from collaboration. And a matched-budget study from April 2026 found that single agents matched or outperformed multi-agent systems when reasoning tokens were held constant. Fan-out only becomes competitive when a single agent stops using its context window well, or when more compute is spent.
Anthropic's workflow guidance is equally direct: move to parallel only when latency is the bottleneck and tasks are independent. Start sequential. Default to sequential. Fan out when you can measure the latency gain against the coordination cost.
The Trade-Off You're Accepting
Fan-out gives you throughput. It costs you determinism.
Every concurrent branch is a place where state can diverge. Every fan-in is a reconciliation you have to design, not hope for. The coordination cost grows with agent count, and the semantic conflict rate grows with the diversity of sources. A benchmark on financial document processing found that hierarchical architectures occupy the most favorable position on the cost-accuracy Pareto frontier, while parallel fan-out with merge sits elsewhere—better latency, different cost profile.
The teams that are winning with fan-out aren't the ones who spawn the most agents. They're the ones who treat concurrency control as a first-class design concern, isolate state per branch, verify at the fan-in, and use durable execution so a crash at branch seven doesn't cost them the whole run.
So here's my question: when your fan-out finishes and the branches disagree, does your architecture surface the conflict or does your synthesis agent quietly pick a winner and move on?
I'd love to hear where you've landed. Adversarial fan-in, durable promises, isolation levels you actually implemented or something simpler that worked because you stayed sequential a little longer than everyone told you to?
Top comments (4)
Quietly picking a winner in the synthesis step is the default trap because LLMs hate leaving a contradiction unresolved. If branch A says revenue fell and branch B says it grew, an unconstrained aggregator prompt will invent an unprompted timeline like 'revenue dipped early in the quarter before surging' just to reconcile the text.
In my loops, the only thing that stopped that was removing narrative synthesis entirely when divergence occurs. I require each fan-out worker to emit a structured claim schema with the raw source URI and timestamp. If the fan-in step detects conflicting values on the same invariant key, it refuses to generate a blended summary, halts the run, and surfaces the raw branch diff. Letting the pipeline fail loud with an unresolved conflict is far cheaper than debugging a plausible hallucination that made it downstream.
This is the comment I wish I'd read two years ago. "Revenue dipped early in the quarter before surging", I've had an aggregator invent that exact sentence, and it took me half a day to figure out that no branch had ever said it. That's the tell: the lie is smoother than the truth. Real data is jagged. Hallucinated reconciliations are narrative.
Your halt-and-diff approach is right, and the structured claim schema is the load-bearing piece. Without the source URI and timestamp on every claim, the fan-in has no way to distinguish "these two agents disagree" from "these two agents read different documents and both are correct." The invariant key is what turns a semantic argument into a set comparison.
The one thing I'd push on: halting is only cheap if the diff is legible. My first version of this failed because it dumped raw branch JSON at the operator, who couldn't tell in under a minute whether it was a real conflict or a unit mismatch. We ended up normalizing claims before comparison — same units, same time window, same entity ID — so the halt surfaces one line: "branch A says -3% QoQ, branch B says +12% YoY, invariant: revenue_growth, non-comparable." Then a human resolves it in seconds instead of minutes.
Curious how you handle the false-positive rate. Did normalizing your invariant keys cut the halts down to something your on-call could actually sustain?
The failure boundary feels like the key design detail. I would make fan out an explicit budget rather than a default pattern, with limits for concurrent branches, shared tool calls, and aggregate retries. That keeps the throughput win measurable without letting one bad branch multiply the whole run.
This is the reframe I wish I'd made two years ago. I treated fan-out as a topology decision when it's actually a resource allocation decision, and conflating the two is how you end up with 40 branches when 6 would have done the job.
The budget framing fixes something I kept getting wrong: I'd set concurrency limits in isolation, then watch a single retry storm blow through them because retries didn't count against the same pool. Aggregate retries being budgeted separately from concurrent branches is the part most people miss, a branch that fails and retries four times has effectively spent five times its allocation, and if that doesn't surface in the same accounting, your "limit of 10" is really a limit of 10 plus whatever the retries decide to do.
The one thing I'd add: make the budget observable before it's enforced. I ran a version for two weeks where the fan-out budget only logged violations instead of blocking them. The distribution of overruns told me which branches were genuinely expensive versus which were retrying on a fixable failure. I would have set the wrong numbers if I'd picked them upfront from intuition.
Did you land on per-run budgets or per-tenant? I found the second one much harder to reason about when tenants share downstream tools.