dubleCC

Posted on Jul 28 • Originally published at heycc.cn

Multi-Agent Orchestration Patterns: What Actually Works and Where They Break

#multiagentsystems #agentorchestration #llmagents #anthropic

Originally published at heycc.cn. This is a mirrored copy — the canonical version is kept up to date at the source.

Multi-Agent Orchestration Patterns: What Actually Works and Where They Break

Every multi-agent framework — Anthropic's internal research system, OpenAI's Agents SDK, Google's Agent Development Kit (ADK), and the dozen open-source clones that followed — collapses down to a small number of coordination topologies. Vendors market them as distinct products, but structurally there are four: agents in a line (sequential/pipeline), agents fanning out and back in (parallel), agents in a tree with one boss (hierarchical orchestrator-worker), and agents arguing with each other (debate/consensus). Everything else — routing rules, tool schemas, memory backends — is plumbing around one of these four shapes.

The marketing material for each pattern describes what it's good for. It's much harder to find, in one place, the concrete mechanism by which agents actually hand off state, and the specific, named ways each topology breaks in production: which step corrupts the pipeline, why the fan-in aggregator becomes a bottleneck, why the lead agent's context window is a hazard, why debate makes correct answers disappear. This piece goes through all four using primary-source evidence from Anthropic's own multi-agent engineering writeup, OpenAI's Agents SDK documentation, Google's ADK docs, and two research papers that measured debate failure rates directly — one from 2025, one from 2026.

The four patterns, mechanically

Before the failure analysis, it's worth being precise about what "handing off state" actually means in each pattern, because this is where most tutorials get vague.

1. Sequential / pipeline

Agent A runs to completion, and its full output (or a designated slice of it) becomes part of Agent B's input context. B's output becomes C's input, and so on. There is no shared memory store by default — state exists only inside each agent's output text, which the orchestration code (or the next agent's system prompt) has to re-parse or re-inject. Google's ADK formalizes this as SequentialAgent, which "executes its sub-agents in the order they are specified in a list," passing session state between them in a fixed order. OpenAI's Agents SDK doesn't have a dedicated sequential primitive — you build it either as chained agent.as_tool() calls from a manager agent, or as a chain of handoff() calls where each handoff transfers full conversation history to the next agent by default.

The mechanism that matters: each stage sees the entire upstream history unless something explicitly trims it. OpenAI's SDK is explicit about this — handoff()'s default behavior gives the receiving agent the entire previous conversation, and the only way to change that is an input_filter function that receives a HandoffInputData object (input_history, pre_handoff_items, new_items, run_context) and returns a trimmed version.

2. Parallel / fan-out-fan-in

A splitter step (code or an agent) divides a task into independent sub-tasks, dispatches them to N agents that run concurrently, and an aggregator step collects and merges the N outputs. Google ADK's ParallelAgent primitive is exactly this: it "executes its sub-agents concurrently," initiating each one's run so "all the agents start running at (approximately) the same time." The decision that determines everything downstream is how much context each parallel worker gets. Anthropic's system gives each subagent a self-contained task description and a fresh context window with near-zero visibility into sibling agents — each one burns tens of thousands of tokens internally but returns only a condensed 1,000–2,000 token summary to whatever aggregates it. That's a deliberate trade: parallelism and isolation, in exchange for the aggregator never seeing the workers' raw reasoning, only their conclusions.

3. Hierarchical / orchestrator-worker

This is fan-out-fan-in with a persistent, stateful boss instead of a stateless splitter/aggregator pair. Anthropic's production research system is the best-documented example: a lead agent (Claude Opus 4) analyzes the query, writes a research plan, and spawns subagents (Claude Sonnet 4) that each act as "an intelligent filter" — iteratively using search tools and returning findings back up. The lead agent then decides whether to spawn more subagents, synthesize, or stop. Crucially, the lead agent doesn't just hold state in its context window — it writes its plan to external memory before proceeding, specifically because Anthropic's context windows get truncated past 200,000 tokens, and losing the plan mid-run would be catastrophic. That's the mechanism: hierarchical orchestration needs a memory write outside the conversation, not just a longer context window.

4. Debate / consensus

N agents (often instances of the same or different models) independently answer, then see each other's answers and reasoning, critique them, and revise across multiple rounds until they converge or a fixed round limit is hit, at which point a vote, judge model, or unanimity rule picks the final answer. The state handoff here is symmetric and repeated: every round, every agent's full response becomes part of every other agent's input, so context grows multiplicatively with rounds and agent count, not additively like the other three patterns.

Comparison table: what each pattern is actually for

Pattern	Best-fit workload	State handoff mechanism	Primary framework example
Sequential / pipeline	Workflows with genuine, unavoidable step order (draft → edit → fact-check → format)	Full prior output flows into next agent's context; no shared store by default	Google ADK `SequentialAgent`; OpenAI SDK chained `handoff()`
Parallel / fan-out-fan-in	Tasks that decompose into independent, non-overlapping subtasks (breadth-first research, batch classification)	Each worker gets an isolated context window; only a condensed summary returns to the aggregator	Google ADK `ParallelAgent`; Anthropic subagents
Hierarchical / orchestrator-worker	Open-ended tasks needing dynamic, adaptive decomposition where subtask count/scope isn't known upfront	Lead holds live context + writes plan to external memory; workers return condensed summaries synchronously	Anthropic's multi-agent research system (Opus 4 lead, Sonnet 4 workers)
Debate / consensus	Narrow, high-stakes single-answer questions where diverse critique might catch an error (safety review, contested classification)	Every agent's full response is re-injected into every other agent's context, every round	Council Mode / multi-agent debate research frameworks

Where sequential pipelines break

The pipeline's defining weakness is that it has no error-correction structure: whatever Agent A gets wrong is invisible to Agent B, because B has no ground truth to check A's output against — B only has A's output. This is error compounding by construction, not an edge case. If A hallucinates a fact, B's job is to build on it, and C's job is to build on B's version of it. There is no pattern-level mechanism (unlike debate) that would surface the error, because nothing in the pipeline is designed to disagree with upstream steps.

The second break point is context bloat under the OpenAI SDK's default handoff behavior: because handoff() passes the entire conversation history forward by default, a five-stage pipeline accumulates the full transcript of stages 1 through 4 by the time stage 5 runs. Without an input_filter to trim it, later stages pay for (and have to attend over) increasingly irrelevant early-stage reasoning — this is a direct instance of what Anthropic's context-engineering writeup calls "context rot": recall degrades as token count grows, independent of and before you hit the hard context-window ceiling.

The third failure is rigidity: a pipeline is a fixed topology decided at design time. If step 3 discovers it actually needs step 1 to redo its work with different parameters, there's no native "go back" — you either hand-roll a loop (ADK's LoopAgent primitive exists precisely because this is common enough to need a dedicated construct) or the pipeline silently produces a degraded final answer built on a wrong early assumption.

Where parallel fan-out/fan-in breaks

The parallel pattern's problems all trace back to one thing: workers can't see each other, by design. That isolation buys speed but costs coordination, and Anthropic's own engineering post documents exactly what goes wrong when task boundaries aren't airtight. Given the ambiguous instruction "research the semiconductor shortage," one subagent explored the 2021 automotive chip crisis while two other subagents duplicated each other investigating current supply chains — three agents burning tokens, two of them redundantly, one of them off-topic — because the lead's task description lacked an explicit objective, output format, and clear task boundary. This is not a hypothetical: it's a production failure Anthropic saw and had to fix.

Anthropic's own list of early-version failure modes for this topology is blunt: agents spawning 50 subagents for what should have been a simple query, agents "scouring the web endlessly for nonexistent sources," subagents duplicating each other's work or leaving gaps between their assigned scopes, and agents "distracting each other with excessive status updates." The fix wasn't architectural — it was procedural discipline layered on top: every subagent task description now requires an explicit objective, an output format, tool/source guidance, and explicit task boundaries, plus hard scaling rules (1 agent for simple fact-finding, 2–4 for comparisons, 10+ only for genuinely complex research with clearly divided responsibilities).

The fan-in side has its own bottleneck: the aggregator only sees each worker's condensed 1,000–2,000 token summary, never the tens of thousands of tokens of exploration behind it. That's the right trade for speed and cost, but it means the aggregator is structurally blind to how a worker reached its conclusion — if a worker's summary is wrong or misleadingly condensed, the aggregator has no way to sanity-check it against the worker's underlying reasoning, because that reasoning never crossed the isolation boundary.

Anthropic quantifies the tradeoff directly: parallel tool calling cut research time by up to 90% for complex queries versus sequential execution — but multi-agent systems use roughly 15x the tokens of a single chat interaction (vs. ~4x for a single agent doing multi-step tool use). Their stated rule of thumb: this pattern is only economically viable when the task's value justifies a 15x token multiplier, which rules out most low-value, high-volume workloads. These are Anthropic's own reported figures from their production system, not independently re-measured for this piece — treat them as a vendor's self-reported benchmark, not a neutral third-party audit.

Where hierarchical orchestrator-worker breaks

This pattern inherits every parallel-fan-out failure mode (it's fan-out-fan-in with a persistent boss) and adds three more specific to the hierarchy itself.

The lead agent is a single point of failure with a context budget problem. The lead has to hold the evolving plan, all workers' condensed summaries, and its own reasoning about what to do next — all inside one context window that's also subject to the 200K-token truncation Anthropic engineers around by writing the plan to external memory. If that write doesn't happen before truncation, the system loses its own plan mid-run. This is why "write to memory before proceeding" is a step Anthropic's design cannot skip, not a nice-to-have.

Anthropic made coordination synchronous specifically to bound error propagation, and that's a real bottleneck, not a free choice. Their own writeup states it plainly: "Lead agents execute subagents synchronously, waiting for each set of subagents to complete before proceeding... This simplifies coordination, but creates bottlenecks... Asynchronous execution would enable additional parallelism, but... adds challenges in result coordination, state consistency, and error propagation." In other words, they evaluated async orchestration, judged the error-propagation risk too high, and traded speed for containment. Every hierarchical system copying this pattern inherits the same choice: either accept the synchronous wait, or accept a harder, still largely unsolved async error-propagation problem.

The pattern is explicitly the wrong tool for tightly coupled work. Anthropic states this as a limitation, not a caveat: multi-agent systems are a poor fit for domains that "require all agents to share the same context or involve many dependencies between agents," and they name most coding tasks as the paradigm example — because coding has fewer genuinely parallelizable subtasks than open-ended research, and current LLM agents are "not yet great at coordinating and delegating to other agents in real time."

That claim is worth pressure-testing rather than repeating as settled fact, because it's contradicted at the margins by systems that do split coding work hierarchically and report it working. Anthropic's own Claude Code supports subagents for delegated, scoped tasks (e.g., a dedicated code-reviewer or test-runner subagent invoked by a primary session) specifically for tasks that are separable — linting, test generation, isolated refactors in files with no cross-dependencies. Several published multi-agent coding frameworks (e.g., MetaGPT's role-based pipeline of product manager, architect, engineer, QA agents) report functional output on bounded, well-specified tasks, not universal failure. The honest reconciliation isn't "Anthropic is wrong" — it's that "most coding tasks" is doing a lot of work in their claim: monolithic, tightly-coupled refactors across a shared codebase are the case they're describing, and that case is real (shared mutable state, one agent's edit invalidating another's assumptions, no clean interface boundary). Coding tasks that are already decomposable into independent files or layers behave more like the research case Anthropic's own architecture is built for. The failure mode isn't "coding" as a category, it's "tasks with implicit cross-cutting dependencies that aren't visible in the task description" — which also explains the semiconductor-shortage failure above. Same root cause, different domain.

The reliability engineering required to make this pattern production-safe is itself evidence of how failure-prone it is: Anthropic layers on retry logic, regular checkpoints so a failed run can resume instead of restarting from scratch, full production tracing to diagnose why an agent failed (state alone isn't enough to debug it after the fact), and "rainbow deployments" that gradually shift traffic between agent versions so agents already mid-execution aren't disrupted by a mid-flight code change. None of that is optional polish — it's the minimum needed to run this pattern at production reliability.

Where debate/consensus breaks

Debate is marketed as multi-agent's error-correction mechanism — the idea that independent agents catching each other's mistakes should beat a single agent's blind spots. Two research papers, thirteen months apart, both tested this directly and both found reasons to be skeptical — but they measured different mechanisms, and it matters which claim comes from which paper.

"Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate" (Andrea Wynn, Harsh Satija, Gillian Hadfield; arXiv 2509.05396; accepted to the ICML 2025 Multi-Agent Systems Workshop) is the earlier of the two. Its core finding is that debate can make a model worse over the course of the conversation, even in mixed-capability settings where stronger models outnumber weaker ones — debate rounds cause models to "frequently shift from correct to incorrect answers in response to peer reasoning," which the authors trace to agents prioritizing agreement with peers over challenging flawed reasoning. The paper investigates sycophancy and social conformity as candidate explanations for this reversal, testing whether making agents less agreeable fixes the problem — and reports that sycophancy alone is an insufficient explanation, meaning something structural about the debate format itself, not just model personality, is driving the accuracy loss.

"The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate" (Blaž Bertalanič and Carolina Fortuna; arXiv 2605.00914; submitted April 2026 — a 2026 paper, not 2025) is the one that names and quantifies three specific failure pathways in homogeneous multi-agent debate:

Sycophantic conformity — agents uncritically adopt the majority peer answer even when their own independent reasoning was correct, with modal adoption reported up to 85.5%.
Contextual fragility — injecting peer rationales into an agent's context destabilizes reasoning that was previously stable and correct, with a vulnerability rate reported up to 70.0%.
Consensus collapse — the correct answer is generated by some agent at some point in the debate, but plurality voting at the consensus step discards it anyway, with an oracle gap (correct-answer-generated vs. correct-answer-selected) reported up to 32.3 percentage points.

The same paper puts a cost on the exercise: unguided multi-agent debate consumes 2.1x–3.4x more tokens than an isolated agent doing self-correction — reported as running up to roughly 28,600 tokens per problem in debate configurations — while landing on accuracy that is merely comparable to, or worse than, the isolated self-correcting baseline. The paper's account of what beats debate on a cost basis is a single agent given a substantially larger output budget instead of peers to argue with; treat the "10x" framing of that budget increase as this article's paraphrase of the paper's cost-comparison argument rather than a number independently re-verified against the paper's own tables for this piece.

Getting the attribution right matters beyond pedantry: if you go looking for "85.5% sycophantic conformity" inside the Wynn/Satija/Hadfield paper, you will not find it — that paper's contribution is the correctness-reversal mechanism and the finding that sycophancy alone doesn't explain it, not these three named pathways or these percentages. The percentages belong to Bertalanič and Fortuna, published roughly seven months later, which independently arrived at a harder, more specific taxonomy of why debate degrades, building in the same direction as the earlier paper's qualitative finding.

The mechanism behind the "tyranny of the majority" effect these papers describe is structural, not incidental: because every agent's full response gets re-injected into every other agent's context every round, a wrong answer held by the majority is reinforced each round simply by virtue of appearing more often in every minority agent's context — nothing in the base pattern distinguishes "this answer is repeated because it's correct" from "this answer is repeated because 2 of 3 agents happen to hold it." Debate has no built-in truth signal; it only has an agreement signal, and those are not the same thing.

Debate/consensus is also the pattern most exposed to deadlock/non-termination risk, because unlike the other three patterns it has no natural stopping condition — a pipeline stops when the last stage finishes, fan-out/fan-in stops when the last worker returns, but debate rounds continue until agents converge or you impose an artificial round cap. Frameworks that add a Wald-SPRT-style statistical stopping rule (sequential hypothesis testing to decide "stop debating now, confidence is high enough") exist precisely because naive round-capped debate either stops too early (locking in a wrong majority) or burns rounds indefinitely chasing consensus that measurement shows won't arrive cheaply.

Picking a topology for your problem

Use this to pick a pattern by the actual shape of the problem, not by which pattern is trendiest — or answer five questions in the Multi-Agent Orchestration Pattern Picker to get the same table's row matched to your specific task:

If your task has this shape...	Use...	...and specifically avoid
Fixed, unavoidable step order; each step needs the previous step's full output	Sequential pipeline	Letting untrimmed history balloon into later stages — add explicit context trimming per stage
Decomposes cleanly into independent, non-overlapping subtasks known in advance	Parallel fan-out/fan-in	Vague task boundaries — write explicit objective + output format + scope per worker, or you'll get duplicated/gapped work
Open-ended, subtask count/scope not knowable upfront, needs adaptive planning	Hierarchical orchestrator-worker	Using it for tightly-coupled work with implicit cross-cutting dependencies — the failure mode is hidden coupling, not "coding" as a category
Narrow, single-answer question where you want independent critique to catch an error	Debate/consensus, WITH a stopping rule and anti-conformity guard	Unguided homogeneous debate — it costs 2.1x-3.4x more tokens and doesn't reliably beat isolated self-correction
Simple fact lookup or single well-defined transformation	None of the above — single agent	Reaching for multi-agent by default; a 15x token multiplier needs to be earned by task value

Firsthand artifact: the arithmetic behind the cost asymmetry, and what it does and doesn't prove

Two things are true at once here, and conflating them is the single easiest way this class of article turns into unverified pattern-matching: the token multipliers cited above (4x for a single agent doing tool use, ~15x for multi-agent orchestration, 2.1x–3.4x for unguided debate) are reported figures from the cited primary sources, not something this piece re-ran or independently measured. What follows is not a substitute for that — it's a derivation of the shape of the cost curve, worked through with concrete numbers, so the "why" behind those reported multipliers is checkable arithmetic rather than an assertion to take on faith.

For a debate with N agents and R rounds, where every agent's response averages T tokens and gets fully re-injected into every other agent's context each round, the cumulative cross-agent re-injection volume scales as:

total_reinjected_tokens = N * R * (N - 1) * T

Three worked examples, holding T = 500 tokens constant, to show how the curve actually moves — this is the part a reader can check with a calculator, not take on faith:

N (agents)	R (rounds)	Reinjection volume	vs. single agent's 500-token answer
2	3	2 × 3 × 1 × 500 = 3,000	6x
3	3	3 × 3 × 2 × 500 = 9,000	18x
5	3	5 × 3 × 4 × 500 = 30,000	60x
3	5	3 × 5 × 2 × 500 = 15,000	30x

The table makes the mechanical point concrete: reinjection volume is quadratic in agent count (going from N=3 to N=5 at fixed rounds is a 3.3x increase in reinjected tokens, not the 1.67x you'd get from a linear relationship) and only linear in round count (N=3 going from R=3 to R=5 is a 1.67x increase, matching the ratio of rounds). That asymmetry — quadratic in agents, linear in rounds — is the mechanical reason the papers above report debate's real-world token multiplier in the single-digit range (2.1x-3.4x for the Bertalanič/Fortuna baseline comparison) rather than the 6x-60x shown in the idealized reinjection-only table: real systems cap N low (2-3 agents is typical) specifically because the quadratic term punishes larger debates disproportionately, and none of the reported multipliers include the compounding effect of each agent's own reasoning tokens on top of the reinjection volume, which is additional and separate from what this formula counts.

This table is original arithmetic, not an empirical measurement — it demonstrates why debate's cost curve has the shape the primary sources report, it does not reproduce their exact multipliers, and it should not be cited as if it were a benchmark result.

If you want to check it against reality rather than take the shape on faith, the measurement is straightforward: run a 3-agent, 3-round debate on a fixed question, and log input plus output tokens from each round's API usage response. Expect your total to land above the formula's estimate, because the arithmetic here counts only context reinjection and ignores each agent's own reasoning tokens.

What's genuinely unresolved

None of the four patterns above is a solved problem, and the primary sources are candid about the gaps that remain open as of this writing:

Async orchestration for the hierarchical pattern is a known-worse-but-sometimes-necessary tradeoff, not a settled design choice — Anthropic explicitly flags it as future work they chose not to ship because error propagation wasn't tractable yet, not because synchronous execution is optimal.
Anti-conformity guards for debate (stopping rules, weighted voting that discounts majority-repetition, forced dissent prompts) are active research area responses to the Bertalanič/Fortuna findings, not a shipped, standardized feature of any major SDK's debate primitive as of this piece's writing — if you build debate today, you are building the guard yourself.
The "hidden coupling" failure mode identified above (tasks that look independent in the task description but share implicit state) has no general automated detector in any of the frameworks reviewed here — Anthropic's fix was procedural (better task descriptions written by humans), not architectural, which means it doesn't scale past the discipline of whoever is writing the orchestrator prompts.

What got verified before this article went live (2026-07-02)

Checked directly against primary sources for this revision:

Fetched arXiv 2509.05396 ("Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate," Wynn/Satija/Hadfield) directly — confirmed it does NOT contain the "sycophantic conformity / contextual fragility / consensus collapse" taxonomy or the 85.5% / 70.0% / 32.3pp figures; confirmed its actual reported mechanisms are correctness reversal and agreement-over-challenge, with sycophancy tested and found an insufficient standalone explanation. Confirmed ICML 2025 MAS Workshop acceptance (2025 paper).
Fetched arXiv 2605.00914 ("The Cost of Consensus," Bertalanič & Fortuna) directly — confirmed the three named failure modes and their exact percentages (85.5%, 70.0%, 32.3pp) originate in this paper, confirmed the 2.1x-3.4x token cost multiplier, and confirmed the submission date (April 2026, not 2025).
Could not independently re-verify the exact wording of the "10x output budget" comparison against the paper's full tables (PDF text extraction was not available in this environment); flagged inline in the artifact section as this piece's paraphrase rather than a re-verified quote.
Anthropic's multi-agent engineering post, Google ADK docs, and OpenAI Agents SDK docs were checked directly; the direct quote "This simplifies coordination, but creates bottlenecks" was verified word-for-word against the source.
The 90% research-time reduction and ~15x/~4x token multiplier figures remain Anthropic's own self-reported production numbers; this piece did not independently reproduce them and says so explicitly in the "Where parallel fan-out/fan-in breaks" section above.