Somewhere around the third time your agent's context window filled up with search results it no longer needed, you had the thought every agent builder eventually has: what if I split this into multiple agents? One to research, one to write, one to review. Maybe a "manager" agent on top. It sounds like an org chart, and org charts feel like architecture.
Sometimes that instinct is exactly right — Anthropic's research system, built as an orchestrator with parallel subagents, outperformed its best single-agent setup by a wide margin on internal evals. And sometimes it's exactly wrong — Cognition (the team behind Devin) published a piece bluntly titled "Don't Build Multi-Agents", arguing that for their domain, splitting the work is how you manufacture inconsistency. Both teams are right, because they're solving different problems.
This article is the decision framework between those two positions: what multi-agent architectures actually buy you, what they cost, the orchestration pattern that works in production, and the failure modes that don't make it into the launch posts.
What a multi-agent system actually is
Strip the buzz and it's simple: a multi-agent system is one where an LLM-driven agent can delegate work to other LLM-driven agents, each running in its own context window, and use their results. The common production shape is orchestrator–worker: a lead agent owns the goal, decomposes it into subtasks, spawns subagents to execute them (often in parallel), and synthesizes what comes back.
That "own context window" clause is the entire point. It's not about the anthropomorphic org chart — it's about context isolation. Everything else follows from it.
Notice what this is not: a pipeline of prompts you call in sequence from your own code is not a multi-agent system — it's just a program (and often, that's exactly what you should build instead). The multi-agent label earns its keep when the decomposition itself is dynamic — when an agent decides at runtime what to delegate, to whom, and with what instructions.
The three things fan-out actually buys you
1. Context isolation. A subagent that greps a codebase or reads twenty web pages burns thousands of tokens on intermediate noise — raw file contents, search results, dead ends. In a single-agent design, all of that lands in the one context window you have, where it dilutes attention for every subsequent step. In an orchestrator–worker design, the noise lives and dies in the subagent's window; the orchestrator receives a compressed conclusion. You've effectively multiplied your usable context by the number of workers.
2. Parallelism. Independent subtasks — "evaluate these five libraries," "check each of these twelve services for the deprecated call" — can run concurrently instead of serially. For research-shaped and audit-shaped work, this is a wall-clock improvement measured in multiples, not percent.
3. Separation of concerns. Each subagent gets a narrow prompt, a curated toolset, and one job. A focused agent with six relevant tools reliably beats a generalist juggling twenty — the same reason you curate tool sets aggressively in context engineering for agents, applied at the architecture level.
Anthropic's engineering write-up on their multi-agent research system put numbers on this: the orchestrator-with-parallel-subagents architecture outperformed their single-agent baseline by 90.2% on an internal research eval, and they found that token usage — how much relevant thinking-and-reading the system could do — explained most of the performance variance. Fan-out is, at bottom, a way to spend more tokens productively on one problem than a single context window physically allows.
What it costs you
The same write-up is refreshingly honest about the bill: in their data, a single agent used roughly 4× more tokens than a chat interaction, and multi-agent systems roughly 15×. That's the entry fee. Multi-agent architectures only make economic sense when the task's value clears it.
The subtler cost is coordination. The orchestrator communicates with subagents through task descriptions — and every ambiguity in those descriptions becomes a subagent doing confidently the wrong thing in a context you can't see. Early versions of Anthropic's system had subagents duplicating each other's work, wandering off-scope, and searching for things other subagents had already found — not because the model was weak, but because the delegation instructions were vague. The fix was unglamorous prompt engineering: each delegation needs an explicit objective, output format, tool guidance, and boundaries where the subtask ends.
The case against — and it's a strong one
Cognition's counter-argument deserves to be taken seriously, because it identifies the precise condition under which multi-agent designs fail: when subtasks are not actually independent.
Their core principles: share full context and trace history, and remember that actions carry implicit decisions. When two subagents work in parallel from partial views of the task, each makes small implicit decisions — naming, structure, interpretation of an ambiguous requirement. When their outputs meet, those decisions conflict. For coding work, where part A and part B must compile, link, and agree on conventions, parallel agents with partial context produce exactly the inconsistency you'd expect from two contractors who never spoke to each other.
So the honest synthesis of the two positions isn't "Anthropic vs. Cognition" — it's a property of the task:
- Read-heavy, decomposable, breadth-first work (research, audits, evaluation sweeps, codebase exploration) parallelizes beautifully. Subagent outputs are findings that an orchestrator can merge, and conflicts between them are cheap to resolve.
- Write-heavy, interdependent, convention-laden work (feature implementation, refactoring, anything where outputs must cohere) punishes fan-out. Here a single agent with a well-managed context — compaction, external memory, just-in-time retrieval — is the stronger architecture.
A useful mnemonic: fan out to read, stay single to write. It's not absolute (isolated write tasks in separate worktrees parallelize fine), but it's the right default, and knowing when to break it is precisely the judgment that a serious course on AI agent architecture spends real time building.
The orchestration patterns that survive production
If your task passes the test above, these are the shapes that hold up:
Orchestrator–worker (the default). One lead agent decomposes, delegates, synthesizes. Workers don't talk to each other — results flow through the orchestrator. This keeps the communication topology a star, not a mesh, which is the difference between debuggable and not. Resist the peer-to-peer "agents chatting with agents" fantasy; in production it mostly produces token-expensive misunderstandings.
Pipeline with isolated stages. Each item flows through fixed stages (find → verify → summarize), each stage its own agent call with its own clean context. No barrier between items — item A can be in stage 3 while item B is in stage 1. Deterministic control flow in your code, LLM judgment inside each stage. This hybrid — code decides the structure, agents fill the slots — is the most underrated pattern in the space, and it's where most "do we even need a framework?" questions dissolve.
Adversarial verification. Spawn N independent agents to refute a finding rather than confirm it, and keep only what survives. This is fan-out used not for speed but for confidence — independent contexts mean independent mistakes, which is exactly what majority voting needs to work. It's the cheapest defense against the plausible-but-wrong output that a single agent will happily double down on.
Three implementation rules that pay for themselves regardless of pattern:
- Delegate like you're writing a ticket for a contractor. Objective, output format, tools to use, scope boundaries. If a competent human would need to ask a clarifying question, your subagent will instead guess — silently.
- Return data, not prose. Subagents should report structured results (schema-validated if your stack allows it), because the orchestrator is a program consuming output, not a reader enjoying it.
- Match effort to value. Simple lookups get one worker with a small budget; open-ended research earns a fleet. An orchestrator that spawns ten subagents for a question one search would answer is the multi-agent version of the microservices monorepo with three users.
The failure modes nobody demos
Every one of these is a thing that happens after the demo works:
- Duplicated work. Two subagents independently discover the same fact at full token price, because the orchestrator's task descriptions didn't partition the space. Partition explicitly.
- Compounding errors. In a pipeline, a stage-one hallucination becomes stage-two's trusted input. Multi-agent systems don't just propagate errors — they launder them: by the final synthesis, the wrong fact arrives with the confidence of having been "processed" three times. This is why verification stages belong early, not only at the end.
- The telephone game. Every orchestrator-mediated hop compresses information. Three hops in, "the test fails intermittently on CI under load" has become "the tests are broken." Keep hierarchies shallow — one level of delegation covers almost every real use case.
- Lost partial work. Ten parallel subagents, one crashes at minute eight. If your harness can't resume without re-running the other nine, you'll pay the 15× token bill more than once. Checkpointing and resumability are boring, and they are the difference between a system and a demo.
And the meta-failure that enables all of the above: not measuring. A multi-agent system has more knobs than anything else you'll build with LLMs — decomposition granularity, worker count, delegation prompts, synthesis strategy — and every knob is a chance to regress silently. The teams that ship these systems treat evaluation as first-class infrastructure: a representative task set, end-to-end outcome scoring (with an LLM judge where rubrics allow), and a re-run on every architectural change. "It seemed better on the three queries I tried" is how 15× token bills get approved for 1× results.
A decision checklist
Before you split one agent into several, put the task through this:
| Question | If yes → | If no → |
|---|---|---|
| Can subtasks run with genuinely independent context? | fan-out is viable | stay single-agent |
| Is the work read/analyze-heavy rather than write/produce-heavy? | fan-out helps | single agent + context engineering |
| Does the task value clear a ~15× token cost? | proceed | simplify the architecture |
| Would a fixed pipeline in code + single agent calls do it? | build that instead | orchestrator–worker |
| Do you have evals to catch coordination regressions? | ship | build evals first |
Note how often the honest answer is the boring one. A single agent with disciplined context management — compaction, external memory, sub-tasking only when context demands it — is the right architecture for most of what gets built today, and wiring either variant into a real product with retries, budgets, and observability is its own production discipline on top of the architecture choice.
Conclusion
Multi-agent systems aren't the next tier above single agents — they're a specialized tool that trades tokens and coordination complexity for context capacity and parallelism. That trade is spectacular for breadth-first, read-heavy, decomposable work, and actively harmful for interdependent, convention-heavy production of artifacts. Fan out to read; stay single to write; let deterministic code own the control flow either way; and put an eval behind every knob.
The teams that get this right aren't the ones with the most agents. They're the ones who can articulate, for their specific task, what a second context window buys — and who kept the architecture exactly one notch more complex than that answer requires.
I build and teach this material at Cursuri-AI.ro, an AI education platform with hands-on English-language tracks on AI agent architecture, context engineering, LLM evaluation, and production integration.
Sources & further reading:
- Anthropic — How we built our multi-agent research system (orchestrator–worker pattern, 90.2% eval result, token-cost data, delegation lessons)
- Anthropic — Building effective agents (workflows vs. agents, orchestrator–workers, "simplest thing that works")
- Cognition — Don't Build Multi-Agents (context-sharing principles, the case for single-threaded agents)
This article is educational content. Architectures, model capabilities, and costs evolve quickly — validate against your own workloads and current documentation.
Top comments (1)
I appreciate how the article highlights the importance of context isolation in multi-agent systems, allowing each subagent to focus on a specific task without diluting the attention of the orchestrator. The example of Anthropic's research system, which outperformed their single-agent baseline by 90.2%, demonstrates the potential benefits of this approach. However, I'm curious about how to balance the trade-off between the benefits of fan-out and the increased complexity of managing multiple agents, particularly in terms of ensuring consistency and avoiding inconsistencies as mentioned in the Cognition article. Have you found any effective strategies for mitigating these risks in your own work with multi-agent systems?