The pitch for multi-agent systems is intoxicating. You take a complex task, decompose it into specialized roles, hand each role to its own agent, and coordinate them through a planner. The planner delegates. The workers execute. The critic reviews. The orchestrator stitches it together. It looks like a software team, except the team is a swarm of LLMs and they never go to lunch.
I bought this pitch in 2024 and built three different multi-agent systems before I admitted that two of them would have been better as a single agent with good tools. The third one genuinely needed multiple agents. It was also the only one I could keep running for more than a quarter without rewriting half of it.
The problem with multi-agent architectures is that they are simultaneously the right answer for a small set of real problems and a tempting wrong answer for a much larger set of problems that look similar but are not. Every conference talk and Twitter thread that hypes the pattern makes the wrong answer look just as valid as the right one, because the surface case is identical.
This post is the framework I now use to decide. It comes from rebuilding the same product twice, once as a five-agent system and once as a single agent with five tools, and learning that the second version was strictly better.
What People Actually Mean by Multi-Agent
The term gets used loosely. Before any of the trade-offs make sense, we need to separate the patterns that actually exist in production.
Sequential pipeline. Agent A produces output, agent B reads that output, agent C reads B's output, and so on. There is no real coordination, just a chain. This is multi-agent in name only. It is a workflow with LLM steps, and it should be reasoned about as a workflow, not as agents.
Specialist crew. A planner agent decides what needs to be done and dispatches sub-tasks to specialist agents (a researcher, a writer, a reviewer). The specialists report back. The planner integrates the results. This is what most people mean when they say multi-agent.
Debate or critic loop. Two or more agents argue or critique each other's output, and the final answer comes from the consensus or the surviving draft. This is a specific subset of crew, optimized for output quality rather than parallel work.
Swarm with shared state. Many agents operate on a shared workspace simultaneously, picking up tasks from a queue and updating shared memory. This looks like the multi-agent ideal but in practice it is rare in production because the coordination overhead is brutal.
When someone says they built a multi-agent system, the right first question is which of these they actually mean. The trade-offs are completely different across the four.
The Single Agent Counterargument
Before you reach for any of the multi-agent patterns, the question to answer is whether a single agent with the right tools could do the job.
A single agent with tools is a model that can call functions, see the results, decide what to do next, and loop until it produces a final answer. It is one execution context, one set of model calls, one conversation history. The model is choosing what to do at every step.
In 2026 this single-agent pattern has gotten dramatically more capable. The tool-use capability of the frontier models is excellent. Context windows are large enough to hold a meaningful working memory. Cache support means you can keep a long system prompt cheap. Reasoning models can plan multi-step approaches inside a single call.
The result is that many tasks that looked like they needed coordination across multiple agents in 2024 are now better handled by one agent that has access to the right tools.
The question becomes: what do you actually gain by splitting into multiple agents?
The honest answer for most projects is: very little, at the cost of a lot.
What You Pay For Multi-Agent
Every coordination boundary you add introduces costs that are easy to underestimate.
Token cost is multiplicative, not additive. Each agent has its own context window, its own system prompt, and its own conversation history. When agent A tells agent B about the work it did, agent B has to read all of that, plus its own prompt, plus its own history. The total token spend across a five-agent system can easily be 5x the spend of a single agent solving the same task. Caching helps, but only when the prompts are stable, which they often are not in dynamic delegation patterns.
Latency stacks up. Every handoff between agents is a round trip to a model. Five agents in a sequential pipeline means five sequential model calls, each waiting for the previous one to finish. Where a single agent might solve the task in two or three model calls in a tool loop, the multi-agent version can take ten or fifteen. The user feels every one of those.
Debugging gets exponentially harder. When a single agent misbehaves, you read its trace and see what it did. When five agents misbehave, you have to figure out which agent introduced the error, whether the error was in its work or in the way the previous agent framed the task, whether a downstream agent compounded the mistake, and whether the planner should have caught it. I covered some of the patterns that help in AI agent observability, but no amount of tooling fully cancels the complexity tax.
Failure modes multiply. Each agent can fail independently. Each handoff can fail. The planner can mis-delegate, the worker can misunderstand, the reviewer can be too lenient or too harsh. You now have to design for retries, partial failures, deadlocks, and infinite delegation loops. None of these problems exist in a single-agent design.
Coordination prompts eat into the work. A meaningful chunk of the prompt budget in a multi-agent system is dedicated to telling each agent how to coordinate with the others. "Wait for the researcher's output before drafting." "Return your result in this format so the planner can integrate it." This is overhead that does not produce value for the user; it just keeps the system from falling apart.
When the gain is real, these costs are worth it. When the gain is imagined, they are pure burn.
When Multi-Agent Is Actually Right
There are three patterns where I have consistently found multi-agent designs to beat single-agent ones in real production work.
Pattern 1: Genuinely Parallelizable Subtasks
If the task can be cleanly decomposed into independent subtasks that have no dependency on each other, multi-agent is a real win. The classic example is research. You give a planner a question, it identifies five independent topics to investigate, dispatches them to five workers in parallel, and integrates the results.
The wins here are concrete. Latency drops because the workers run in parallel. Each worker has a focused context with only the data relevant to its piece, so quality goes up. The planner does not need to know how each worker did its job, only what each one returned.
The key word is independent. If worker B's task depends on worker A's findings, you have a sequential pipeline, not a parallel crew, and you lose the latency win.
This is the pattern that I have seen produce actual wins in production research tools, due diligence agents, and competitor analysis bots.
Pattern 2: Specialist Knowledge Boundaries
Sometimes a task spans multiple domains where the prompts and tools needed are genuinely different. A code review might need a security specialist, a performance specialist, and a style specialist. Each of them has a different system prompt, a different set of tools, and a different evaluation criterion.
You can technically pack all of this into a single mega-prompt with all the tools, but in practice the specialists do better work when each one has a focused prompt. The single-agent version starts to confuse the criteria. Should it prioritize security or style? It tries to balance both and ends up doing neither well.
The split here is justified when the specializations are distinct enough that each agent benefits from a fundamentally different prompt and toolset. If the agents are ninety percent the same prompt with five percent variation, you do not need them; you need a single agent with conditional logic in the prompt.
Pattern 3: Output Quality Through Critique
Some tasks have a quality bar that a single pass cannot reliably hit. Long-form writing, complex code, formal proofs. The first draft is rarely good enough, and the model knows this in retrospect but not in the moment of generating it.
A two-agent setup, writer and critic, produces noticeably better outputs on these tasks. The writer drafts. The critic reads with fresh eyes (a fresh context, no commitment to the draft) and points out problems. The writer revises. Sometimes you loop two or three times before committing.
This pattern is closer to a debate than a delegation, and the win is purely on output quality. The cost is real (you are doubling or tripling the model calls) but for tasks where quality matters more than latency, it is worth it.
When Multi-Agent Is The Wrong Answer
The mirror image of the patterns above is where most multi-agent projects fail. Here are the anti-patterns I have lived through.
Tasks that are sequential by nature. If the task is "do A, then do B with the result of A, then do C with the result of B," you do not have multi-agent. You have a workflow. Build it as a workflow with explicit steps. Use a durable workflow engine like the ones I compared in Temporal vs Inngest vs Vercel Workflow. The structure of the work is the structure of your code; do not hide it behind agent personalities.
Tasks where the specialization is shallow. If your "researcher" and your "writer" share 90% of the same context and the same instructions, they are not specialists. They are one agent with two prompts and twice the cost. The split is only justified when the specialists are doing fundamentally different things.
Tasks where the planner is just routing. If your planner agent is just looking at the input and dispatching to one of three workers, replace it with code. A regex, a classifier, or a simple if-statement is faster, cheaper, and more reliable than an LLM doing a routing decision. Save the LLM for things that actually need an LLM.
Tasks where coordination overhead is most of the work. I once built a five-agent customer support triage system that spent 80% of its tokens on agents telling each other what they were doing. When I rebuilt it as a single agent with the same tools, it produced better answers, ran 4x faster, and cost a quarter as much. The coordination was the bug, not the feature.
Tasks where the user expects fast feedback. Multi-agent systems are slower. If the user is waiting on the output, every handoff adds latency they feel. Single agents with streaming feel fast. Multi-agent systems feel like they are thinking too long, even when they are thinking well.
A Hybrid That Often Wins
The shape that I have ended up using most often in 2026 is not pure multi-agent and not pure single-agent. It is a single primary agent that can spawn focused sub-agents for specific subtasks, but only when the subtask is independent and parallelizable.
The primary agent has the full context of the user request. It does most of the work. When it identifies a subtask that benefits from a fresh context (large research dive, isolated code generation, parallel investigation of multiple options), it spawns a sub-agent with a narrow prompt and a defined return shape. The sub-agent does its thing, returns the result, and the primary agent integrates it.
This pattern keeps the simplicity of single-agent for the common path and only invokes the complexity of multi-agent where it actually pays off. The primary agent is in charge. The sub-agents are tools, not peers.
In code, this often looks like the primary agent has a delegate_subtask tool that takes a focused prompt, runs a separate model call with a clean context, and returns the result. The orchestration is implicit in how the primary agent uses the tool.
The reason this works is that it inverts the multi-agent default. Instead of "always coordinate, sometimes do work directly," it is "always do work directly, sometimes delegate." The default path is the cheap path, and you only pay multi-agent costs when you need them.
Memory and State Across Agents
If you do go multi-agent, the state question becomes load-bearing. Each agent has its own context. None of them automatically know what the others did. You have to design how state flows between them.
Three patterns I have seen work.
Explicit return values. Each agent returns a structured object. The next agent reads it. State is passed by value, never shared. This is simple and reliable but has limits when the state is large.
Shared scratchpad. Agents read and write to a shared memory store (a key-value store, a markdown file, a database). The orchestrator gives each agent a pointer to the relevant section. This scales better but introduces concurrency bugs that are nightmare fuel.
Message passing through the planner. Workers do not talk to each other. They only talk to the planner, which integrates everything and decides what to send to whom. This is the cleanest from a debugging perspective but the planner becomes a bottleneck.
The right choice depends on the pattern, but the meta-rule is: the simpler the state model, the easier the system is to debug. Most production multi-agent systems start with explicit return values and only move to shared scratchpads when they have to. I went deeper on this in AI agent memory and state persistence, but the short version is: pass state explicitly until you cannot.
Cost Modeling Before You Build
Before you commit to multi-agent, do the cost math. This is the step everyone skips and regrets.
For a typical multi-agent system with N agents and an average of T tokens per agent context, the per-task cost is roughly N × T tokens. For a single-agent equivalent, it is roughly T tokens (sometimes a bit more if the single agent has to keep more context).
If your task volume is high, that multiplier is your monthly bill. A five-agent system processing a million tasks a month at 5x the token cost of a single-agent equivalent is a meaningful budget difference.
The ways to fight this in a multi-agent design:
Cache aggressively. If the system prompt of each specialist is stable, prompt caching will dramatically reduce the per-call cost of the static parts. I went deep on this in prompt caching and the same techniques apply to every agent in the crew.
Use cheaper models for narrow workers. A specialist with a focused task often does not need the frontier model. Use Haiku or Gemini Flash for narrow workers and reserve the frontier model for the planner.
Trim the coordination prompts. Anything in the system prompt that is not load-bearing should go. In multi-agent designs, prompts tend to bloat with instructions about how to coordinate; audit ruthlessly.
Batch parallel subtasks. If your specialists run in parallel, batch the calls. Most providers offer batch APIs at meaningful discounts for non-real-time work.
If after all of this the multi-agent design is still 3x more expensive than the single-agent equivalent and the quality difference is marginal, you have your answer.
The Decision Framework
Here is the short version that I now run through before reaching for multi-agent.
Start with a single agent and the right tools. Try to solve the task that way. If it works, ship it.
If it does not work, ask why. Is the agent confused because the task is genuinely two different jobs (specialist split is justified)? Is the agent slow because subtasks could run in parallel (parallelizable split is justified)? Is the output quality consistently below the bar even with good prompting (critic loop is justified)?
If none of those apply but the agent is still failing, the problem is probably in the prompt, the tools, or the eval set, not in the architecture. Multi-agent will not fix a bad prompt. It will hide it under a coordination layer.
If one of the patterns does apply, prefer the hybrid: a primary agent that delegates specific subtasks, rather than a planner-and-workers crew. This keeps the orchestration tractable.
If you really do need a full crew, design state passing explicitly, pick the cheapest model that works for each role, and budget for the latency tax. Build observability before you build features. The system will surprise you, and the surprises are more expensive when you cannot see what each agent did.
The multi-agent pattern is real and useful in a small number of places. It is not the default. The default is one agent with good tools, and you should fight to keep it that way as long as you can.
The systems I have seen succeed in production are almost always smaller than they look from the outside. One agent doing real work, sometimes spawning a focused helper, with disciplined tools and good evals. That setup ships. Five-agent crews delegating to each other in baroque hierarchies usually do not.
Build the smallest thing that works. Add agents only when you have evidence they are paying for themselves.
Top comments (0)