I spent three weeks debugging a multi-agent system that worked perfectly in every demo and failed unpredictably in production. The culprit wasn't the models. It wasn't the prompts. It was the architecture we'd been told to build: a central orchestrator that routed every task, held every decision, and as we eventually discovered—quietly became the reason our reliability numbers kept sliding.
The industry has spent the last two years standardizing on a pattern that feels intuitive and scales terribly.
The Centralized Trap
The supervisor pattern is the default for a reason. It's easy to reason about, easy to debug, and maps cleanly onto every framework you've read about. One agent sits at the top, reads the goal, decides which specialist to call, and routes the work. LangGraph's create_supervisor, AutoGen's GroupChatManager, and CrewAI's hierarchical process all implement some version of this.
The architecture looks like this:
Goal → Supervisor → [Agent A | Agent B | Agent C] → Supervisor → Result
Every task passes through the supervisor. Every decision lives inside an LLM call. Every state mutation depends on the supervisor's context window.
It works for three to seven agents. The 2026 literature is blunt about this: centralized orchestration has limited scalability (3–7 agents), low fault tolerance (single point of failure), and high debugging ease—the last being the reason teams keep choosing it. You can step through each decision because there's only one decision-maker.
And then you go to production.
What Actually Breaks
I've collected the failure modes from post-mortems across three different teams. They're consistent.
The supervisor becomes the bottleneck. Everything runs through it. Every task, every handoff, every state transition. The latency compounds with each hop, and the supervisor's context window saturates as workflows grow. The 2026 survey literature identifies three recurring failure modes: task assignment errors, context saturation, and plan drift on long runs.
The single point of failure is literal. AWS's own agent reliability guidance states the problem plainly: "The control plane is treated as a single point of failure with in-memory state, so its failure loses coordination context and disrupts every agent at once". I watched a production system lose 40 minutes of work because the orchestrator pod restarted. Every sub-agent was fine. The state was gone.
Error propagation is worse than you think. A 2026 study found that centralized orchestration limited error propagation to approximately 4.4×. Without it—in systems where agents communicate directly—errors amplified up to 17× before being caught. Centralization contains blast radius. That's the trade-off. You're choosing a controlled explosion over an uncontrolled one.
The centralized pattern is a reliability strategy built on containment. You accept a single point of failure because you can at least see where it fails.
The Decentralized Alternative
The shift I made was conceptual. Instead of a supervisor deciding what happens, I defined a graph where each agent is a node and edges represent the flow of control. No central router. Each agent decides for itself whether to answer or hand off to a peer.
This is the swarm pattern in LangGraph. Each agent is a node with a handoff_to_<peer> tool for each legal target. The shared graph state holds the conversation history and the active-agent pointer. When an agent invokes a handoff tool, control transfers to the named peer and continues.
The architectural difference is subtle but consequential:
| Dimension | Centralized (Supervisor) | Decentralized (Swarm/Handoff) |
|---|---|---|
| Control flow | Single supervisor | Peer negotiation |
| Scalability | Limited (3–7 agents) | High |
| Fault tolerance | Low (SPOF) | High |
| Debugging ease | High | Low |
| Token cost profile | Grows with conversation length | Grows with agent count |
| Task suitability | Well-defined workflows | Open-ended exploration |
The data comes from a comprehensive 2026 survey of LLM-based multi-agent orchestration frameworks. The trade-off is real: you gain scalability and fault tolerance, and you lose the comfortable single point of observation.
Who's Actually Shipping This
Lyft rebuilt their customer support system on a router-based multi-agent architecture using LangGraph. A meta agent acts as a stateful router, dispatching to specialized subagents that are themselves full StateGraph instances. The results are hard to argue with: agent development accelerated from roughly six months to just a few weeks, and non-technical domain experts can now build and refine agents directly. The router pattern isn't fully decentralized—Lyft still uses a meta agent—but the architecture is a graph of graphs rather than a single supervisor calling tools. Each subagent owns its own state machine.
Included Health built a federated multi-agent architecture they call the "Dot supergraph." Different product teams own different parts of the graph. A shared coverage sub-agent is inherited by every Deep Agent, so coverage questions can be answered mid-conversation without threading a capability through every routing path. The key insight: distributed development maps naturally onto graph composition. Each team builds and owns their service independently, and the graph stitches them together.
SWARM+, a research system for fully decentralized workload management, scales to 1,000 distributed agents with nearly equal workload distribution and reduced coordination overhead. The results: 99% job completion rate under single-agent failure, and at most 7.5% impact under 50% agent failure. That's the fault tolerance argument in numbers.
The Guardrails You Need
Decentralized orchestration is not a free lunch. The primary failure mode is obvious once you've seen it: handoff cycles. Agent A hands to B, B hands back to A, forever. No progress. No error. Just a loop burning tokens.
The fix is structural. A robust swarm detects handoff cycles (a peer already in the handoff trace), detects dead ends (no peer fits), and escalates to a human when either happens or the hop budget runs out. Microsoft's agent framework documentation recommends a chain depth limit of three to four handoffs to prevent context management from becoming unmanageable.
The load-bearing rule is simple: A → B → A → B without progress is the primary failure mode. Your graph must make that impossible.
Where This Fits in Your Architecture
I'm not going to pretend decentralized orchestration is always the right call. The 2026 survey is clear about task suitability: centralized patterns excel at well-defined workflows. Decentralized patterns excel at open-ended exploration.
Use centralized orchestration when your workflow is a pipeline with clear stages and a bounded set of agents. Use decentralized orchestration when your problem is a graph traversal, when agents need to negotiate, or when the topology of the work isn't known at design time.
The teams that are winning with this are the ones treating agent orchestration as a distributed systems problem, not a prompt engineering problem. They're writing state schemas. They're designing topology. They're thinking about failure domains and blast radius.
The Question I Keep Coming Back To
If your supervisor could talk, what would it say about why it keeps dropping tasks on long runs?
And more importantly, are you designing a system that can answer that question without guessing?
I'd love to hear where you've landed. Supervisor, swarm, or something in between?
Top comments (0)