Let me be blunt: I've spent the last three years watching teams build multi-agent systems and then quietly abandon them because they couldn't scale. The postmortems are always the same, the manager pattern. You know the one: a central "orchestrator agent" that routes every task, holds every decision, and becomes the single point of failure for your entire pipeline.
We keep building AI systems that look like 2010s microservice monoliths. And they fail the same way.
The Manager Pattern Is a Bottleneck You Designed
The manager pattern feels intuitive. You give an LLM a list of sub-agents, a description of their capabilities, and ask it to route tasks. It's elegant in a notebook. In production, it collapses under three pressures:
- Context window saturation. The manager must hold the full state of every task it routes. As workflows grow, the manager's context becomes the bottleneck—not the model's reasoning.
- Opaque failure modes. When the manager misroutes, you have no idea why. The decision lived inside an LLM call. There's no trace, no intermediate state, no way to debug.
- No parallel execution. A manager processes tasks sequentially. Your sub-agents are idle while the manager "thinks."
I watched a team spend four months tuning their manager prompt. They eventually discovered that simply allowing two sub-agents to run in parallel cut their pipeline latency by 60%—but the manager architecture couldn't express that.
The Graph Pattern: Agents as Nodes, State as Edges
The shift is conceptual. Instead of a manager deciding what happens, you define a graph where each agent is a node, edges define the flow of control, and a shared state object carries context between them.
LangGraph is the clearest implementation of this. You model your workflow as a state machine—literally a StateGraph with nodes and edges. Each node is a function, an LLM call, or an entire sub-agent. Edges can be conditional, enabling branches and loops based on the current state.
Here's the core insight from LangGraph's docs:
from langchain.agents import create_agent
from langgraph.graph import StateGraph, START, END
agent = create_agent(model="openai:gpt-5.5", tools=[...])
def agent_node(state: State) -> dict:
result = agent.invoke({
"messages": [{"role": "user", "content": state["query"]}]
})
return {"answer": result["messages"][-1].content}
workflow = (
StateGraph(State)
.add_node("agent", agent_node)
.add_edge(START, "agent")
.add_edge("agent", END)
.compile()
)
That State object is the key. Every node reads from it and writes to it. No manager holds the entire context. Each agent operates on exactly what it needs.
What Actually Scales: Expansion-Contraction
The most interesting production pattern I've seen comes from a paper by AWS engineers on Expansion-Contraction—a graph traversal pattern for compound AI systems.
Here's how it works:
- Expansion phase: Starting from a query origin, walk a domain graph outward. At each node, dynamically spawn an ephemeral specialist agent. Each agent operates on a small local context—just the data at that node and its neighbors.
- Contraction phase: Aggregate findings inward to produce a verdict.
The agent topology emerges from the data graph itself rather than being hand-designed. The results are striking:
| Metric | Expansion-Contraction | Single-Agent Baseline |
|---|---|---|
| Production supply chain accuracy | 98.2% | 84% |
| Public benchmark accuracy | 100% | 86% |
| NLP-complex microservice tracing | 85% | 55% |
| Token usage reduction (with caching) | 93.9% | — |
| Speedup from concurrent paths | 1.43× | — |
The key architectural advantage: avoiding context window saturation. A single agent trying to reason over a large graph degrades as complexity increases. The graph traversal pattern maintains accuracy because each agent works on a small, local context.
Who's Actually Using This
Lyft rebuilt their customer support system on a router-based multi-agent architecture using LangGraph. They route rider and driver requests across specialized subagents with safety checks, state management, and handoffs built into the graph flow. The results:
- Managing millions of interactions for riders and drivers
- Agent development accelerated from six months to a few weeks
- Non-technical domain experts can now build and refine agents directly
Cortex Grid, presented at IEEE, uses LangGraph-based orchestration with MCP for inter-agent communication. They represent each AI agent as a node in a graph, with directed edges defining interactions. Their recruitment use case runs parallel workflows for candidates and employers—resume evaluation, skill-gap analysis, AI interviews, and course recommendations—all in one orchestrated graph.
Eluna, a production warehouse operations system, encodes SOPs as directed acyclic graphs and delegates independent tasks to parallel sub-agents.
DynTaskMAS orchestrates asynchronous parallel operations through dynamic task graphs, achieving near-linear scalability for LLM-based multi-agent systems.
Why This Matters for Your Architecture
The graph pattern gives you three things the manager pattern cannot:
Observability. Every state transition is a checkpoint. You can replay the exact sequence of events that led to a failure. Lyft uses LangSmith for tracing and LLM-as-a-judge evaluation—you can't trace what you can't see.
Composability. A subgraph can be a node in a larger graph. You can embed a multi-agent system as a single step in a custom workflow. Lyft registers each specialized subagent as a subgraph node in their meta agent.
Resilience. Partial failures are recoverable. If one branch of the graph fails, the rest continues. The Expansion-Contraction paper shows graceful degradation as graph complexity increases—the opposite of the manager pattern's cliff-edge failure mode.
The Trade-Off You're Accepting
Graph-based orchestration isn't free. You're writing more code upfront. You're designing the topology instead of hoping an LLM figures it out. You're thinking about state schemas and edge conditions.
But here's what I've learned: the teams that treat agent orchestration as an architecture problem rather than a prompt engineering problem are the ones shipping to production.
The manager pattern is a shortcut that feels productive in development and becomes a liability in production. The graph pattern is more work upfront but scales linearly—not exponentially—with complexity.
So here's my question for you: If your multi-agent system's manager could talk, what would it say about why it keeps dropping tasks? And more importantly, are you designing a system that can answer that question, or one that just hopes the prompt gets better?
Top comments (0)