Most multi-agent demos work great in a notebook and fall apart within two weeks of production traffic. Not because the model is wrong because the orchestration layer was never designed for the failure modes that only show up at scale: partial tool failures, context drift across long conversations, and agents that silently disagree with each other.
Here's what actually breaks, and the patterns that fix it.
- Agents don't fail loudly — they fail "plausibly"
A single-agent system that hits an API timeout usually throws an error you can catch. A multi-agent system where Agent A calls Agent B, which calls a tool that returns a malformed but parseable response, tends to produce output that looks reasonable and is wrong. This is the failure mode that costs you trust, not uptime.
The fix isn't more prompt engineering — it's treating every inter-agent handoff like an API contract:
\`python
class AgentResponse(BaseModel):
result: dict
confidence: float
tool_calls: list[ToolCall]
validation_status: Literal["verified", "unverified", "failed"]
def handoff(agent_output: AgentResponse) -> AgentResponse:
if agent_output.validation_status != "verified":
return escalate_to_human_or_retry(agent_output)
return agent_output
`\
Every handoff between agents should validate against a schema before the next agent trusts it. This sounds obvious written down; it's the single most common thing missing from agent frameworks used as-is out of the box.
2. State management is the actual hard problem
Prompt engineering gets the attention, but the thing that determines whether your agent system survives a 40-turn conversation is how you manage state across agents that each have partial visibility into the task.
Three patterns, in order of how much complexity they can handle:
- Shared scratchpad — a single mutable context object all agents read/write to. Simple, but doesn't scale past 3–4 agents before agents start stepping on each other's writes.
- Message-passing with a supervisor — a coordinator agent owns state, worker agents only see what's relevant to their sub-task. Scales better, costs more latency per hop.
- Externalized state store — state lives outside any agent's context window entirely (a database, a graph), agents read/write via tools rather than holding state in-context. This is the pattern that survives long-running, multi-day workflows, and it's the one worth building toward if you're past the prototype stage — it's close to the orchestration pattern we use for regulated, multi-step workflows where every state transition needs to be auditable, not just functional.
3. "Agent disagreement" needs an explicit resolution strategy
When two agents produce conflicting outputs (a common failure once you have more than 2–3 agents with overlapping responsibility), most systems just take whichever one finishes last. That's not a resolution strategy, it's a race condition wearing a costume.
Decide up front, per workflow: does disagreement trigger (a) a third arbitration pass, (b) escalation to a human, or (c) a deterministic tie-break rule? Pick one and encode it — don't let it emerge by accident.
The uncomfortable summary
Multi-agent orchestration isn't a prompting problem. It's a distributed-systems problem wearing an AI costume — the same discipline you'd apply to any system with partial failures, async handoffs, and conflicting writers applies here, almost unchanged. If you're building this and it feels like you're reinventing microservices patterns, that's because you are.
What's breaking in your agent systems? Curious what patterns others have landed on for state management specifically — drop it in the comments.
Top comments (0)