Imagine an AI agent with 15 tools in a long conversation. Initially, it works fine. But as the conversation grows, it starts skipping steps. Then it tries to use a tool that doesn't exist, with made-up inputs. Nothing crashes. The agent just slowly gets worse at its job.
That's why more developers now build multi-agent systems. Instead of one agent doing everything, you split the work across several smaller agents, each with a single, clear job. A coordinator decides which agent handles which task. Developers call this agent orchestration. In this primer, you'll learn the main design patterns, how agents pass information to each other, how to build a simple orchestrator, and how to keep the whole system safe in production.
Why Single-Agent Falls Over
A single agent works well for narrow jobs. It breaks in three predictable ways as the job grows.
Context window bloat
Every tool schema, retrieved document, and prior turn competes for the same context window. Add tools and history, and the most important instructions get diluted. The model doesn't fail loudly; it starts missing steps.
One model doing three jobs
You ask the same prompt to plan, retrieve, and execute. Each job wants different instructions, different tools, and often a different model. Cramming them together forces compromises in all three.
Opaque debugging
When it goes wrong, you get one long trace. There are no natural seams, so you can't tell whether the plan, the retrieval, or the tool call broke. Splitting the work into agents gives you those seams for free: each agent has its own input, output, and logs.
The trade-off is costly.
Anthropic found that its multi-agent research system, a Claude Opus 4 lead agent with Claude Sonnet 4 subagents, outperformed single-agent Claude Opus 4 by 90.2% on its internal research eval. The same system used about 15× more tokens than a chat interaction. Multi-agent wins on hard, parallelizable work. On simple tasks, you pay more for no gain.
Three Agent Orchestration Patterns, and Where Each One Breaks
Most production multi-agent systems use one of three topologies. Pick based on where you can afford to fail.
Start with the orchestrator-worker. It gives you one place to log, route, and stop things, and you can move to a hybrid later. Most teams who start peer-to-peer end up adding a coordinator anyway, just to see what's happening.
State and Context Handoff Between Agents
This is where most multi-agent prototypes quietly fail. You have two ways to move state between agents, and each breaks differently.
Shared state feels simpler at two agents. At five, nobody can say which agent wrote a value, or when. Use a shared store for read-mostly reference data, such as product catalogs or embeddings. Pass working state as explicit payloads.
Schema drift: the silent killer
Agent A stops sending customer_tier because someone simplified its prompt. Agent B still expects it, defaults to None, and prices every customer at the standard rate. No error fires. Revenue just leaks.
Three habits prevent this:
Validate at every boundary. Parse each agent's output into a schema (Pydantic, Zod or JSON Schema) and fail loudly on missing fields.
Version the payload. Add a schema_version field so a consumer can reject a shape it doesn't understand.
Never let free text cross a boundary. Ask the model for structured output, and treat prose as a bug in an agent's contract.
from pydantic import BaseModel
class PricingRequest(BaseModel):
schema_version: int = 2
customer_id: str
customer_tier: str # required: B fails loudly if A drops it
seats: int
A Minimal Orchestrator Pattern
Here's an orchestrator–worker skeleton in plain Python, standard library only. It runs as-is. Swap the lambdas for LLM calls and classify() for a model that returns one label from a fixed set.
It does three things most tutorials skip: every agent returns a structured result, every agent has its own timeout and circuit breaker, and every call writes an audit line.
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass, field
import time
@dataclass
class AgentResult:
agent: str
status: str # "ok" | "error" | "skipped"
data: dict = field(default_factory=dict)
class CircuitBreaker:
def init(self, max_failures=3, cooldown_s=60):
self.max_failures, self.cooldown_s = max_failures, cooldown_s
self.failures, self.opened_at = 0, None
def allow(self):
if self.opened_at and time.time() - self.opened_at < self.cooldown_s:
return False
return True
def record(self, ok):
self.failures = 0 if ok else self.failures + 1
self.opened_at = time.time() if self.failures >= self.max_failures else None
class Agent:
def init(self, name, handler, timeout_s=10):
self.name, self.handler, self.timeout_s = name, handler, timeout_s
self.breaker = CircuitBreaker()
pool = ThreadPoolExecutor(max_workers=4)
def run_agent(agent, request):
if not agent.breaker.allow():
return AgentResult(agent.name, "skipped", {"reason": "circuit open"})
try:
data = pool.submit(agent.handler, request).result(timeout=agent.timeout_s)
agent.breaker.record(ok=True)
return AgentResult(agent.name, "ok", data)
except Exception as exc: # includes timeouts
agent.breaker.record(ok=False)
return AgentResult(agent.name, "error", {"error": repr(exc)})
def classify(request):
# Swap in an LLM call that returns one label from a fixed set.
text = request["text"].lower()
if "price" in text or "quote" in text:
return "pricing"
if "book" in text or "meeting" in text:
return "scheduling"
return "fallback"
AGENTS = {
"pricing": Agent("pricing", lambda r: {"quote_usd": 1200}),
"scheduling": Agent("scheduling", lambda r: {"slot": "2026-10-08T10:00"}),
"fallback": Agent("fallback", lambda r: {"route_to": "human"}),
}
def orchestrator(request):
task = classify(request)
result = run_agent(AGENTS[task], request)
if result.status != "ok":
result = run_agent(AGENTS["fallback"], request)
print(f"audit agent={result.agent} status={result.status} data={result.data}")
return result
if name == "main":
orchestrator({"text": "Can I get a price for 3 seats?"})
orchestrator({"text": "Book a meeting next week"})
orchestrator({"text": "My login is broken"})
Output:
audit agent=pricing status=ok data={'quote_usd': 1200}
audit agent=scheduling status=ok data={'slot': '2026-10-08T10:00'}
audit agent=fallback status=ok data={'route_to': 'human'}
Note what the orchestrator doesn't do: it never parses free text from a worker. Each agent returns an AgentResult, so the orchestrator, and any agent downstream, can act on it reliably. In production, swap print for structured logging and the in-memory breaker for a shared one (Redis works) so every replica sees the same state.
Where Multi-Agent Systems Actually Break in Production
The biggest production risk isn't one agent failing. It's one agent failing quietly.
Cascading failures
An extraction agent misreads a date. The validation agent trusts it. The scheduling agent books the wrong slot, and the notification agent confirms it to the customer. Every agent did its job correctly with bad input. In a multi-agent system, errors compound at every handoff instead of staying put.
Containment is an architecture problem
A 2026 Dataiku survey, run by Harris Poll, found that only 5% of UAE CIOs said their organisation could reliably identify and contain a problematic AI agent within one to two hours. The global figure was 10%.
Read that as an engineer, and it's rarely a process failure. It's an AI architecture gap: no circuit breaker at the orchestration layer, no per-agent kill switch, and no way to halt one agent without taking down the whole pipeline. If stopping one misbehaving agent means a redeploy, containment will always take hours.
The cost of skipping this shows up later.
Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, or inadequate risk controls. Weak controls are the one cause on that list that engineers directly own.
The takeaway: design the circuit breaker before you need it, not after your first incident.
Guardrails Worth Building In From the Start
Each of these is cheap on day one and painful to retrofit after launch.
A few specifics:
Time out per agent, not just per request - A global 30-second timeout tells you something was slow. A per-agent timeout tells you which agent, and lets the orchestrator fall back while the others finish.
Make the breaker controllable - Expose a flag your on-call engineer can flip to disable one agent without a redeploy. That flag is your containment plan.
Log the decision, not just the call - Record which agent ran, its input, its output, and what the orchestrator did next. One end-to-end trace won't show you where a cascade started.
*Gate irreversible actions *- Let agents draft the payment, the email, or the filing. Let a human, or at least a deterministic rule, approve it.
Picking a Framework
The framework matters less than the patterns above, but it shapes how fast you move. Here's the landscape most teams compare today, with no endorsement.
One note if you're evaluating AutoGen: Microsoft now describes Agent Framework as its successor and keeps AutoGen on bug fixes and security patches. For new builds, start with Agent Framework.
Don't underrate rolling your own. If your workflow has three agents and a clear routing rule, a framework can add more abstraction than value. Good LLM development practice applies either way: pin model versions, test each agent in isolation, and keep prompts in version control next to the code.
Wrapping Up
Start with the orchestrator–worker. Pass typed payloads instead of shared state. Give every agent its own timeout, breaker, and audit log. Then scale the number of agents, not the size of their prompts.
If you're scoping a production multi-agent build, not just a prototype, here's how we approach the architecture and integration side: AI Agent Development.
FAQs
- How many agents should a first multi-agent system have?
As few as the workflow needs, usually two to four. Every agent adds a handoff, a schema, and a failure point. Split an agent only when its prompt, tools, or context window clearly overload it.
- Do all agents need to run on the same LLM?
No, and they often shouldn't. Use a stronger model for the orchestrator or planner and smaller, cheaper models for narrow workers such as classification or extraction. Anthropic's research system pairs an Opus lead agent with Sonnet subagents for this reason.
- How do I test a multi-agent system before production?
Test each agent in isolation against fixed inputs and expected structured outputs, like a unit test. Then replay logged real requests through the full pipeline and compare results. Your audit logs from the orchestrator double as the replay dataset.
- Should agents share one vector database?
Share it for read-mostly reference data, such as documentation or a product catalogue. Don't use it as a scratchpad for working state between agents. Pass that state as typed payloads so you can trace who changed what.
- How do I stop multi-agent costs from spiralling?
Measure tokens per agent, not just per request. Cap each agent's iterations, cache repeated retrievals, and route simple requests past the expensive agents entirely. Multi-agent systems can use around 15× the tokens of a chat, so budget per agent.
- When should I use a framework instead of my own orchestrator?
Pick a framework when you need durable checkpointing, long-running workflows, or complex branching and loops. Roll your own when you have a handful of agents and a clear routing rule. You can migrate later if you keep agents behind a clean interface.
Top comments (1)
Official Platform Update
Security protocols have been updated for all developer accounts.