Most "multi-agent" demos are one agent with a longer prompt. A real supervisor does something simpler and more useful: it decides who should handle a request, passes the right context along, and stops when it should.
Quick disclosure: my day job is enterprise AI agent deployments at AI Hive. Nothing below depends on it. The code is plain Python with a stub where your model call goes.
Do you need more than one agent?
Probably not at first. Every extra agent adds latency, cost, and a new place for things to go wrong. I only split work when two jobs need different instructions, different tools, or different permissions.
A billing question and a product bug are a good example. They need different knowledge and different access, and mixing them in one prompt makes both answers worse. That's the case I'll build.
The pieces
There are three parts: a router that picks a specialist, the specialists themselves, and a loop that limits how many handoffs can happen. First, the model call and the specialists.
import json
from dataclasses import dataclass
def call_model(system: str, messages: list[dict]) -> str:
"""Replace with your provider's client. Must return plain text."""
raise NotImplementedError
@dataclass
class Specialist:
name: str
system: str
SPECIALISTS = {
"billing": Specialist(
"billing",
"You answer billing questions in three sentences or fewer. "
"If the question is not about billing, reply with 'HANDOFF: ' "
"and one line saying what is needed.",
),
"tech": Specialist(
"tech",
"You troubleshoot product problems step by step. "
"If the problem is really about billing, reply with 'HANDOFF: ' "
"and one line saying what is needed.",
),
}
ROUTER_PROMPT = (
"Route the customer message to exactly one specialist. "
'Reply with JSON only: {"agent": "billing" or "tech" or "none", '
'"reason": "short reason"}'
)
The HANDOFF: prefix is the whole handoff protocol. A specialist that realises the request isn't its job says so in a fixed format, and the supervisor reads it.
The router
The router asks the model for JSON and never trusts it blindly. If the output doesn't parse, or names an agent that doesn't exist, I treat it as "none".
def route(message: str) -> dict:
raw = call_model(ROUTER_PROMPT, [{"role": "user", "content": message}])
try:
decision = json.loads(raw)
except json.JSONDecodeError:
return {"agent": "none", "reason": "router output was not valid JSON"}
if not isinstance(decision, dict) or decision.get("agent") not in SPECIALISTS:
return {"agent": "none", "reason": "no matching specialist"}
return decision
That small check matters. Routers fail in boring ways, like extra text around the JSON, and a crash here takes the whole conversation with it.
Keep the router prompt short and list the allowed values explicitly. When you add a third specialist, update this prompt and your test messages first, before you touch the specialist itself.
The supervisor loop
Now the loop. It routes, calls the specialist, and checks for a handoff. It gives up after max_hops and returns an escalation instead of looping forever.
def handle(message: str, max_hops: int = 2) -> dict:
log = []
context = message
for hop in range(max_hops):
decision = route(context)
log.append({"hop": hop, "route": decision})
if decision["agent"] == "none":
break
specialist = SPECIALISTS[decision["agent"]]
reply = call_model(specialist.system, [{"role": "user", "content": context}])
log.append({"hop": hop, "agent": specialist.name, "reply": reply})
if not reply.startswith("HANDOFF:"):
return {"status": "answered", "reply": reply, "log": log}
note = reply.removeprefix("HANDOFF:").strip()
context = f"{message}\n\nNote from the previous specialist: {note}"
return {"status": "escalated", "log": log}
Why is max_hops 2 and not 10? Every hop is another model call, so a bad handoff loop gets expensive fast. Two hops covers the usual case where the first specialist realises the request belongs elsewhere. Anything past that means the routing is wrong, and a person should see it. If you raise the limit, raise it for a reason you can point to in the logs.
Two details I'd keep in any version of this. The original message always travels with the handoff note, so the next specialist never works from a summary alone. And the function returns the full log, which is what you'll want open when a customer says the agent bounced them around.
Testing it without a model
Swap in a fake call_model and check the routing with plain asserts. No API key needed.
def fake_model(system: str, messages: list[dict]) -> str:
text = messages[-1]["content"].lower()
if system == ROUTER_PROMPT:
if "invoice" in text:
return '{"agent": "billing", "reason": "invoice question"}'
if "crash" in text:
return '{"agent": "tech", "reason": "product issue"}'
return '{"agent": "none", "reason": "no match"}'
return "Here is the answer."
call_model = fake_model
assert handle("I was charged twice on my invoice")["status"] == "answered"
assert handle("The app crashes on login")["status"] == "answered"
assert handle("What is the weather?")["status"] == "escalated"
Once this passes, collect 20 or 30 real messages with the specialist you expect for each one. Run them through the real router before you change a prompt, and again after. A routing change that looks harmless can quietly send one category to the wrong place.
What I'd add next
Three things, in this order. Per-specialist tool allow-lists, so billing can't call anything in the support system. A confidence field from the router, with a low score going straight to a human. And a trace ID on every log line so one conversation can be followed across agents.
If you'd rather build this kind of routing on a canvas, the Agent Studio and multi-agent sections of the AI Hive platform features page cover the visual and Python SDK sides of the same idea.
How do you decide when a second agent is worth the extra moving parts?
Top comments (0)