Single-agent chatbots hit a wall fast — one model trying to plan, research, write, and verify all at once ends up doing all four badly. This article breaks down a real multi-agent architecture where specialized agents hand off work to each other, with the actual orchestration code behind it.
Overview: Why Multi-Agent Beats Single-Agent
A single LLM call trying to do everything runs into predictable problems:
- No separation of concerns — planning logic gets tangled with execution logic
- No verification step, so mistakes compound silently
- Context windows get bloated with irrelevant history from earlier steps
Splitting the work across specialized agents — a planner, a researcher, an executor, and a critic — fixes all three, because each agent only sees what it actually needs.
Before building your own orchestration layer from scratch, it's worth checking what's already out there. A software hub is useful for comparing agent frameworks (LangGraph, CrewAI, AutoGen) side by side before you commit to building everything custom.
Step 1: Define the Agent Roles
Each agent gets one job and a tightly scoped system prompt — no agent should be trying to do everything.
AGENTS = {
"planner": {
"role": "Break the user's goal into a numbered list of concrete subtasks.",
"model": "gpt-4o-mini"
},
"researcher": {
"role": "Given a subtask, gather relevant facts using available tools. Do not write final answers.",
"model": "gpt-4o-mini"
},
"executor": {
"role": "Given research and a subtask, produce the final output for that subtask.",
"model": "gpt-4o"
},
"critic": {
"role": "Review the executor's output against the original goal. Approve or request revisions.",
"model": "gpt-4o-mini"
}
}
Step 2: The Planner Breaks Down the Goal
from openai import OpenAI
client = OpenAI()
def run_agent(role, user_input, context=""):
system_prompt = AGENTS[role]["role"]
response = client.chat.completions.create(
model=AGENTS[role]["model"],
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"{context}\n\nTask: {user_input}"}
]
)
return response.choices[0].message.content
def plan(goal):
output = run_agent("planner", goal)
steps = [line.strip() for line in output.split("\n") if line.strip()]
return steps
steps = plan("Write a technical comparison of vector databases for a RAG pipeline")
print(steps)
Step 3: Researcher + Executor Hand-Off
The researcher gathers facts, and only then does the executor turn that into a final answer — keeping "finding information" and "writing the answer" as separate concerns.
def research(subtask):
return run_agent("researcher", subtask)
def execute(subtask, research_notes):
context = f"Research notes:\n{research_notes}"
return run_agent("executor", subtask, context)
results = []
for step in steps:
notes = research(step)
draft = execute(step, notes)
results.append({"step": step, "draft": draft})
Step 4: The Critic Loop (Where Quality Actually Comes From)
This is the step most demos skip — and it's the one that matters most. Without a verification pass, agents confidently ship wrong answers just as often as right ones.
def critique(goal, draft):
context = f"Original goal: {goal}"
verdict = run_agent("critic", draft, context)
return "approve" in verdict.lower()
def refine_until_approved(goal, step, notes, max_retries=2):
draft = execute(step, notes)
for attempt in range(max_retries):
if critique(goal, draft):
return draft
notes += "\nPrevious attempt was rejected. Improve accuracy and completeness."
draft = execute(step, notes)
return draft # return best effort after max retries
Step 5: Orchestrating the Full Workflow
Tie it all together with a simple state machine — no need for a heavy framework to get this working end-to-end.
def run_workflow(goal):
steps = plan(goal)
final_output = []
for step in steps:
notes = research(step)
approved_draft = refine_until_approved(goal, step, notes)
final_output.append(approved_draft)
return "\n\n".join(final_output)
result = run_workflow("Write a technical comparison of vector databases for a RAG pipeline")
print(result)
Each agent only sees the slice of context relevant to its job — the planner never sees research notes, and the researcher never sees the critic's feedback. That isolation is what keeps a multi-agent system from turning into one giant confused context window.
Handling Failures Gracefully
Agentic workflows fail in ways single-call apps don't — an agent can get stuck in a loop, hallucinate a tool call, or just time out. Always bound retries and log every hand-off.
import logging
logging.basicConfig(level=logging.INFO)
def run_agent_safe(role, user_input, context="", timeout_retries=2):
for attempt in range(timeout_retries):
try:
output = run_agent(role, user_input, context)
logging.info(f"[{role}] succeeded on attempt {attempt + 1}")
return output
except Exception as e:
logging.warning(f"[{role}] failed: {e}")
raise RuntimeError(f"Agent '{role}' failed after {timeout_retries} attempts")
Keeping Costs Under Control
Running four agents per task instead of one call adds up fast in token costs. Not every piece of the stack needs a paid tier though — plenty of orchestration frameworks, tracing tools, and evaluation libraries are free and open-source. Check an alternative of free softwares list before paying for commercial agent-monitoring platforms; tools like LangSmith's free tier or open-source tracing often cover what small teams actually need.
Final Thoughts
The difference between a flashy agent demo and a system that reliably finishes real tasks comes down to structure: clear role separation, a research-then-execute hand-off, and a critic loop that catches mistakes before they ship. Skip any one of those three, and the whole thing degrades back into a single confused agent guessing its way through a task.
Agent frameworks and underlying models change fast — what worked last month might already be deprecated. Check the updates software version website before deploying a multi-agent system to production, to make sure your framework and model versions are still current.
Top comments (0)