A multi-agent system without an orchestrator is just a collection of agents. Each one is capable, but none of them coordinated. They might all be excellent at their individual jobs - searching the web, writing code, calling APIs - but without something deciding what gets done, in what order, by whom, and what to do when a result comes back wrong, the system does not behave like a system. It behaves like a group project with no project manager.
The orchestrator is the project manager. Its job is not to do the work. Its job is to make sure the work gets done - and that is a harder, more subtle problem than it sounds.
What an orchestrator is responsible for
An orchestrator does four things, and only these four things:
1. Decompose the goal. Turn a high-level objective into a concrete set of subtasks. This is a planning problem, not an execution problem. The orchestrator decides what needs to happen, not how to do it.
2. Route tasks to the right workers. Match each subtask to an agent capable of doing it. This requires knowing what tools and capabilities each worker has - not in detail, but well enough to delegate correctly.
3. Manage state across the workflow. As workers return results, the orchestrator decides what those results mean for the remaining plan. Sometimes a result changes the plan entirely. Sometimes it confirms the next step. The orchestrator holds the full picture.
4. Synthesise the final output. Worker outputs are partial. The orchestrator assembles them into a coherent response and decides when the goal has been met.
Notice what is absent: the orchestrator does not call APIs, does not run code, does not search the web. It reasons about work and routes it. The moment an orchestrator starts executing, it loses the focus that makes it good at coordination.
Building one from scratch
Here is a minimal orchestrator in Python. It plans upfront, delegates to type workers, and synthesizes results:
import json
def orchestrator(goal: str, workers: dict) -> str:
# Step 1: plan
plan_prompt = f"""
Goal: {goal}
Available workers: {list(workers.keys())}
Return a JSON array of steps: [{{"task": "...", "worker": "..."}}]
Return JSON only, no explanation.
"""
plan = json.loads(llm(plan_prompt))
# Step 2: execute each step, collect results
results = []
for step in plan:
worker = workers[step["worker"]]
result = worker.run(step["task"])
results.append({"task": step["task"], "result": result})
# Step 3: synthesise
synthesis_prompt = f"""
Original goal: {goal}
Worker results: {json.dumps(results, indent=2)}
Synthesise a final answer from these results.
"""
return llm(synthesis_prompt)
Three things in this snippet are worth pulling apart.
The orchestrator calls llm() twice - once to plan, once to synthesize - but never once to execute. Execution is entirely delegated. If you find yourself adding a tool call directly inside the orchestrator loop, stop and ask whether a worker should own that instead.
The plan is a first-class object - a list of typed steps, not an implicit chain of thought. This means you can inspect it, log it, replay it, and re-plan from any point when something goes wrong.
Results are kept verbatim before synthesis. The orchestrator does not summarise early. It gives the synthesis step the full picture, letting the model decide what is relevant. Premature summarization is where context gets lost.
Re-planning when reality diverges
A fixed plan fails the moment a worker returns an unexpected result. A real orchestrator needs to decide: Does this change the remaining plan?
def orchestrator_with_replan(goal: str, workers: dict) -> str:
plan = initial_plan(goal, workers)
completed = []
while plan:
step = plan.pop(0)
result = workers[step["worker"]].run(step["task"])
completed.append({"task": step["task"], "result": result})
# check if the result warrants re-planning
replan_prompt = f"""
Remaining plan: {plan}
Latest result: {result}
Does this result change what should happen next?
If yes, return a revised plan as JSON. If no, return null.
"""
revision = llm(replan_prompt)
if revision and revision != "null":
plan = json.loads(revision)
return synthesise(goal, completed)
This is the plan-and-execute pattern from the last post made concrete. The orchestrator works from a checklist but checks after each result whether the checklist still makes sense. The loop terminates when the plan is empty, not when a fixed number of steps has run.
What frameworks add
Writing an orchestrator from scratch gives you control and understanding, but production use cases introduce problems the snippet above does not handle: state that needs to persist across restarts, workflows that need branching and loops, and debugging when something goes wrong three levels deep. This is where the frameworks come in.
LangGraph
LangGraph represents a workflow as a directed graph - nodes are agents or functions, edges are transitions between them, and a centralized StateGraph holds shared state across the whole run.
from langgraph.graph import StateGraph, END
from typing import TypedDict
class WorkflowState(TypedDict):
goal: str
plan: list
results: dict
final_answer: str
graph = StateGraph(WorkflowState)
graph.add_node("planner", planner_node)
graph.add_node("researcher", researcher_node)
graph.add_node("coder", coder_node)
graph.add_node("synthesiser", synthesiser_node)
graph.add_conditional_edges(
"planner",
route_to_worker, # function that reads state and picks next node
{"research": "researcher", "code": "coder"}
)
graph.add_edge("researcher", "synthesiser")
graph.add_edge("coder", "synthesiser")
graph.add_edge("synthesiser", END)
graph.set_entry_point("planner")
app = graph.compile()
The graph-based model earns its complexity because it makes branching and cycles explicit. The conditional edge above is not hidden inside a prompt - it is code. You can read the graph and know exactly what routes exist. LangGraph also ships with checkpointers that persist state to disk or a database, so a workflow that crashes halfway through can resume from the last checkpoint rather than starting over.
The honest tradeoff: a simple workflow that would take 40 lines in plain Python takes closer to 120 in LangGraph. You pay in boilerplate. You get auditability, resumability, and explicit control flow in return.
AutoGen
AutoGen takes a different approach. Instead of a graph, it models orchestration as message-passing between agents. A GroupChat manager decides which agent speaks next based on context, and agents broadcast their replies so everyone shares the same conversation history.
from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager
planner = AssistantAgent("planner", system_message="Break goals into tasks.")
researcher = AssistantAgent("researcher", system_message="Search and retrieve facts.")
coder = AssistantAgent("coder", system_message="Write and run Python code.")
critic = AssistantAgent("critic", system_message="Review outputs for errors.")
group_chat = GroupChat(
agents=[planner, researcher, coder, critic],
messages=[],
max_round=12
)
manager = GroupChatManager(groupchat=group_chat)
planner.initiate_chat(manager, message="Build a data pipeline for X.")
The GroupChatManager is the orchestrator here. AutoGen v0.4 rebuilt this around an actor model where each agent runs independently and communicates through typed messages - a cleaner design for truly concurrent workflows.
The full-context broadcast is AutoGen's key architectural bet: every agent sees the whole conversation, which means specialist agents can catch problems earlier in the chain. The cost is that the context window fills faster in long workflows.
CrewAI
CrewAI uses a role-driven model. You define a crew of agents with named roles, assign tasks, and let the framework handle delegation. Configuration-first rather than code-first - most of the setup lives in YAML:
from crewai import Agent, Task, Crew, Process
researcher = Agent(role="Researcher", goal="Find accurate information",
backstory="Expert at web research", tools=[search_tool])
writer = Agent(role="Writer", goal="Produce clear summaries",
backstory="Experienced technical writer")
research_task = Task(description="Research topic X", agent=researcher)
write_task = Task(description="Write a summary of the research", agent=writer)
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, write_task],
process=Process.sequential
)
crew.kickoff()
Process.hierarchical switches to an orchestrator model where a manager agent routes tasks dynamically rather than following a fixed sequence. CrewAI is the fastest path from idea to running prototype, but teams consistently report hitting its ceiling 6–12 months in when workflows grow beyond sequential or simple hierarchical patterns - at which point a migration to LangGraph tends to follow.
The most honest guidance: start from scratch to understand the pattern, then adopt a framework when you need what it specifically offers - not because frameworks are the default. LangGraph is the current production default for teams that need long-running, resumable workflows with explicit control flow. CrewAI is the right choice when you need something working this week and can accept the ceiling. AutoGen is strongest when agents genuinely need to debate and revise rather than execute a fixed plan.
The orchestrator's failure modes
The orchestrator is the single point of failure in a multi-agent system, which means its failure modes are expensive.
Over-delegation. The orchestrator sends a task to a worker that is underspecified. The worker returns garbage. The orchestrator synthesises the garbage. Build task schemas - typed descriptions of what a worker expects - and validate them before dispatch, not after.
Plan rigidity. An upfront plan that doesn't re-evaluate when results diverge will execute confidently toward a wrong answer. Build in a replan check after any result that introduces new information.
Silent worker failures. Workers that fail quietly return None or an empty string, which the orchestrator may synthesise as if it were real output. Workers should fail loudly with typed errors that the orchestrator can inspect and route around.
Context overload. The orchestrator accumulates results from every worker. In a long workflow, its context fills with raw worker outputs. Pass summaries, not raw transcripts, unless the raw content is genuinely needed for synthesis.
The orchestrator's value is its deliberate ignorance of the details. It does not know how to search the web, run code, or call an API. It knows what needs doing and who should do it. That separation - reasoning about work versus doing work - is what allows multi-agent systems to scale beyond what any single agent could manage alone.
The loop is still the heartbeat. The orchestrator is the brain that decides how many loops to start, what they should do, and when to stop.
Top comments (0)