Day 24: Multi-Agent Orchestration — Supervisor, Worker and Evaluator Patterns

As a workflow grows, handing one agent every tool and a long list of responsibilities makes it worse. The prompt gets crowded, the model picks the wrong tool and the context window fills up. The fix is the one human organisations use: divide the work among specialists who collaborate.

In plain terms: A hospital does not give one doctor every job. A coordinator sends each patient to the right specialist, and a second doctor reviews the plan before treatment begins.

Diagram: Giving one agent every tool and a huge prompt dilutes its focus and causes tool-selection mistakes. Splitting work across specialised agents with narrow scopes and short prompts is usually more reliable and easier to test. See the animated version.

Three patterns

• Worker (specialisation). A small agent with a narrow scope and its own limited tools. A log-parser worker can only read logs. A database worker has a read-only SQL connection. A tight scope improves accuracy and limits damage.
• Supervisor (routing and delegation). The coordinator. It reads a high-level request, splits it into sub-tasks, hands them to the right workers and combines the results into one answer.
• Evaluator (validation and guardrails). An independent agent that reviews worker output against policy, accuracy rules and business logic before anything is committed or shown to the user. Keeping it separate matters, because a worker checking its own work tends to approve it.

Diagram: A supervisor splits a request into sub-tasks and routes them to narrow worker agents, each with its own limited tools. An independent evaluator reviews their output against policy and quality rules before anything is committed or returned to the user. See the animated version.

State and orchestration

Coordinating several agents calls for a real state machine. Frameworks such as LangGraph (Python) and Mastra (TypeScript) model the system as a directed graph.

• Nodes are agents, tool executions or human-approval checkpoints.
• Edges are routing rules based on what earlier nodes returned.
• Durable state records progress after each step (a checkpoint), so if a pod crashes mid-run the workflow resumes from the last confirmed node instead of starting over.

Diagram: Frameworks such as LangGraph or Mastra model multi-agent workflows as directed graphs. Nodes are agents, tool calls or human approval checkpoints, and edges are routing decisions based on the outputs of earlier nodes. See the animated version.

Diagram: Saving a checkpoint after each step means that if a pod crashes mid-run the workflow resumes from the last confirmed step instead of restarting from the beginning. Frameworks such as LangGraph provide checkpointing for this. See the animated version.

Here is the pattern without any framework, so the idea is clear. The ask_* functions stand in for model calls with different system prompts and tools, and a framework adds the graph, retries and persistent checkpoints for you.

def run_workflow(request: str, state: dict) -> str:
    # Supervisor splits the request into sub-tasks (cached in state so a restart skips done work)
    state.setdefault("plan", ask_supervisor(request))            # e.g. ["parse_logs", "check_db"]

    for task in state["plan"]:
        if task in state.setdefault("results", {}):
            continue                                              # resume: this step already finished
        worker = WORKERS[task]                                    # each worker has its own narrow tools
        state["results"][task] = worker(request)
        save_checkpoint(state)                                    # durable state after every step

    draft = ask_supervisor_combine(state["results"])
    verdict = ask_evaluator(draft, policy=POLICY)                 # independent quality gate
    if verdict != "approved":
        return "Held for human review"                            # an approval node in a real graph
    return draft

Structuring agents as modular teams mirrors high-performing engineering organisations: clear roles, small scopes and a reviewer, which is what makes complex tasks dependable.

Coming Up Next

Day 25: Automated CI/CD: automating the software lifecycle for AI models.

MultiAgentSystems #LangGraph #SystemDesign #AgenticAI #SoftwareEngineering

---

Originally published at https://sureshpallapothu.in/blog/day-24-multi-agent-orchestration, where this post includes animated diagrams.