Multi-Agent Design Patterns: Beyond Sequential Chains

When engineers build their first LLM application, the journey almost always starts with a linear chain: Prompt -> LLM -> Output. If things get slightly more complex, they reach for a standard ReAct loop where a single model reasons, picks a tool, observes the result, and loops until finished.

In simple demos, this works. In production, it breaks.

In my previous deep-dive on building a real-time ASR engine, the primary architectural lesson was decoupling-separating audio ingestion from model processing to eliminate WebSocket backpressure. Multi-agent systems face a remarkably similar challenge: forcing a single generalist LLM to handle parsing, reasoning, code generation, and quality control creates cognitive backpressure, context contamination, and compounding failure rates. If node 2 hallucinates a variable, node 5 crashes.

As we explored when engineering the Autonomous Business Discovery Engine, moving past naive linear pipelines requires structured multi-agent choreography. When an agentic system must handle non-deterministic real-world inputs, decomposition and specialized control loops are the only way to build resilience.

Here is an architectural breakdown of the key multi-agent design patterns that take systems beyond sequential chains, how to implement them in LangGraph, and the trade-offs that come with each.

1. The Orchestrator-Worker (Router & Fan-Out) Pattern

In a naive linear pipeline, each task runs serially. If an agent needs to extract requirements from meeting transcripts, parse UI screenshots, and review API documentation, running them sequentially creates massive latency and forces irrelevant tokens into the shared context.

The Orchestrator-Worker pattern uses a central supervisor to classify the objective, break it down into independent tasks, and dispatch them to specialized sub-agents in parallel before synthesizing the final output.

Key Engineering Benefits:

• Context Isolation: Each worker operates inside a minimal, dedicated context window. The vision model only sees image payloads; the code parser only receives AST trees and schemas.
• Latency Reduction: Independent ingestion jobs execute concurrently via asyncio or worker pools.
• Specialized Routing: You can route specific nodes to targeted models (e.g., lightweight vision models for image processing, specialized coder models for code generation) rather than paying for a giant generalist model on every task.

LangGraph Implementation:

from typing import Annotated, List, TypedDict
import operator
from langgraph.graph import StateGraph, START, END
from langgraph.types import Send

class OverallState(TypedDict):
    objective: str
    subtasks: List[str]
    # Reducer appends parallel worker outputs concurrently
    worker_results: Annotated[List[dict], operator.add]
    final_output: str

class WorkerState(TypedDict):
    subtask: str

def orchestrator(state: OverallState) -> dict:
    # Break down the user objective into distinct, isolated work units
    tasks = [
        "extract_functional_specs",
        "parse_ui_wireframes",
        "audit_api_contracts"
    ]
    return {"subtasks": tasks}

def worker_node(state: WorkerState) -> dict:
    # Each worker executes independently in its own scoped context
    task = state["subtask"]
    result = f"Processed {task} successfully"
    return {"worker_results": [{"task": task, "result": result}]}

def fan_out(state: OverallState):
    # Dispatch tasks dynamically to parallel instances of worker_node
    return [Send("worker_node", {"subtask": t}) for t in state["subtasks"]]

def synthesizer(state: OverallState) -> dict:
    # Combine results once all workers complete
    combined = "\n".join([f"- {r['task']}: {r['result']}" for r in state["worker_results"]])
    return {"final_output": f"Consolidated Summary:\n{combined}"}

# Build the Graph
builder = StateGraph(OverallState)
builder.add_node("orchestrator", orchestrator)
builder.add_node("worker_node", worker_node)
builder.add_node("synthesizer", synthesizer)

builder.add_edge(START, "orchestrator")
builder.add_conditional_edges("orchestrator", fan_out, ["worker_node"])
builder.add_edge("worker_node", "synthesizer")
builder.add_edge("synthesizer", END)

graph = builder.compile()

2. The Generator-Critic (Evaluator-Optimizer) Pattern

One of the most persistent failure points in automated generation is assuming LLM output will compile or meet business rules on the first pass.

In our Business Discovery POC build, when our code generation node attempted to create an in-memory dashboard, it repeatedly hallucinated unimported libraries and attempted to open persistent database connections on an ephemeral runtime.

The Generator-Critic pattern solves this by introducing a closed feedback loop:

1. Generator Node: Drafts the initial artifact (code, document, plan).
2. Evaluator / Critic Node: Runs multi-tier validation (e.g., deterministic AST static analysis, runtime mocking, semantic checks).
3. Conditional Routing: If validation passes, emit to downstream consumers. If validation fails, route structured diagnostics back to the Generator to self-correct.

Deterministic vs. Semantic Evaluation

A common mistake is using another LLM to grade every aspect of the generator's output. Semantic evaluation is slow, costly, and can hallucinate its own critique.

The most resilient agent systems combine deterministic validators (linters, AST checks, schema validators, sandbox test runs) with an LLM-based semantic evaluator:

import ast

def lint_generated_python(code_str: str) -> list[str]:
    """Deterministic AST pass to catch syntax and forbidden imports before LLM critique."""
    errors = []
    try:
        tree = ast.parse(code_str)
    except SyntaxError as e:
        return [f"SyntaxError on line {e.lineno}: {e.msg}"]

    banned_modules = {"sqlite3", "psycopg2", "mysql"}
    for node in ast.walk(tree):
        if isinstance(node, ast.Import):
            for alias in node.names:
                if alias.name.split(".")[0] in banned_modules:
                    errors.append(f"ForbiddenImport: `{alias.name}` is disallowed. Use in-memory state.")
    return errors

When deterministic linters catch an error, they format a raw traceback directly into the graph state. The Generator receives targeted diagnostic feedback rather than a vague prompt like "Please fix your mistakes."

3. Circuit Breakers & State Checkpointing

Cyclic graphs and self-healing loops introduce a new operational risk: infinite execution loops. If an agent cannot resolve a compilation error, it will burn through inference compute indefinitely.

Every production multi-agent system requires hard circuit breakers and durable state management:

1. Loop Counters & Graceful Degradation

Always attach an explicit iteration counter to the graph state. When the retry threshold is reached (e.g., iteration_count >= 3), route execution to a fallback node instead of crashing. This fallback can surface a partial artifact alongside the diagnostic log for human-in-the-loop review.

def should_continue(state: AgentState) -> str:
    if state["validation_status"] == "PASSED":
        return "deploy_node"
    if state["iteration_count"] >= 3:
        return "fallback_human_review_node"
    return "generator_node"

2. Upgrading Checkpointers for Concurrency

In local experimentation, LangGraph's ephemeral MemorySaver works fine. However, in enterprise microservices where requests are distributed across multiple worker nodes (e.g., Celery / FastAPI), in-memory checkpoints cannot be shared.

Upgrading to persistent state backends like PostgresSaver or RedisSaver ensures that:

• Any available worker can resume execution on an active thread_id.
• Human-in-the-loop approvals can pause state safely for hours or days without consuming memory.
• If a worker node crashes mid-generation, the agent resumes execution from the last successful node checkpoint rather than restarting the entire workflow.

Pattern Comparison Matrix

Architecture Pattern	Best Used For	Latency Profile	Primary Failure Mode	Mitigation Strategy
Sequential Chain	Fixed, deterministic pipelines (ETL, step-by-step summaries)	Low / Predictable	Cascading error propagation	Strong schema validation between nodes
Orchestrator-Worker	Multimodal inputs, large document parsing, independent sub-queries	Medium (Parallelized)	Incomplete task decomposition	Structured schema outputs on Orchestrator
Generator-Critic	Code generation, complex reasoning, structured JSON extraction	Variable (Depends on retries)	Infinite retry loops & token burn	Strict retry counters + AST/Deterministic linters
Human-in-the-Loop Swarm	High-stakes workflows (Financial transactions, live DB migrations)	High (Awaits user approval)	Worker desynchronization & state drift	Distributed persistent checkpointers (PostgresSaver)

Key Takeaways

1. Stop overloading single prompts: Decompose complex tasks into specialized sub-agents. Give each worker minimal, isolated context to eliminate noise and reduce context window exhaustion.
2. Combine deterministic linters with LLM critics: Don't rely solely on LLMs to judge other LLMs. Use AST trees, unit tests, and schema validators to give generator nodes precise, actionable bug tracebacks.
3. Engineer for failure: Build loop counters, circuit breakers, and persistent checkpointer backends from day one so your agents degrade gracefully instead of looping infinitely.

As agent systems continue to evolve, the differentiator between fragile prototypes and production systems isn't model size-it is architecture, isolation, and deterministic control loops.

What multi-agent design patterns have you found most effective in production? Let me know in the comments or connect with me on GitHub/LinkedIn!