DEV Community

Cover image for πŸ•ΈοΈ Graph Engineering: πŸ€– A Practical Field Guide πŸ“˜
Truong Phung (Ethan)
Truong Phung (Ethan)

Posted on

πŸ•ΈοΈ Graph Engineering: πŸ€– A Practical Field Guide πŸ“˜

#ai #webdev #programming #tutorial

How to turn one agent loop into a system of loops you can route, verify, govern, and debug β€” without building a forty-agent cathedral nobody can reason about.

Synthesized from the July–August 2026 wave that named the pattern (Peter Steinberger's "are we still talking loops?", Gao Dalie, Aishwarya Srinivasan, Louis Bouchard, AI Builder Club), the enterprise-hardening writeups (TrueFoundry, Analytics Vidhya, puppyone), the counterweight literature (Cognition's Don't Build Multi-Agents, LangChain's How and when to build multi-agent systems, the MAST failure taxonomy, Towards a Science of Scaling Agent Systems), and the durable-execution debate (Diagrid, LangGraph docs). Full source list at the end.

πŸ“‹ Table of Contents

⚑ TL;DR

Graph engineering is the practice of making your agent system's topology an explicit, versioned artifact instead of an emergent property of whatever code you happened to write.

Nodes do work. Edges declare the permitted transitions. State carries what the next node needs. That's it β€” the rest is craft.

The one-line difference from loop engineering:

In a loop, you set the goal and the bar, and the agent picks its own route to clear it.
In a graph, you declare the valid routes and the checks along them.

Five things to take away before you read anything else:

  1. A graph is what you get when one loop is no longer enough. A loop is a one-node graph with an edge back to itself. The move is additive: keep the loop, make it a node, split off the step that keeps failing.
  2. Most tasks are still one job with one verifier β€” that's a loop. Reach for a graph only when the work has genuine branches, genuine parallelism, genuine specialists, or gates where consequence concentrates.
  3. You are designing four graphs, not one: what runs (execution), what each node sees (context), what each node is allowed to do (authority), and what actually ran (the runtime work graph). Teams draw the first and get paged about the other three.
  4. Nothing shipped in July 2026 that you couldn't build in 2025. LangGraph, Google ADK, and AutoGen were doing this before the word existed. What changed is that the nodes now interpret their tasks instead of following fixed rules β€” so state, vetoes, and budgets have to be explicit in a way Airflow never needed.
  5. Agents checking agents produces extremely organized nonsense. Somewhere in your graph, evidence has to come from outside the model: a test that actually ran, a schema that actually validated, a human who actually clicked approve.

If you remember one sentence: an edge is a promise about what crosses it. Most graph failures are edges nobody wrote down.

1. 🧭 What graph engineering actually is

Around July 18–19, 2026, Peter Steinberger asked, more or less in passing, "Are we still talking loops or did we shift to graphs yet?" β€” and within a fortnight "graph engineering" was the term of art. Nothing new shipped that week. What happened is that a lot of teams simultaneously admitted they had outgrown the single loop and had been quietly building topologies without naming them.

Here's the honest definition, stripped of hype:

Graph engineering is designing the topology of an AI system as an explicit artifact β€” which nodes exist (agents, deterministic functions, routers, joins, validators, human checkpoints), which transitions between them are permitted, what data crosses each transition, and how the runtime graph is allowed to mutate while it runs.

Contrast that with what most teams actually have in production today: a while loop, a 600-line prompt, and a growing thicket of if statements around tool results. That thicket is a graph. It's just an implicit one β€” undrawn, unversioned, untestable, and impossible to hand to a new engineer.

1.1 The concrete symptom that sends you here

You don't adopt graph engineering because you read a blog post. You adopt it because you hit one of these:

Symptom What it actually means
"The agent does step 4 before step 3 about 10% of the time." Ordering is implicit in a prompt instead of explicit in an edge.
"It re-runs the expensive search when it comes back from a retry." No checkpoint boundary; the loop has no memory of phase completion.
"The reviewer agent always approves its own work." Reviewer shares context with the author. Fresh-context boundary missing.
"One bad tool result poisons the rest of the run." No error edge β€” failures fall through into the happy path.
"Nobody can tell me what this run cost, per step." Cost is attributed to "the agent," not to nodes.
"We added a fifth if to the router prompt and two others broke." Routing logic lives in natural language instead of code.
"Two subagents built incompatible halves of the same feature." Parallel writes with no shared decision record.

Every one of those is a topology problem wearing a prompt-engineering costume.

1.2 The value proposition, precisely

Graph engineering buys you exactly four things. It's worth knowing them so you can tell when you're paying for something you're not getting:

  1. Legibility. You can look at the graph and enumerate every path the system can take. In a nested-conditional loop, you cannot.
  2. Isolation. A node runs with fresh context and a narrow mandate, so a mistake in node A doesn't silently corrupt node D.
  3. Enforceability. Hard constraints live in routing code, not in prompt text the model can talk itself out of. if revision_count >= 3: escalate is a promise; "please don't loop more than three times" is a suggestion.
  4. Attributability. Cost, latency, failure, and approval all get a node identifier, so you can debug and bill against structure instead of vibes.

If a proposed graph doesn't buy you at least two of those, you're adding complexity for the aesthetic.

2. 🚫 What it is not (the knowledge-graph confusion)

This trips up roughly half the people who hear the term, and it's worth being blunt about it.

Knowledge graphs / GraphRAG model your data: entities and the relationships between them, so retrieval can traverse Customer β†’ Order β†’ Refund instead of hoping cosine similarity finds it.

Graph engineering (2026 sense) models your execution: which node runs next, what state it receives, and how control flows.

Knowledge graph Graph engineering
Models What the system knows What the system does
Nodes are Entities (people, orders, docs) Steps (agents, functions, gates)
Edges are Relationships (works_at, cites) Permitted transitions (on_reject β†’ revise)
Lives in Neo4j, Neptune, a vector+graph store LangGraph, ADK, Agent Framework, your own runtime
Changes when Your domain data changes Your workflow design changes
Failure looks like Wrong answer, missing context Wrong path, stuck run, runaway spend

They compose happily. A retrieval node inside your execution graph can query a knowledge graph. Just don't let a vendor sell you one when you asked for the other β€” it happens constantly, because the words are identical.

There's a third meaning worth naming so you can dismiss it: dependency graphs / DAG schedulers (Airflow, Dagster, Argo). Structurally these are the closest prior art β€” a decade of it. The distinction isn't the shape. It's that in Airflow, a node executes a fixed rule; in an agent graph, a node interprets its task. That single change is why state, vetoes, budgets, and stop conditions have to become explicit artifacts. Airflow never needed a spend ceiling because a Python operator can't decide to call GPT eleven more times.

3. πŸͺœ The five-layer stack: prompt β†’ context β†’ harness β†’ loop β†’ graph

The layers compose; they don't replace each other. Every "X engineering is dead, long live Y engineering" post gets this wrong.

flowchart TB
    subgraph L5 [" "]
        GRAPH["πŸ•ΈοΈ <b>Graph</b> β€” topology across many nodes<br/>routing Β· parallelism Β· gates Β· handoffs"]
    end
    subgraph L4 [" "]
        LOOP["πŸ”„ <b>Loop</b> β€” one agent's observeβ†’actβ†’check cycle<br/>stop conditions Β· verifier Β· budget"]
    end
    subgraph L3 [" "]
        HARNESS["πŸ”§ <b>Harness</b> β€” the world around the model<br/>tools Β· sandbox Β· memory Β· retries Β· logs"]
    end
    subgraph L2 [" "]
        CONTEXT["πŸ“¦ <b>Context</b> β€” what the model perceives<br/>retrieval Β· compaction Β· memory"]
    end
    subgraph L1 [" "]
        PROMPT["πŸ’¬ <b>Prompt</b> β€” a single request"]
    end

    GRAPH --> LOOP --> HARNESS --> CONTEXT --> PROMPT

    classDef top fill:#7c2d12,color:#fff,stroke:#431407,stroke-width:2px;
    classDef mid fill:#0e7490,color:#fff,stroke:#083344,stroke-width:2px;
    classDef low fill:#374151,color:#fff,stroke:#111,stroke-width:2px;
    class GRAPH top;
    class LOOP,HARNESS mid;
    class CONTEXT,PROMPT low;

The crucial and under-stated consequence:

Graph sophistication multiplies loop-engineering requirements.

Fan-out means every worker needs its own stop condition. Retries mean every node needs idempotency. Concurrency means your state needs merge semantics. Dynamic node spawning means your budget needs to be enforced at the tree level, not the call level.

A sloppy loop wrapped in a beautiful graph is five sloppy loops. Fix the loop first. If you haven't got a real verifier and a real stop condition for a single agent, a graph will amplify the problem, not contain it.

3.1 Where the harness fits

"Harness engineering" is the layer people skip and then blame the model for. It's everything outside the weights: tool definitions, the file system or sandbox, session storage, retry policy, timeouts, spend caps, approval hooks. Without a harness the model can't persist state, can't recover, and can't be stopped.

Rule of thumb for where to spend your next week:

  • Agents can't resume or lose state between runs β†’ harness.
  • Single-pass output is wrong and there's a deterministic check available β†’ loop.
  • Work genuinely splits into specialties, branches, or parallel tracks β†’ graph.

4. 🧬 The three primitives: state, nodes, edges

Every graph runtime worth using reduces to the same three things.

flowchart LR
    A["<b>Node A</b><br/>reads state<br/>does work<br/>returns update"] -->|"edge:<br/>what crosses?"| B["<b>Node B</b>"]
    B -->|"conditional edge:<br/>route(state) β†’ next"| C{"<b>Router</b>"}
    C -->|pass| D["<b>Node D</b>"]
    C -->|fail| A

    S[("<b>State</b><br/>typed Β· versioned Β· checkpointed")] -.reads/writes.- A
    S -.reads/writes.- B
    S -.reads/writes.- D

    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef r fill:#92400e,color:#fff,stroke:#451a03;
    classDef s fill:#374151,color:#fff,stroke:#111;
    class A,B,D n;
    class C r;
    class S s;

State β€” a typed, shared data structure that persists across the run and flows along edges. It is the single source of truth. Not a chat transcript. Not a global variable. A schema.

Node β€” a function that reads state, performs work (an LLM call, a tool invocation, a database query, a human prompt), and returns a partial state update. Nodes should be small enough to name in three words.

Edge β€” a routing instruction. Direct edges always fire. Conditional edges inspect state and decide. The set of edges is the specification of what your system can do.

Minimal skeleton, LangGraph-flavored (the API most other runtimes rhyme with):

from typing import Literal, TypedDict
from langgraph.graph import END, START, StateGraph

class ReviewState(TypedDict, total=False):
    diff: str
    findings: list[dict]
    verdict: Literal["pass", "fail"]
    revision_count: int

def analyze(state: ReviewState) -> ReviewState:
    return {"findings": run_analysis(state["diff"])}

def judge(state: ReviewState) -> ReviewState:
    blocking = [f for f in state["findings"] if f["severity"] == "high"]
    return {"verdict": "fail" if blocking else "pass"}

def route(state: ReviewState) -> Literal["revise", "done"]:
    if state["verdict"] == "pass":
        return "done"
    if state.get("revision_count", 0) >= 3:      # hard stop lives in CODE
        return "done"
    return "revise"

builder = StateGraph(ReviewState)
builder.add_node("analyze", analyze)
builder.add_node("judge", judge)
builder.add_node("revise", revise)
builder.add_edge(START, "analyze")
builder.add_edge("analyze", "judge")
builder.add_conditional_edges("judge", route, {"revise": "revise", "done": END})
builder.add_edge("revise", "analyze")

graph = builder.compile(checkpointer=checkpointer)
Enter fullscreen mode Exit fullscreen mode

Three things to notice, because they generalize to every framework:

  1. The stop condition is in Python, not in a prompt. revision_count >= 3 cannot be argued with.
  2. Nodes return partial updates, not whole state. This is what makes parallel merges possible.
  3. The graph is a value you can print, diff, and test before a single token is spent.

5. πŸ—ΊοΈ The four graphs you are actually designing

This is the section I'd keep if I could keep only one. Almost every serious production incident I've seen written up in 2026 comes from a team that drew one graph and assumed the other three were the same shape.

flowchart TB
    subgraph G1 ["1️⃣ Execution graph β€” what runs"]
        E1[Planner] --> E2[Researcher] --> E3[Writer] --> E4[Reviewer]
    end
    subgraph G2 ["2️⃣ Context graph β€” what each node sees"]
        C1[Planner] -.->|"topic + constraints"| C2[Researcher]
        C2 -.->|"notes only<br/>NOT transcript"| C3[Writer]
        C1 -.->|"original spec"| C4[Reviewer]
        C3 -.->|"draft only"| C4
    end
    subgraph G3 ["3️⃣ Authority graph β€” what each may do"]
        A1["Planner<br/>πŸ”’ no tools"] 
        A2["Researcher<br/>🌐 read-only web + docs"]
        A3["Writer<br/>πŸ“ write to draft store"]
        A4["Reviewer<br/>🚫 read-only + veto"]
    end
    subgraph G4 ["4️⃣ Work graph β€” what actually ran"]
        W1[Planner] --> W2[Researcher]
        W2 --> W2b[Researcher retry Γ—2]
        W2b --> W3[Writer]
        W3 --> W4[Reviewer]
        W4 -->|reject| W3
    end

    classDef ex fill:#0e7490,color:#fff,stroke:#083344;
    classDef ct fill:#4c1d95,color:#fff,stroke:#2e1065;
    classDef au fill:#7c2d12,color:#fff,stroke:#431407;
    classDef wk fill:#374151,color:#fff,stroke:#111;
    class E1,E2,E3,E4 ex;
    class C1,C2,C3,C4 ct;
    class A1,A2,A3,A4 au;
    class W1,W2,W2b,W3,W4 wk;

5.1 The execution graph β€” what runs, in what order

This is the one everyone draws. Nodes, edges, routing, parallelism, gates. It answers: what happens next?

5.2 The context graph β€” what each node can see

The most under-designed of the four, and the source of the most surprising bugs.

Execution order does not automatically define information visibility.

A node running after another does not have to receive that predecessor's transcript. Most frameworks default to "everything is in shared state, everyone reads everything," and that default is wrong in two directions at once:

  • Too much visibility β†’ your "independent reviewer" reads the author's reasoning, gets anchored, and rubber-stamps. Your context window blows up. A credential fetched in step 2 is visible in step 9. One poisoned tool result contaminates every downstream node.
  • Too little visibility β†’ the classic Cognition failure: one subagent builds a Flappy Bird clone with a Super Mario background while another builds an incompatible sprite, because neither saw the other's implicit decisions.

The design question to ask at every single edge:

Which exact fields, messages, and artifacts must cross this edge β€” and which must not?

Write the answer down. In code. As a projection function:

def context_for_reviewer(state: ReviewState) -> dict:
    """The reviewer sees the artifact and the spec. Never the author's reasoning."""
    return {
        "spec": state["spec"],
        "draft": state["draft"],
        # deliberately excluded: writer_scratchpad, tool_transcripts, credentials
    }
Enter fullscreen mode Exit fullscreen mode

Two heuristics that resolve the tension above:

  • For verification edges β†’ starve the context. A reviewer that shares the author's context is not a reviewer, it's a co-author. Fresh context, different model where you can afford it.
  • For handoff edges β†’ share the decisions, not the transcript. Pass a structured digest of what was decided and why, not 40k tokens of raw messages. Full traces are what Cognition argues for; in practice a compressed decision log gets you most of the coherence at a fraction of the cost β€” and it's the only thing that scales past a few hops.

5.3 The authority graph β€” what each node may do

Independent of both order and visibility: what tools, credentials, and side effects is this node permitted?

The rule that does the most work here:

The node that retrieves data should never be the node that writes to production.

Retrieval nodes ingest untrusted content. Writer nodes hold dangerous permissions. Keeping those in the same node means any indirect prompt injection in a fetched document inherits your write credentials. Split them, and an injection is contained to a node that can only read.

Practical shape:

Node Tools Credentials Side effects
researcher web.search, docs.read read-only token none
planner none none none
writer draft.write scoped to draft bucket reversible
deployer deploy.run prod token irreversible β†’ human gate

5.4 The work graph β€” what actually ran

Your execution graph is the intended topology. The work graph is the actual one for a given run: this run retried the researcher twice, spawned four workers instead of the usual two, took the escalation edge, and skipped the reviewer because the router short-circuited.

The orchestrator must record the runtime work graph, not just the intended one. Without it:

  • Your cost dashboard says "graph execution: $84" and you cannot find the node that burned it.
  • Your postmortem says "the graph did it," which is an audit black hole.
  • Your A/B test between two topologies compares two things you never actually measured.

The frontier problem in enterprise graph engineering right now is exactly this: letting the runtime work graph mutate (spawn workers, take recovery paths) while a stable, versioned org graph holds the policy constant.

6. 🧩 Node taxonomy β€” and the one rule that matters

Seven kinds of node cover essentially everything in production:

# Node type Does Non-negotiables
1 Agent node Full tool-using loop with its own stop condition Own budget, own max-iterations, own verifier
2 LLM call Single model invocation, structured output Schema-validated output, no tools
3 Deterministic function SQL, HTTP, parsing, math, business rules No model. Testable with normal unit tests.
4 Router Reads state, returns the next node's name Prefer code; use a model only for semantic routing
5 Validator / gate Tests, schema checks, policy checks Must be able to fail the run, not just annotate it
6 Human checkpoint Pauses and waits for a person Produces an approval record; survives process restart
7 Subgraph A whole graph as one node Own state schema; explicit input/output projection

6.1 The rule

Use an LLM only where the ambiguity lives. Everything else is a function.

Known business rules stay deterministic. If you can express it as if amount > 10_000: require_approval(), do not ask a model to reason about it β€” you're paying tokens for a coin flip on something you already know the answer to.

The reverse failure is just as common: teams write a 400-line regex router to classify user intent when a small model does it better and cheaper. The line is semantic interpretation, generation, planning, and genuine ambiguity on one side; everything with a knowable answer on the other.

A useful audit: go through your graph, and for each LLM node ask "what is the ambiguity this node resolves?" If you can't answer in one sentence, it should probably be a function β€” or shouldn't exist.

6.2 Node contracts

Every node needs a written contract before it needs an implementation. Six fields:

node: security_reviewer
inputs:                       # exact state fields read
  - diff
  - changed_files
outputs:                      # exact state fields written
  - security_findings
tools: [ "sast.scan", "deps.audit" ]   # nothing else is reachable
timeout: 120s
retry: { max: 2, on: [transient_http, rate_limit], backoff: exponential }
side_effects: none            # or: describe them, and whether they're idempotent
budget: { max_tokens: 60000, max_usd: 0.40 }
Enter fullscreen mode Exit fullscreen mode

If you write these six fields for every node, you have already prevented most of the incident classes in Β§13 and Β§21. The YAML doesn't need to be real config β€” a docstring is fine. What matters is that somebody decided.

Why outputs has to be a schema, not a paragraph. Watch what happens when a node's output is free-form text instead of a typed field:

research_node  β†’  "I think setting up Stripe subscriptions this way should probably work."
writer_node    β†’  reads it as settled fact, implements it
reviewer_node  β†’  reads the writer's confident code, approves it
Enter fullscreen mode Exit fullscreen mode

Three edges, zero verification, and a hedge ("I think... probably") has silently become ground truth by the time it reaches the reviewer β€” who has no way to tell a confirmed finding from a guess, because both arrive as the same shape of prose. That's an industrial-scale hallucination factory, and it's built entirely out of edges nobody constrained.

The fix is the outputs field, taken seriously β€” force uncertainty into the schema instead of letting it hide in tone:

{
  "findings": ["Stripe subscriptions support monthly + annual plans natively"],
  "evidence": ["docs.stripe.com/billing/subscriptions#plans"],
  "unknowns": ["webhook retry behavior on failed renewal β€” not yet confirmed"],
  "confidence": 0.82
}
Enter fullscreen mode Exit fullscreen mode

Now a downstream node β€” or a router β€” can act on unknowns and confidence instead of inferring shakiness from word choice. A reviewer approves because evidence and confidence clear a bar, not because the prose sounded sure of itself. This is the node-contract discipline from Β§6.2 applied specifically to the failure mode in Β§16: an anchor only works if the thing it's checking is structured enough to check.

6.3 Naming nodes

Name nodes for specialties, not for personas. security_reviewer is a specialty. Alice, the Senior Security Architect Agent is cosplay that costs you 200 tokens per call and encourages the model to perform a role instead of doing a job.

A node earns a name when it has: a distinct input projection, a distinct tool set, and a distinct success criterion. Two "agents" with the same tools and the same context are one node with a loop.

7. ➑️ Edge taxonomy and the edge contract

Edge type Fires when Typical use
Direct Always Fixed pipeline stages
Conditional Router function returns a name Pass/fail, classify-and-dispatch
Fan-out One node β†’ N nodes in parallel Independent subtasks, multi-source research
Fan-in / join N nodes β†’ one aggregator Merge findings, vote, synthesize
Loop-back Verifier rejects Bounded revision cycles
Error Node raises or times out Retry, fallback model, escalate, dead-letter
Human Policy demands approval Irreversible or high-consequence actions
Event / interrupt External signal arrives Webhooks, cancellation, new information mid-run

7.1 The edge contract

Every edge in a production graph should have three answers written down. This is the single highest-leverage habit in graph engineering:

EDGE: researcher ──▢ writer

1. DATA:      what crosses?      β†’ notes[], sources[]        (NOT: raw transcript, api keys)
2. AUTHORITY: what carries over? β†’ nothing; writer has its own scoped token
3. FAILURE:   if downstream fails, what happens?
              β†’ retry writer Γ—2 β†’ on repeat failure route to `escalate`,
                 do NOT re-run researcher (expensive, already checkpointed)
Enter fullscreen mode Exit fullscreen mode

Most graph bugs are edges where nobody answered #3. The happy path gets designed lovingly and the failure path gets except: pass.

7.2 Error edges deserve first-class design

Classify failures, because they want different edges:

Failure class Example Right edge
Transient 429, 503, timeout Retry with backoff, same node
Capability Model can't do it, output fails schema 3Γ— Fallback edge β†’ bigger model or different tool
Input Missing field, malformed upstream output Route back to the producing node, not the consumer
Policy Guardrail blocked, budget exceeded Halt edge β†’ human, with the reason attached
Unknown Unhandled exception Dead-letter node that preserves state for replay

A graph with one generic retry policy for all five is a graph that retries prompt-injection attempts and gives up on rate limits.

8. πŸ—ƒοΈ State design: the part everyone gets wrong

If nodes and edges are the skeleton, state is the bloodstream β€” and the most common cause of a graph that "works in the demo and dies in production" is a state object that grew into a garbage bag.

8.1 Four state layers, four owners

Do not collapse these into one blob called state:

Layer Holds Lifetime Who owns it
Control-flow state Current node, routing flags, counters, checkpoints The run The graph runtime
Model context Messages/prompts a given node sees One node invocation The context projection (Β§5.2)
Durable business context Artifacts, decisions, approvals, IDs Beyond the run Your database
Knowledge / memory Entities, relations, long-term facts Indefinite Your KG / vector store

Collapsing them is how you end up with a 300KB state object that gets serialized to Postgres on every step, a reviewer that can read the deploy token, and a "memory" that is really just an unbounded message list.

8.2 Rules for the state schema

Type it. TypedDict or a Pydantic model. Untyped dict state means every node is a guess.

Store references, not blobs. Anything over a few kilobytes β€” documents, query results, images, full transcripts β€” goes to S3/Postgres/a vector store, and state carries the ID.

# βœ— bad: 40k tokens of PDF in state, checkpointed on every step
{"document": "<entire contract text>"}

# βœ“ good
{"document_ref": "s3://contracts/8f31.pdf", "document_summary": "...", "clause_ids": [...]}
Enter fullscreen mode Exit fullscreen mode

Write reducers for anything parallel. When three nodes write to the same key concurrently, "last write wins" silently destroys two thirds of your work.

from operator import add
from typing import Annotated

class ResearchState(TypedDict, total=False):
    topic: str
    findings: Annotated[list[dict], add]   # concurrent appends merge
    verdict: str                            # single-writer; no reducer needed
Enter fullscreen mode Exit fullscreen mode

For non-list merges, write the merge function explicitly and decide the conflict policy β€” union, max-severity-wins, first-writer-wins β€” rather than inheriting whatever the framework does by default.

Make it inspectable. You should be able to dump state at any checkpoint and read it as a story of the run. If you can't tell what happened from the state object, neither can your on-call engineer at 3am.

Version it. State schemas change. Old checkpoints exist. Put a schema_version field in from day one and write the migration when you bump it, or every deploy orphans every in-flight run.

8.3 What does not belong in state

  • Credentials and tokens (fetch them at the node from a secret store, scoped)
  • Raw model transcripts (summarize, or store by reference)
  • Anything you wouldn't want a compromised node to read
  • Derived values you can recompute cheaply

9. πŸ“š The pattern library (10 shapes that cover ~95% of real work)

Each pattern below has the same five fields: shape Β· use when Β· fails when Β· cost profile Β· the detail people miss. Compose them; real graphs are three or four of these stacked.

9.1 πŸ”— Pipeline (prompt chaining with gates) 

flowchart LR
    S([start]) --> A[Extract] --> G1{gate:<br/>schema ok?}
    G1 -->|no| F([fail fast])
    G1 -->|yes| B[Transform] --> G2{gate:<br/>rules ok?}
    G2 -->|no| F
    G2 -->|yes| C[Render] --> E([done])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    class A,B,C n;
    class G1,G2 g;

Use when the task decomposes cleanly into fixed sequential subtasks and you'd rather trade latency for accuracy. Outline β†’ draft. Extract β†’ validate β†’ load. Translate β†’ back-translate β†’ compare.

Fails when the steps aren't actually independent, so step 3 needs information step 1 threw away. Symptom: you keep widening the state object to smuggle context forward.

Cost: lowest of any multi-node pattern. Linear in stages. Latency is the sum, not the max.

The detail people miss: the gates are the pattern, not the chain. A chain without programmatic checks between stages is just a long prompt with extra API calls and worse latency. Every stage boundary is a free place to fail fast and save the rest of the run.

9.2 🚦 Router (classify and dispatch) 

flowchart TD
    IN([request]) --> R{"Router<br/>classify"}
    R -->|refund| A[Refund flow<br/>small model]
    R -->|technical| B[Technical flow<br/>large model + tools]
    R -->|abuse| C[Escalate to human]
    R -->|unknown| D[Clarify + re-route]
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef r fill:#92400e,color:#fff,stroke:#451a03;
    class A,B,C,D n;
    class R r;

Use when inputs fall into distinct categories that deserve different prompts, tools, or models β€” and handling them all in one prompt makes every branch worse.

Fails when categories overlap, or the taxonomy drifts. The router becomes the bottleneck and every misroute is invisible because nobody logs the rejected branches.

Cost: one cheap classification call buys you large savings downstream β€” routing simple queries to a small model is the highest-ROI cost lever in most agent systems.

The detail people miss: always ship a default/unknown branch, and log routing decisions with confidence. Your router's misclassification rate is a first-class metric (Β§22). Also: prefer deterministic routing whenever the signal exists in structured data β€” if user.plan == "enterprise" beats asking a model to infer it.

9.3 🌿 Fan-out / fan-in (sectioning, map-reduce) 

flowchart TD
    P[Planner<br/>split into N sections] --> W1[Worker 1]
    P --> W2[Worker 2]
    P --> W3[Worker 3]
    W1 --> J["Join / reduce<br/>merge + dedupe + rank"]
    W2 --> J
    W3 --> J
    J --> V{Verifier}
    V -->|gaps| P
    V -->|ok| OUT([result])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    class P,W1,W2,W3,J n;
    class V g;

Use when subtasks are genuinely independent and read-only. Multi-source research, scanning 40 files for a pattern, running five different linters, screening one input against several guardrails at once.

Fails when the subtasks write. Parallel writers make conflicting implicit decisions and you get two incompatible halves of a feature. This is the single sharpest rule in multi-agent design: parallelize reads, serialize writes.

Cost: latency β‰ˆ the slowest branch; tokens β‰ˆ the sum of all branches, plus the join. Fan-out is where budgets die β€” an uncapped fan-out inside a retry loop is the agent equivalent of a fork bomb.

The detail people miss: the join node is where the real engineering is. Merging N findings means dedupe, conflict resolution, and ranking. Teams spend a week on workers and ten minutes on the reducer, then wonder why output quality dropped. Also: cap N in code, and make each worker's task specific. "Research the semiconductor shortage" farmed out to four subagents produces four overlapping essays; "find 2026 fab capacity numbers for TSMC, with sources" produces an answer.

9.4 πŸ‘” Orchestrator–worker (supervisor) 

flowchart TD
    O{{"Supervisor<br/>decompose Β· assign Β· synthesize"}}
    O -->|task 1| W1["Worker: search"]
    O -->|task 2| W2["Worker: code"]
    O -->|task 3| W3["Worker: test"]
    W1 -->|summary| O
    W2 -->|summary| O
    W3 -->|summary| O
    O --> D{"done?<br/>budget left?"}
    D -->|no| O
    D -->|yes| OUT([deliver])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef s fill:#1f2937,color:#fff,stroke:#111;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    class W1,W2,W3 n;
    class O s;
    class D g;

Use when you can't predict the subtasks in advance. This is the difference from Β§9.3: fan-out splits a known list; the supervisor decides the list at runtime.

This is the 2026 production default for anything non-trivial, and for good reason: clear accountability, one place to debug, predictable-ish cost, and it maps onto how people already work.

Fails when the supervisor's context saturates. Empirically that happens somewhere around 8–12 worker cycles if you replay everything. It's also a single point of failure and an over-centralization bottleneck at high volume.

Cost: ~20–40% more tokens per run than a flat swarm, but usually cheaper overall because it eliminates duplicate work β€” one reported reduction was ~30% average token consumption once supervision was added.

The detail people miss β€” three context rules that make or break it:

  1. Don't share a system prompt between supervisor and workers. It conflates roles and you pay the supervisor's prompt cost on every worker call.
  2. Workers return a structured summary, not their transcript. A worker's job is to compress its own exploration into a decision.
  3. Don't replay the full history on every supervisor wakeup. Compress older turns into a structured digest with a cheap model; keep a sliding window of full-fidelity messages.

This shape β€” one main loop carries state, subagents are stateless workers with narrow scope β€” is what Claude Code's Task tool, OpenAI's agents-as-tools, and Anthropic's research system all converged on independently. That convergence is the strongest signal in the field.

9.5 βš–οΈ Evaluator–optimizer (maker–checker) 

flowchart LR
    G[Generator] --> C{"Critic<br/>fresh context<br/>different model"}
    C -->|"reject + reasons"| G
    C -->|approve| OUT([ship])
    C -.->|"revisions β‰₯ 3"| H([escalate to human])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    class G n;
    class C g;

Use when you have clear evaluation criteria and iteration measurably improves the output: translation, code against a test suite, copy against a style guide, extraction against a schema.

Fails when the critic shares the generator's context (rubber-stamping), when the criteria are vague ("make it better"), or when there's no revision cap and the pair ping-pongs forever.

Cost: 2–4Γ— a single generation. Worth it when the failure is expensive; wasteful when a linter would have caught it.

The detail people miss: rank your checkers. A deterministic checker (tests, schema, linter, compiler) beats a model checker every single time and costs ~nothing. Use the model critic only for what tests can't express β€” tone, completeness, whether the answer is actually responsive to the question. And always cap revisions in code with an escalation edge, because two models disagreeing politely is an infinite loop with a credit card attached.

9.6 🀝 Handoff / swarm 

flowchart LR
    A["Triage agent"] -->|handoff| B["Billing agent"]
    B -->|handoff| C["Refund agent"]
    C -->|handoff| D["Notify agent"]
    B -.->|handoff back| A
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    class A,B,C,D n;

Use when each agent can locally tell that someone else is better suited, and the domains are cleanly separated: customer-support triage, read-heavy exploration, high-volume routing where the logic is self-evident at each hop.

Fails when the chain gets long. Drift compounds across 8–10 sequential handoffs, there's no central registry so agents duplicate each other's work, and a cascading failure has nothing to stop it. Handoff swarms also measured worse on multi-domain tasks than the subagent pattern: 7+ API calls and 14,000+ tokens vs. ~5 calls and ~9,000 tokens.

Cost: cheaper per hop than supervision, more expensive in aggregate once you exceed a handful of hops.

The detail people miss: a handoff is an edge with a payload contract, not a vibe. Define exactly which context variables cross, and log every handoff with a reason. Undocumented handoffs are how you get circular delegation β€” agent A hands to B hands to A β€” which shows up in the MAST data as one of the most common coordination failures.

9.7 πŸ™‹ Human checkpoint (approval gate) 

flowchart LR
    P[Prepare action] --> R{"Risk<br/>classifier"}
    R -->|low| X[Execute]
    R -->|high| H(["⏸️ interrupt<br/>human approves"])
    H -->|approve| X
    H -->|reject + notes| P
    X --> OUT([done])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    classDef h fill:#4c1d95,color:#fff,stroke:#2e1065;
    class P,X n;
    class R g;
    class H h;

Use when the next action is irreversible, expensive, externally visible, or regulated: sending email, moving money, deploying, deleting, filing.

Fails when it's everywhere (approval fatigue β€” humans start rubber-stamping, which is worse than no gate because it launders responsibility) or nowhere.

Cost: zero tokens, enormous latency. Design for hours of wall-clock pause.

The detail people miss: the gate must be structural, not advisory. "The agent asks for permission in its prompt" is not a gate; the model can talk itself past it. A real gate is a node that suspends the run, persists state, and cannot proceed without an external resume β€” and it produces an approval record with who, when, and what exact payload was approved.

from langgraph.types import interrupt, Command

def approval_gate(state):
    decision = interrupt({
        "action": state["pending_action"],
        "diff": state["diff_preview"],
        "estimated_blast_radius": state["impact"],
        "allowed": ["approve", "reject", "modify"],
    })
    return {"approved": decision["action"] == "approve",
            "approver": decision["actor"], "approved_at": now()}

# resumed later, possibly days later, from a durable checkpoint:
graph.invoke(Command(resume={"action": "approve", "actor": "truong@…"}), config)
Enter fullscreen mode Exit fullscreen mode

Position gates where consequence concentrates, not where uncertainty concentrates. Those are different places, and the mistake is gating the uncertain-but-harmless step while the irreversible one runs unattended.

9.8 πŸͺœ Fallback ladder (escalation) 

flowchart TD
    T[Attempt: cheap model] --> C1{ok?}
    C1 -->|yes| OUT([done])
    C1 -->|no| T2[Attempt: large model]
    T2 --> C2{ok?}
    C2 -->|yes| OUT
    C2 -->|no| T3[Attempt: model + extra tools]
    T3 --> C3{ok?}
    C3 -->|yes| OUT
    C3 -->|no| H([human / dead-letter])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    class T,T2,T3 n;
    class C1,C2,C3 g;

Use when most instances are easy and a minority are hard. This is the cost-control pattern: pay small-model prices for the 80%, large-model prices only for the tail.

Fails when the check between rungs is weak, so you escalate on noise β€” or worse, don't escalate on real failures and ship the cheap model's wrong answer.

Cost: dramatically lower average cost, higher p99 latency. Know which one your product sells.

The detail people miss: measure the escalation rate per rung and alert on drift. A ladder whose first rung suddenly stops succeeding is your earliest warning that a model changed, a prompt regressed, or your input distribution shifted.

9.9 πŸ—³οΈ Vote / debate ensemble 

flowchart TD
    IN([input]) --> A[Judge A]
    IN --> B[Judge B]
    IN --> C[Judge C]
    A --> V{"Aggregate<br/>majority / any-veto"}
    B --> V
    C --> V
    V --> OUT([verdict])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    class A,B,C n;
    class V g;

Use when you need higher confidence on a classification and the cost of a wrong answer is high: safety screening, security review, fraud triage. Any-veto aggregation (one flag blocks) is the useful variant.

Fails when the judges share a model and a context β€” then you're not sampling independent opinions, you're sampling the same opinion three times. Models systematically agree with each other and prefer their own outputs. Debate patterns in particular are great in research papers and expensive circular disagreement in production.

Cost: NΓ— per decision, plus aggregation. Only defensible for high-stakes, low-volume decisions.

The detail people miss: diversity is the whole product. Different models, different prompts, or β€” best β€” one model judge plus one deterministic checker. Three calls to the same model with temperature=1 is a confidence theater, not an ensemble.

9.10 πŸ“¦ Subgraph composition (team of teams) 

flowchart TD
    ROOT{{"Root supervisor"}} --> SG1["<b>Subgraph: research</b><br/>plan β†’ fan-out β†’ join"]
    ROOT --> SG2["<b>Subgraph: implement</b><br/>write β†’ test β†’ fix"]
    SG1 --> ROOT
    SG2 --> ROOT
    ROOT --> GATE(["human gate"]) --> SHIP([ship])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef s fill:#1f2937,color:#fff,stroke:#111;
    classDef h fill:#4c1d95,color:#fff,stroke:#2e1065;
    class SG1,SG2 n;
    class ROOT s;
    class GATE h;

Use when the graph exceeds ~10 nodes and clusters into coherent phases. Subgraphs are how you keep a big graph readable and independently testable.

Fails when the subgraph's state schema leaks into the parent's, or the parent passes its whole state down. Then you have one big graph with extra indirection.

Cost: structural, not token cost. The win is testability β€” a subgraph is the unit you can evaluate in isolation.

The detail people miss: subgraphs are also your checkpoint granularity control. Compile the noisy interior subgraph without a checkpointer and the outer graph with one, so you persist at phase boundaries (search β†’ synthesize β†’ review) instead of on every micro-step. Checkpointing everything is the most common cause of a slow, expensive graph that spends more time serializing than thinking.

9.11 Pattern selection table

Pattern Reach for it when Token cost Latency Complexity Danger
Pipeline Fixed, decomposable stages 🟒 Low Sum of stages 🟒 Low Gates omitted
Router Distinct input categories 🟒 Low (saves more) +1 hop 🟒 Low No default branch
Fan-out/in Independent reads πŸ”΄ High (Ξ£ branches) Max of branches 🟑 Med Uncapped N; parallel writes
Orchestrator–worker Subtasks unknown up front 🟑 Med-High Serial-ish 🟑 Med Supervisor context saturation
Evaluator–optimizer Clear criteria + iteration helps 🟑 2–4Γ— 2–4Γ— 🟒 Low Shared context; no revision cap
Handoff/swarm Clean domain separation, few hops 🟑 Med Serial 🟑 Med Drift past ~8 hops; loops
Human gate Irreversible / regulated action 🟒 Zero ⏰ Hours 🟒 Low Advisory instead of structural
Fallback ladder Long tail of hard cases 🟒 Low avg High p99 🟒 Low Weak inter-rung checks
Vote / debate High-stakes classification πŸ”΄ NΓ— Max of judges 🟑 Med Correlated judges
Subgraph >10 nodes, clear phases βž– Neutral βž– πŸ”΄ High State leakage

10. πŸ§ͺ The "keep it a loop" test β€” when not to build a graph

The most valuable thing in this guide might be the permission to not do it.

The evidence is unkind to enthusiastic multi-agent architecture:

  • Princeton NLP found a single agent matched or outperformed multi-agent systems on 64% of benchmarked tasks given the same tools and context β€” multi-agent added ~2.1 percentage points of accuracy at roughly double the cost.
  • The 2026 scaling study across 260 configurations found gains ranging from +80.8% on decomposable financial reasoning to βˆ’70.0% on sequential planning. Architecture–task alignment, not architecture sophistication, determines the outcome.
  • Multi-agent systems consume roughly 15Γ— the tokens of a chat interaction.
  • Tool-heavy tasks appear to incur multi-agent overhead β€” the coordination cost exceeds the parallelism benefit.
  • Coordination gains saturate once the single-agent baseline is already strong.

10.1 The five-question test

Run this before you draw a single box. If you answer "no" to all five, keep the loop.

  1. Does the work genuinely branch? Not "sometimes it's different" β€” are there distinct paths with distinct tools or distinct success criteria?
  2. Is there real parallelism, and is it read-only? Independent reads parallelize. Writes don't.
  3. Do you need a check by something that must not see the author's reasoning? That's a fresh-context boundary a loop cannot give you.
  4. Is there a point where consequence concentrates and a human or hard policy must intervene? That's a structural gate.
  5. Do different steps need genuinely different models, tools, or permission scopes? Cost, capability, or blast-radius reasons β€” not aesthetic ones.

One "yes" usually means: split off exactly that one thing. Three or more: you have a graph.

10.2 The diagnostic that saves the most money

When something is failing, ask whether the failure is architectural or basic.

Most of the time it's basic β€” the task was too broad, the state was invisible, the tool descriptions were bad, the verifier was fake. Adding a swarm topology on top of invisible state doesn't fix invisible state; it distributes it across more agents. Adding an LLM-as-judge on top of a mega-prompt doesn't fix the mega-prompt.

Teams reach for architectural novelty because it feels like progress. Fix the boring thing first:

You're tempted to add Try first
A reviewer agent A test, a schema, a linter
A supervisor A narrower task and a real stop condition
A swarm One agent with better tool descriptions
A debate ensemble One good rubric and one judge
More nodes Deleting the node that never fires

10.3 And the meta-warning

Do not respond to a meme by building a forty-agent graph that runs overnight.

Start with one recurring task, a real verifier, inspectable state, and hard stops. Expand only when the workload actually pushes back.

11. πŸ”€ How to migrate from a loop to a graph (the safe path)

The move is additive, and doing it additively is what separates a two-day migration from a two-month rewrite.

flowchart LR
    S1["<b>Step 1</b><br/>Your loop<br/>= one node"] --> S2["<b>Step 2</b><br/>Split off the<br/>failing step"]
    S2 --> S3["<b>Step 3</b><br/>Add the gate<br/>you keep doing<br/>manually"]
    S3 --> S4["<b>Step 4</b><br/>Parallelize the<br/>read-only part"]
    S4 --> S5["<b>Step 5</b><br/>Extract subgraphs<br/>past 10 nodes"]
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    class S1,S2,S3,S4,S5 n;

Step 1 β€” Make the existing loop a node. Don't redesign it. Wrap it. You now have a one-node graph, a state schema, and a checkpointer. That alone gives you resumability and traces, which is often 60% of the value.

Step 2 β€” Split off the one step that keeps failing. Look at your failure log. There's one step that's responsible for most of it β€” usually because it needs different tools, a different model, or context the main loop poisoned. Make that a node with its own contract. Stop there for a week.

Step 3 β€” Turn the manual check into a gate. Whatever you personally look at before letting the run continue: that's a node. If it's mechanical, make it a validator. If it's judgment, make it a human checkpoint. This is where the audit trail starts.

Step 4 β€” Parallelize the read-only work. Only now. Cap the fan-out, write the reducer, set a per-branch budget.

Step 5 β€” Extract subgraphs. When you pass ~10 nodes and can name coherent phases, group them. Test each subgraph independently.

At every step the graph must stay runnable and better than the previous version. If a step makes things worse, you learned something cheap. A big-bang rewrite from loop to twenty-node graph teaches you nothing except that it doesn't work.

12. πŸ—οΈ Making it survive production: durability, idempotency, budgets

A graph that works on your laptop and a graph that survives a Kubernetes eviction at step 7 of 12 are different artifacts.

12.1 Checkpoints are not durable execution

This distinction matters more than any pattern in Β§9:

Checkpointing says: "I saved your state. You take it from here."
Durable execution says: "Your workflow will run to completion. Period."

Most agent frameworks give you the first and let you believe you have the second. Concretely, with checkpoint-only frameworks:

  • You must detect the failure. A crashed process stays dead.
  • You must find the thread ID and re-invoke.
  • You must handle concurrent recovery attempts β€” there's typically no distributed lock, so two recovery workers can duplicate side effects.
  • A tool exception raised inside an agent node can take down the whole run unless you catch it and return it as a value.

True durable execution β€” the Temporal/Restate/DBOS model β€” checkpoints at every await point automatically, replays completed steps from cache instantly, restarts workflows without human intervention, and rebalances across cluster nodes. Closing that gap in a checkpoint-based framework means building it yourself or putting one of those runtimes underneath.

Decision rule:

Your runs What you need
Seconds to a few minutes, retryable end-to-end Checkpointer is fine
Minutes to hours, expensive steps Checkpointer + your own supervisor/reaper process
Hours to days, human gates, real money A durable execution engine under the graph

12.2 Idempotency is not optional

Retries and replays mean every node with a side effect will eventually run twice. Design for it:

def send_notification(state):
    key = f"notify:{state['run_id']}:{state['node_id']}:{hash(state['message'])}"
    if store.seen(key):                 # dedupe on a stable idempotency key
        return {"notified": True}       # replay returns cached success
    provider.send(state["message"])
    store.mark(key, ttl=7 * 86400)
    return {"notified": True}
Enter fullscreen mode Exit fullscreen mode

Rules:

  • Every side-effecting node has an idempotency key derived from (run_id, node_id, payload hash).
  • Wrap non-determinism (clock, random, UUID, external reads) in tasks whose results are checkpointed, so replay reproduces the original run instead of a new one.
  • Prefer reversible actions before irreversible ones, so a failed run leaves a mess you can clean up rather than an email you can't unsend.

12.3 Budgets and hard stops, at three levels

An agent graph without ceilings is an unbounded liability. Enforce at all three:

Level Cap Enforced by
Node max tokens, max tool calls, max wall-clock Node contract
Branch/subtree total spend under a fan-out, max spawned children Orchestrator, before dispatch
Run total USD, total steps, total wall-clock, deadline Graph runtime

And four hard stops every graph needs, borrowed straight from loop engineering and then made per-node:

  1. Max iterations on every cycle in the graph. Every loop-back edge has a counter.
  2. No-progress detection β€” if state hasn't meaningfully changed in N steps, halt. Step repetition is one of the most common documented multi-agent failure modes (~16% of observed failures).
  3. Spend ceiling with the run killed, not warned.
  4. Wall-clock deadline, because "still running" is a failure mode with a monthly invoice.

12.4 Concurrency hygiene

  • Cap fan-out width in code, and cap the depth of dynamic spawning. A supervisor that can spawn supervisors needs an explicit depth limit or you get an exponential tree.
  • Give parallel branches separate state keys or a reducer. Never both writing the same scalar.
  • Set per-branch timeouts so one hung worker doesn't hold the join forever; decide up front whether the join is all-must-complete or best-effort-with-quorum.
  • Make cancellation real: when the run is killed, in-flight branches must actually stop, not finish and bill you.

13. πŸ›‘οΈ Governance and security at graph scale

A single agent has a blast radius. A graph has a blast radius and a propagation path.

13.1 Identity: every node is a caller

If every node runs under one shared service credential, then "the graph did it" is the most precise answer your audit log can ever produce.

Give every independently governed node a resolved identity, and propagate correlation identifiers on every model and tool call:

Authorization: Bearer <node-scoped-token>
X-Agent-Metadata: {
  "graph_id":  "release-review",
  "graph_version": "v7",
  "run_id":    "run-8f31",
  "node_id":   "security-reviewer",
  "parent_node_id": "supervisor",
  "actor":     "svc-agent/security-reviewer"
}
Enter fullscreen mode Exit fullscreen mode

This one header is the foundation for Β§14 (correlation), cost attribution, per-node rate limits, and every after-the-fact question you'll be asked.

Note the trust-model choice hiding here: implicit peer trust β€” all nodes share one credential, trust inherited for the session, no per-interaction authentication β€” is the default in most frameworks and is directly vulnerable to privilege escalation. Choose deliberately.

13.2 Least privilege, per node

Scope each node to its mandate with a tool registry, not with prompt instructions:

  • What nodes may reach β€” registry/allowlist configuration, per node.
  • How tools authorize the caller β€” credentials scoped to the node, not the graph.
  • Whether an operation needs pre-approval β€” structural checkpoints on the sensitive edges.

And the structural rule from Β§5.3, repeated because it's the highest-value one: retrieval nodes and write nodes are never the same node. Untrusted content and dangerous permissions must not meet inside one context.

13.3 Cross-agent prompt injection: the graph-scale threat

Here's the failure that is genuinely new at graph scale:

flowchart LR
    WEB[("🌐 untrusted<br/>web page")] -->|injected text| R["Researcher"]
    R -->|"unvetted output<br/>becomes sibling input"| W["Writer"]
    W --> D["Deployer πŸ”‘"]
    D -->|"executes attacker's<br/>instruction with prod creds"| BOOM(["πŸ’₯"])
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef bad fill:#7f1d1d,color:#fff,stroke:#450a0a;
    class R,W n;
    class WEB,D,BOOM bad;

One node ingests poisoned content; its output crosses an edge and becomes another node's trusted input. Guardrails that only inspect the user's original request never see it.

Mitigations, in order of effectiveness:

  1. Assume injection will succeed and design so the blast radius is bounded. This is the only posture that survives contact with reality.
  2. Filter at edges, not just at the boundary. Apply output guardrails on inter-node communication, the same way you'd apply input guardrails on user text. The four useful hooks: llm_input, llm_output, pre_tool_invoke, post_tool_invoke.
  3. Provenance-tag content. Mark which parts of state came from untrusted sources and forbid those fields from reaching privileged nodes.
  4. Never let retrieved content select a route. If a router reads model output influenced by fetched documents, an attacker controls your topology.
  5. Human gate on irreversible actions β€” the last line, and the one that actually holds.

Residual risk is real: cross-agent injection remains an open problem at graph scale. Bound it; don't claim to have solved it.

13.4 The seven-question production checklist

Borrowed from the enterprise graph literature and worth running verbatim before any graph handles real consequences. Each unanswered item is a plausible incident vector.

  1. Does every independently governed caller have a resolved identity?
  2. Do model and tool calls carry stable graph, run, and node identifiers?
  3. Does the orchestrator record the actual runtime work graph, not just the intended topology?
  4. Can orchestration traces be correlated with cost, policy, latency, and tool records?
  5. Are budget rules mapped to nodes, not just to the graph?
  6. Are sensitive tool actions protected by explicit approval checkpoints?
  7. Are model changes isolated behind a routing abstraction, so an A/B test or migration doesn't silently change behavior?

14. πŸ”­ Observability: the intended graph vs. the runtime work graph

You cannot debug a non-deterministic system you cannot replay.

14.1 Three layers, one correlation key 

flowchart TB
    ORCH["<b>Orchestrator layer</b><br/>topology + actual work graph<br/>(source of truth)"]
    GW["<b>Gateway layer</b><br/>model + tool calls<br/>latency Β· cost Β· policy outcome"]
    APP["<b>Application layer</b><br/>business outcome<br/>did the thing actually work?"]
    KEY[("correlation key<br/><b>graph_id + run_id + node_id</b>")]
    ORCH -.-> KEY
    GW -.-> KEY
    APP -.-> KEY
    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef k fill:#374151,color:#fff,stroke:#111;
    class ORCH,GW,APP n;
    class KEY k;

Without the shared key, you have three sets of isolated traces and a dashboard that says "aggregate cost: $840" with no way to find the node that spent it.

14.2 What to emit per node execution

Instrument with OpenTelemetry GenAI semantic conventions where you can (gen_ai.* attributes β€” still marked experimental in 2026, but vendor-neutral and portable, which beats a proprietary schema you'll migrate off).

Per node, emit a span with:

graph_id, graph_version, run_id, node_id, parent_node_id, attempt
model, input_tokens, output_tokens, cached_tokens, cost_usd
tool_calls[], tool_errors[]
routing_decision, routing_confidence, edge_taken
state_delta_keys        # which state fields this node changed
duration_ms, status, error_class
guardrail_outcomes[], approval_id
Enter fullscreen mode Exit fullscreen mode

Two of those are easy to skip and painful to lack: edge_taken (otherwise you can never reconstruct the work graph) and state_delta_keys (otherwise you can't tell which node corrupted a field).

14.3 Reconstruct and diff the work graph

The habit that pays for itself: after every run, render the actual work graph and diff it against the intended one.

intended:  plan β†’ research β†’ write β†’ review β†’ ship
actual:    plan β†’ research β†’ research(retry) β†’ research(retry) β†’ write β†’ review β†’ write β†’ review β†’ ship

⚠️  research retried 2Γ— (transient? or a bad tool?)
⚠️  reviewβ†’write loop-back fired once (expected: <10% of runs; actual this week: 34%)
Enter fullscreen mode Exit fullscreen mode

Those two lines find more real problems than any eval suite. Track the distribution of work-graph shapes over time; a shift in shape is a regression signal that precedes a quality regression by days.

15. πŸ“ Evaluating a graph (node, edge, and trajectory level)

Evaluating a graph is not evaluating one prompt N times. There are three distinct levels and you need all three.

Level Question Method Fails you when
Node Does this node do its job given known input? Golden-set unit evals per node; deterministic assertions where possible Passes while the system fails β€” nodes are individually fine, composition is wrong
Edge / routing Did we take the right path? Confusion matrix over routing decisions on a labeled set Ignored entirely by most teams
Trajectory Was the whole run sensible and efficient? Reference-free LLM-judge over the trace; milestone checks Judge is the same model family that produced the run
Outcome Did the thing actually work in the world? Tests, business metrics, human acceptance You don't have it (see Β§16)

15.1 Practical protocol

  1. Start with ~20 examples. Not 2,000. Twenty real cases you understand beats a large synthetic set you don't.
  2. Freeze a golden set per node. Node evals are cheap, fast, and let you swap a model in one node without re-running everything.
  3. Score routing separately. Build a confusion matrix for your router. Misroutes are usually the largest single source of bad outcomes and the least measured.
  4. Use reference-free trajectory judging for the whole run β€” no gold path required; the judge reads the observed trace and rates it against a rubric. Score at turn, milestone, and trajectory granularity.
  5. Diff the work-graph shape distribution between versions (Β§14.3). Shape changes are often the first observable symptom of a regression.
  6. Keep humans in the loop on evaluation itself. Automated judging drifts; periodic human review of a sample is the calibration.

15.2 The trap

An LLM judge scoring an LLM graph shares the failure modes of what it's judging. Models prefer their own outputs and agree with each other. Your eval suite can be green while the product is wrong β€” confidently, consistently, and at scale. Which brings us to the most important section in this guide.

16. βš“ Anchors: keeping the graph honest

Agents checking agents produce extremely organized nonsense.

This is the deepest critique of graph engineering and it deserves to be taken literally. Multiple agents on the same model, reading the same flawed context, produce beautifully structured, internally consistent, well-cited, wrong output β€” and every internal check agrees.

An anchor is an external, fixed reference that the optimizing machinery is forbidden to rewrite.

16.1 Anchor types, ranked by strength

Strength Anchor Example
🟒 Strongest Reality Money reached the bank. The customer renewed. The deploy stayed up 24h.
🟒 Strong Execution Tests actually ran and passed. The build compiled. The migration applied.
🟑 Medium Formal Schema validated. Types checked. Policy engine approved. Invariant held.
🟑 Medium Human A person reviewed and accepted, with their name on it.
πŸ”΄ Weak Model judgment An LLM said it looked good.

Every graph needs at least one anchor from the top three rows. A graph whose only checks are model judgments is a machine for generating confident agreement.

16.2 Four anchoring practices

Metrics never travel alone. Pair every optimization target with a counter-metric and an anchor metric that resists gaming. Optimize "tickets resolved" alone and you'll get tickets closed without resolution. Goodhart's law applies with unusual force here because the optimizer is fluent.

References have owners. A target belongs to a slower, higher-level loop β€” not to a config value the fast loop can edit. If the graph can change its own success criterion, it will.

Cadence separation. Different loops run at different speeds and only slower loops may adjust faster ones' targets: per-run tuning, weekly ops review, quarterly strategy, annual audit. A fast loop that can rewrite a quarterly target is an unsupervised optimizer.

Intentional freezing. Some nodes are explicitly not tunable: held-out test sets, safety constraints, ground-truth checks. Write "this node is frozen" in the code and mean it. Freezing is a feature, not technical debt.

16.3 The structural failure modes anchors defend against

Failure What it looks like
Goodhart drift The metric goes up; the thing it measured stopped happening.
Upward blindness Loops optimize hard and cannot question whether the target is right.
Inter-loop conflict Two subsystems each hit their targets while undermining each other.
Measurement decay The sensor drifts; the system stays internally consistent and externally wrong.

Graphs of loops will fail in exactly this way wherever they're built without anchors. Some evidence has to come from outside the agent system: tests that actually ran, money that reached the bank, customers who stayed.

17. πŸ› Anti-patterns and graph smells

# Smell Why it hurts Fix
1 The org chart graph Nodes named after job titles ("VP of Research Agent"). Persona cost, no capability gain. Name nodes for specialties with distinct tools and success criteria.
2 God state One dict holding transcripts, blobs, credentials, and flags. Four state layers (Β§8.1); references not blobs.
3 Everyone sees everything Default shared state; reviewers get anchored, injections propagate, context bloats. Explicit context projection per edge.
4 Prompt-enforced constraints "Never do X more than 3 times" in a system prompt. Constraints in routing code with counters.
5 The fake verifier A node that scores 8/10 and always passes. Deterministic checks first; make the verifier able to fail the run.
6 Uncapped fan-out Dynamic spawning with no width or depth limit, inside a retry. Cap N and depth in code; subtree budgets.
7 Parallel writers Two nodes writing the same artifact, making conflicting implicit decisions. Parallelize reads, serialize writes.
8 The infinite polite disagreement Generator and critic ping-pong with no cap. Revision counter + escalation edge.
9 Happy-path-only edges Every node has a success edge; failures land in except: pass. Error edges per failure class (Β§7.2).
10 Advisory human gate Model "asks permission" in text and proceeds. Structural interrupt that suspends the run.
11 Correlated ensemble Three judges, same model, same prompt, same context. Diversify model/prompt, or use one model + one deterministic check.
12 Checkpoint everything Serializing 200KB state on every micro-step. Checkpoint at phase boundaries; subgraph without checkpointer inside.
13 Graph as procrastination Rebuilding topology instead of fixing a bad tool description. Β§10.2 β€” fix the boring thing first.
14 Untracked work graph Only the intended topology is recorded. Record and diff the actual run (Β§14.3).
15 Shared credential Every node runs as one service account. Node-scoped identity + tool registry.
16 Zombie nodes Nodes that haven't fired in three months, still maintained. Track per-node hit rate; delete what never fires.

Two more from the human side, which are the ones that actually get people in trouble:

  • Delegating review to your teammates. Opening a PR with a thousand lines of agent-generated code you haven't read yourself isn't shipping fast; it's moving the work onto the reviewer. Graphs make it easier to produce volume, which makes this failure easier to commit.
  • Comprehension debt. A graph you can't explain on a whiteboard is a graph you can't debug at 3am. If a new engineer can't trace one run end-to-end in 20 minutes, the topology is too clever.

18. 🧰 Framework landscape 2026 and how to choose

Framework Model Best at Watch out for
LangGraph StateGraph β€” nodes, edges, typed state, interrupts, subgraphs Fine-grained control, human-in-the-loop, auditability, regulated environments Checkpoints β‰  durable execution (Β§12.1); you supply the supervisor process
Google ADK (v2.0, GA May 2026) Graph-based execution engine; sequential/parallel/loop agents; event sourcing Google ecosystem, native A2A with auto-generated Agent Cards Caller still detects failure and retries with the right invocation ID
Microsoft Agent Framework (1.0 GA, Apr 2026) Typed workflows; merged AutoGen + Semantic Kernel .NET shops, Python + .NET + Go parity Newer surface; AutoGen's GraphFlow is in maintenance β€” don't start there
OpenAI Agents SDK Handoff as the core primitive; sandboxed execution Fast handoff-shaped systems, tight OpenAI integration Handoff drift past a few hops (Β§9.6)
Claude Agent SDK Harness-first; built-in coding tools, subagents as stateless workers Coding agents, orchestrator–worker done well by default Opinionated harness; less of a generic graph DSL
CrewAI Role/crew abstraction Fast prototyping, role-shaped problems @persist saves after success; you build the skip logic on resume
Temporal / Restate / DBOS Durable workflow engines The reliability spine under any of the above Not agent frameworks; you bring the agent layer
Roll your own A dict, a dispatch table, and a while loop Small graphs (<8 nodes) with unusual requirements You will rebuild checkpointing, retries, and tracing β€” budget for it

18.1 How to actually choose

The 2026 norm is two or three tools, not one: a vendor SDK for its native capabilities plus a framework for orchestration, and often a durable engine underneath.

Choose on these axes, in this order:

  1. Do you need durable execution? (runs > 15 min, human gates, real money) β†’ put Temporal/Restate/DBOS underneath whatever else you pick. This decision is structural and expensive to retrofit.
  2. Do you need auditability and hard control? β†’ LangGraph or Agent Framework. Avoid anything with hidden prompts or an enforced architecture β€” full framework control is a production requirement, not a preference.
  3. Do you need cross-vendor agent interop (A2A)? β†’ ADK or CrewAI narrow the field.
  4. What language does your team actually maintain? A graph in a language your team can't debug is worse than a simpler graph in one they can.
  5. Can you get the topology out as data? If you can't serialize, diff, and version the graph, you don't have graph engineering β€” you have a framework.

Migration insurance: keep node implementations as plain functions with typed inputs and outputs, and keep the graph wiring in one thin file. Then swapping runtimes is a day, not a quarter. Framework churn in this space is fast β€” AutoGen's GraphFlow went to maintenance within a year β€” and the vocabulary itself will keep churning. Don't build your business logic inside somebody's DSL.

19. 🏭 Worked example: a monorepo release-review graph

Concrete beats abstract. Here's a graph for a real, common job: reviewing a pull request in a polyglot monorepo (Go API + Python ML service + React frontend) and deciding whether it can ship.

It composes five patterns from Β§9: router, fan-out/fan-in, evaluator–optimizer, human gate, and fallback.

flowchart TD
    START([PR opened]) --> TRI{"<b>Triage</b> (deterministic)<br/>which surfaces changed?"}

    TRI -->|go| GO["<b>go_reviewer</b><br/>layering Β· error wrap<br/>tx patterns"]
    TRI -->|python| PY["<b>py_reviewer</b><br/>types Β· async<br/>statelessness"]
    TRI -->|frontend| FE["<b>fe_reviewer</b><br/>strict TS Β· query<br/>error surfacing"]
    TRI -->|migrations| MIG["<b>migration_reviewer</b><br/>⚠️ never edit applied"]
    TRI -->|always| SEC["<b>security_reviewer</b><br/>read-only + veto"]

    GO --> JOIN
    PY --> JOIN
    FE --> JOIN
    MIG --> JOIN
    SEC --> JOIN

    JOIN["<b>join</b><br/>dedupe Β· rank by severity<br/>any-veto from security"] --> TESTS

    TESTS["<b>run_tests</b> βš“<br/>make test (real execution)"] --> VERDICT{"<b>verdict</b>"}

    VERDICT -->|"blocking findings<br/>and under 2 revisions"| FIX["<b>fixer</b><br/>address findings"]
    FIX --> TESTS
    VERDICT -->|"revisions β‰₯ 2"| ESC(["πŸ™‹ escalate to human"])
    VERDICT -->|"clean + low risk"| SHIP([βœ… approve])
    VERDICT -->|"clean + touches<br/>migrations or auth"| GATE(["πŸ™‹ human approval"])
    GATE -->|approve| SHIP
    GATE -->|reject| FIX

    classDef n fill:#0e7490,color:#fff,stroke:#083344;
    classDef g fill:#92400e,color:#fff,stroke:#451a03;
    classDef h fill:#4c1d95,color:#fff,stroke:#2e1065;
    classDef a fill:#065f46,color:#fff,stroke:#022c22;
    class GO,PY,FE,MIG,SEC,JOIN,FIX n;
    class TRI,VERDICT g;
    class ESC,GATE h;
    class TESTS a;

19.1 Why each design decision

Decision Reason
Triage is deterministic (a path glob), not an LLM The answer is knowable from the diff. No ambiguity to resolve β†’ no model (Β§6.1).
Reviewers fan out in parallel Pure reads over the same diff. Independent, no write conflicts (Β§9.3).
Each reviewer gets only its own files + conventions Context projection: the Go reviewer never sees the React diff, so it can't hallucinate cross-stack advice (Β§5.2).
security_reviewer runs on every PR and holds a veto Any-veto aggregation; the highest-consequence check is not conditional (Β§9.9).
run_tests is the anchor βš“ Real execution, outside the model. Without it, five reviewers can agree on nonsense (Β§16).
Revision cap of 2 with an escalation edge Bounded evaluator–optimizer; no infinite polite disagreement (Β§9.5).
Human gate only on migrations or auth Consequence concentrates there β€” irreversible schema changes, security boundaries. Gating everything causes rubber-stamping (Β§9.7).
Reviewers are read-only; only fixer writes Retrieval and write privileges never share a node (Β§5.3, Β§13.2).

19.2 The state schema 

from typing import Annotated, Literal, TypedDict
from operator import add

class ReviewState(TypedDict, total=False):
    # identity / correlation
    graph_version: str
    run_id: str
    pr_number: int

    # inputs (references, not blobs)
    diff_ref: str                 # s3://…  not the diff text itself
    changed_surfaces: list[str]   # ["go", "migrations"]

    # parallel writes -> reducer
    findings: Annotated[list[dict], add]

    # anchor
    test_result: dict             # {"passed": bool, "failed": [...], "ran_at": ...}

    # control flow (explicit, bounded)
    revision_count: int
    security_veto: bool
    risk_tier: Literal["low", "high"]
    human_decision: dict          # {"actor":…, "action":…, "at":…}
Enter fullscreen mode Exit fullscreen mode

Note what is not in state: the diff text, reviewer scratchpads, credentials, and raw model transcripts. All by reference or not at all.

19.3 The routing function β€” every hard rule in code 

def verdict(state: ReviewState) -> Literal["fix", "escalate", "gate", "ship"]:
    blocking = [f for f in state["findings"] if f["severity"] == "high"]

    # hard stop before anything else: security veto is absolute
    if state.get("security_veto"):
        return "escalate"

    if blocking or not state["test_result"]["passed"]:
        if state.get("revision_count", 0) >= 2:      # bounded revision
            return "escalate"
        return "fix"

    if state["risk_tier"] == "high":                  # migrations / auth
        return "gate"

    return "ship"
Enter fullscreen mode Exit fullscreen mode

Every constraint that matters β€” veto, revision cap, risk gating β€” is a Python expression. None of it lives in a prompt where a model can reason its way around it.

19.4 What this graph costs

Rough shape for a mid-size PR, so you can sanity-check your own:

Node Calls Notes
triage 0 Deterministic
2–5 reviewers (parallel) 1 each Latency = slowest reviewer, not the sum
join 0–1 Deterministic dedupe; 1 call only if ranking is semantic
run_tests 0 Real test suite, real minutes
verdict 0 Deterministic
fixer (0–2Γ—) 1–3 each The expensive path

Typical: 3–7 model calls on the happy path, 10–20 with two revision rounds. The single biggest cost lever is making triage narrow so you run two reviewers instead of five.

20. πŸ’Έ The cost and latency model of a graph

Build the model before you build the graph. A napkin estimate prevents most cost incidents.

run_cost  β‰ˆ  Ξ£ over nodes [ calls Γ— (in_tokens Γ— in_price + out_tokens Γ— out_price) ]
             + Ξ£ retries
             + fan_out_width Γ— per_branch_cost
             + revision_rounds Γ— loop_body_cost

run_p50_latency β‰ˆ Ξ£ (serial nodes) + max(parallel branches) + human_wait
run_p99_latency β‰ˆ the above, with every retry and every escalation firing
Enter fullscreen mode Exit fullscreen mode

Four properties worth internalizing:

  1. Parallelism buys latency, never cost. Fan-out is Ξ£ tokens, max latency. If you're fanning out to save money, you have it backwards.
  2. Loops multiply. A 3-node body with 3 revision rounds is 9 node executions, and each may itself retry.
  3. Cached prefixes are the biggest lever. Stable, shared system prompts across nodes hit prompt caching; per-node bespoke preambles don't. Structure prompts so the invariant part comes first.
  4. The cheap-model rung is the second biggest lever. Fallback ladders (Β§9.8) and small-model routing (Β§9.2) routinely cut cost by more than half with no measurable quality loss on the easy majority.

Set the budget as a design constraint, not a monitor. Decide "this graph may cost $0.40 per run" first, then design a topology that fits. Retrofitting a budget onto a graph designed without one usually means deleting nodes.

21. πŸš‘ Debugging playbook

When a graph misbehaves, work in this order. It's roughly cheapest-to-most-expensive, and the first three catch most of it.

1. Reconstruct the work graph. What actually ran? Compare to intended (Β§14.3). Nine times out of ten the surprise is right here β€” a node ran three times, or an edge you forgot about fired.

2. Find the first divergent node. Walk the trace forward to the first node whose output is wrong. Everything after it is downstream noise. Debug that node in isolation with its exact recorded input.

3. Check the context that node received. Not the state β€” the projection. Almost always one of: it got too much (anchored/confused/blown window), too little (missing a decision made upstream), or stale (a field written after it read).

4. Classify the failure against the taxonomy. The empirical distribution across 1,600+ traces:

Category Share Representative modes
Specification & design ~42% Disobey task specification (~12%), step repetition (~16%), unaware of termination conditions (~12%)
Inter-agent misalignment ~37% Lost messages, circular handoffs, ignored input from peers
Verification gaps ~21% No independent validation; premature or incorrect termination

Notice: ~79% of failures are specification and coordination, not model capability. The instinct to swap in a better model is usually the wrong first move.

5. Ask the three edge questions (Β§7.1) for the edge into the failing node. Data? Authority? Failure behavior? One of them is unanswered β€” that's your bug.

6. Only now consider topology. Should this node be split? Merged? Does it need a fresh-context boundary? Restructure last, because restructuring invalidates everything you've learned.

21.1 Symptom β†’ cause quick table

Symptom Most likely cause
Run never terminates No max-iteration counter on a loop-back edge; termination condition only in a prompt
Same step repeats No no-progress detection; state not actually updated by the node
Reviewer always approves Shared context with the author
Cost spike, no output change Uncapped fan-out or a retry storm inside a loop
Works alone, fails in graph Context projection wrong β€” the node isn't getting what it got in your test
Non-reproducible Non-determinism not wrapped in checkpointed tasks; no seed; no recorded inputs
Duplicate side effects Missing idempotency keys; replay re-executing
Silent wrong answers No anchor; every check is a model judging a model

22. πŸ“ˆ Metrics that actually tell you something

Metric Why it matters Alert when
Task success rate (against an anchor) The only outcome metric that isn't self-graded Drops vs. rolling baseline
Cost per successful run Cost per run hides the retries Rises while success is flat
Work-graph shape distribution Regressions change the shape before they change the score Shape mix shifts week over week
Node hit rate Finds zombie nodes and dead branches A node fires <1% or 100% of runs
Router confusion matrix Misroutes are the biggest under-measured error source Any class drops below its baseline
Loop-back rate per cycle Directly measures maker–checker health Exceeds its designed rate (e.g. >20%)
Escalation rate per ladder rung Earliest signal of model or input drift Rung-1 success falls
Human gate: approve/reject ratio ~100% approval means rubber-stamping Approval rate >95% for >2 weeks
p50 / p99 latency split by path Aggregate latency hides the escalation path p99 grows while p50 is flat
Guardrail trigger rate at edges Injection attempts and content policy hits Any sustained increase
Retries per node Locates flaky tools and bad contracts One node dominates retries
Budget-exhaustion rate Runs killed by ceilings >2% of runs hit a hard stop

The two most neglected on that list are router confusion and human approval ratio. A router quietly misrouting 8% of traffic and a gate that approves everything are both invisible to conventional dashboards and both fully corrosive.

23. βœ… Adoption ladder and quick-start checklist

23.1 The ladder

Rung You have You add Time
0 A prompt A real verifier and a stop condition β†’ you have a loop days
1 A loop Wrap it as a one-node graph: typed state + checkpointer + traces 1 day
2 A one-node graph Split off the one step that keeps failing 2–3 days
3 2–3 nodes The gate you currently perform manually (validator or human) 2 days
4 A gated pipeline Parallelize the read-only work, capped, with a real reducer 1 week
5 A working graph Node identity, per-node budgets, work-graph recording 1 week
6 An observable graph Node + routing + trajectory evals; a golden set per node ongoing
7 An evaluated graph Durable execution underneath; subgraph extraction as needed

Do not skip rungs. Every rung above 4 assumes the anchor from rung 0 exists. Teams that jump from rung 1 to rung 5 build impressive topologies over fake verifiers.

23.2 Pre-flight checklist

Before a graph touches anything that matters:

Design

  • [ ] The graph is drawn, versioned, and committed β€” not implied by code
  • [ ] Every node has a contract: inputs, outputs, tools, timeout, retry, side effects, budget
  • [ ] Every LLM node answers "what ambiguity does this resolve?" in one sentence
  • [ ] Every edge answers: what data crosses, what authority crosses, what happens on failure
  • [ ] Every cycle has a counter and an escalation edge

State

  • [ ] State is typed and versioned (schema_version)
  • [ ] Blobs are references; no credentials or raw transcripts in state
  • [ ] Every concurrently written key has a reducer with a stated conflict policy
  • [ ] Context projection is explicit per edge β€” no implicit "everyone sees everything"

Truth

  • [ ] At least one anchor from reality / execution / formal validation (Β§16.1)
  • [ ] The verifier can actually fail the run, not just annotate it
  • [ ] Frozen nodes (held-out sets, safety checks) are marked and not tunable

Safety

  • [ ] Retrieval nodes and write nodes are separate
  • [ ] Each node has scoped credentials, not a shared service account
  • [ ] Guardrails run on inter-node edges, not just the user boundary
  • [ ] Human gates are structural interrupts on irreversible actions, and produce approval records

Operations

  • [ ] Hard stops: max iterations, no-progress, spend ceiling, wall-clock deadline
  • [ ] Side-effecting nodes are idempotent with stable keys
  • [ ] graph_id + run_id + node_id propagate on every model and tool call
  • [ ] The actual runtime work graph is recorded and diffable
  • [ ] Cost is attributed per node, not per graph
  • [ ] Fan-out width and spawn depth are capped in code

Sanity

  • [ ] A new engineer can trace one run end-to-end in 20 minutes
  • [ ] Every node fires in >1% and <100% of runs (no zombies, no pointless universals)
  • [ ] You can articulate what this graph does that a single loop couldn't

23.3 The one-paragraph version

Start with one recurring task, a real verifier, inspectable state, and hard stops. Make your existing loop a node. Split off the step that keeps failing. Add the gate you're already doing by hand. Parallelize only the reads, and cap them. Give every node an identity, a budget, and a contract. Record what actually ran. Put at least one anchor outside the model in the path. Then β€” and only then β€” expand.

πŸ“– Sources & further reading

On graph engineering (the discipline, July–August 2026):

Enterprise hardening (governance, identity, observability):

The counterweight (when not to):

Durability, runtime, and tooling:

Adjacent:

πŸ—ΊοΈ Companion reads

These documents live in this same repo and pair directly with the topics above. Read them in the order that matches where you are right now.

Document Why it pairs with this guide
πŸ€– The Agentic Loop πŸ”„ Loop Engineering: A Practical Field Guide πŸ“˜ Read this first if you haven't. Graph engineering assumes a good loop underneath β€” real verifier, real stop condition. Every node in Β§6 is one of those loops.
πŸ—οΈ Building High-Quality AI Agents πŸ€– β€” A Comprehensive, Actionable Field Guide πŸ“š The harness layer beneath the loop: ACI design and tool ergonomics. Most "graph problems" in Β§21 are actually tool-description problems.
πŸ“˜ The Complete Guide to LLMs and AI Agents πŸ€– Broad grounding on agent architectures; useful before the pattern library in Β§9.
⚠️ Common Issues πŸͺ² with LLMs & AI Agents πŸ€– β€” and How to Fix Them πŸ› οΈ The failure catalogue at the single-agent level β€” pair with Β§21's debugging playbook and the MAST taxonomy.
πŸ€– Optimizing AI Agents: Token Economics πŸ’°, the Harness & Context Engineering βš™οΈ Directly extends Β§20. Context projection, caching, and compaction are where graph cost is actually won.
🏒 Building Enterprise-Ready AI Agents πŸ€– β€” A Practical Field Guide πŸ“š Governance, identity, and audit at organizational scale β€” the long form of Β§13.
🌱 Supspec Orchestration πŸ€– β€” From Spec to Evidenced Draft PRs, Autonomously A concrete orchestration implementation to read against Β§9.4 and Β§9.10.

Suggested reading path:

  1. πŸ€– The Agentic Loop β€” get one loop right
  2. β†’ πŸ—οΈ Building High-Quality AI Agents β€” get the harness right
  3. β†’ This guide β€” get the topology right
  4. β†’ πŸ€– Optimizing AI Agents: Token Economics β€” get the bill right
  5. β†’ 🏒 Building Enterprise-Ready AI Agents β€” get it past review

Last updated: September 2026. The vocabulary in this field churns fast β€” "graph engineering," "org graph," "work graph" may not survive as standard terms. The requirements underneath will: governed access, budgets, guardrails, identity, traces, and evidence that comes from outside the model.

If you found this helpful, let me know by leaving a πŸ‘ or a comment!, or if you think this post could help someone, feel free to share it! Thank you very much! πŸ˜ƒ

Top comments (0)