Validate Your LLM Workflow Before It Runs (Python + YAML Topology Checks)

Your LLM app boots fine. It answers the first few messages fine. Then, ten minutes into a real conversation, it quietly walks from the support step into the billing step that doesn't exist anymore, or loops faq → support → faq → support until something times out. Nothing crashed at startup. The mistake was in the workflow's shape the whole time — you just didn't find out until runtime, in production, with a user watching.

This is the same class of bug as a malformed config file: the error exists the moment you write it, but the tool waits until the worst possible time to tell you. The fix is the same too. Catch it at load time, before a single step executes.

In this tutorial we'll build a small, declarative LLM workflow as YAML, then validate its topology — the graph of which step can hand off to which — before we run anything. We'll catch dead ends, transitions to steps that don't exist, and loops that can never terminate. All the code here runs against a real open-source package, llm-workflow-router, so you can follow along end to end.

Why runtime is the wrong place to find structural errors

Most guardrails for LLM systems limit volume: a max number of tool calls, a timeout, a retry cap. Those are useful, but they treat the symptom. A runaway loop gets cut off after N iterations — it doesn't get prevented, and you learn nothing about why the workflow could loop in the first place.

Structural validation asks a different question: before this workflow ever handles a request, is its graph even valid? A step that transitions to a nonexistent target is invalid no matter what any model says. A step with no way out is invalid. A cycle through a step that isn't allowed to be re-entered is invalid. None of that depends on content, so none of it needs to wait for runtime to be caught.

The mental model: your workflow is a directed graph. Nodes are steps ("containers"). Edges are the transitions each step permits. Validating the workflow means analyzing that graph and refusing to start if it's broken.

The workflow as YAML

Here's a minimal support flow. A user enters at support_entry, which may hand off to faq or refuse; faq may only refuse. REFUSE is a terminal state, not a step.

# good_config.yaml
entrypoints:
  - support_entry

containers:
  support_entry:
    allow_transitions:
      - faq
      - REFUSE
    allow_reentry: false
    max_invocations: 3

  faq:
    allow_transitions:
      - REFUSE
    allow_reentry: true
    max_invocations: 5

Three things are being declared per step, and each one is a rule the validator can check:

• allow_transitions — the only steps this one may hand off to. Anything else is an invalid transition, full stop.
• allow_reentry — whether the step may be entered more than once. This is what turns "is there a cycle?" into "is there a cycle that's actually a problem?"
• max_invocations — a hard ceiling on how many times the step may run. Must be greater than zero, or the step can never execute.

entrypoints names the authoritative roots of the graph. Declaring them explicitly means the validator doesn't have to guess where the workflow starts — and it can flag any step that no path can reach.

Validating from the command line

Install the package:

pip install llm-workflow-router

Then validate the config before you wire it into anything:

python -m router.cli validate good_config.yaml

{
  "ok": true
}

Clean exit, exit code 0. Now let's break it on purpose — the kind of breakage that's easy to introduce and impossible to spot by eye once a config gets large:

# broken_config.yaml
containers:
  support_entry:
    allow_transitions:
      - faq
      - billing        # typo: this step does not exist
    allow_reentry: false
    max_invocations: 3

  faq:
    allow_transitions:
      - support_entry   # loops back into a no-reentry step
    allow_reentry: true
    max_invocations: 5

  orphaned_step:
    allow_transitions:
      - REFUSE
    allow_reentry: false
    max_invocations: 1

There are three separate problems hiding in there. Run validate and it stops you at the first hard error:

python -m router.cli validate broken_config.yaml

{
  "ok": false,
  "error": "CONFIG_VALIDATION_ERROR",
  "message": "Container 'support_entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}."
}

Exit code 2. Notice what the error message actually does: it names the offending step (support_entry), the bad value (billing), and the set of things that would have been valid. That last part is the difference between an error that stops you and an error that helps you. More on that below.

Seeing every problem at once

validate is pass/fail — good for CI, where you want a nonzero exit and the first blocking reason. But when you're actually fixing a workflow, you want the whole picture, not one error at a time. That's what analyze is for: it inspects the topology and reports everything, graded by severity, without raising.

python -m router.cli analyze broken_config.yaml

{
  "container_count": 3,
  "entry_containers": [
    "orphaned_step"
  ],
  "entrypoints_declared": false,
  "cycle_count": 1,
  "cycles": [
    ["faq", "support_entry"]
  ],
  "issues": [
    {
      "severity": "ERROR",
      "code": "CYCLE_WITH_REENTRY_BLOCKED",
      "message": "Cycle ['faq', 'support_entry'] includes containers with allow_reentry=false: ['support_entry']"
    },
    {
      "severity": "ERROR",
      "code": "UNKNOWN_TARGET",
      "message": "Container 'support_entry' transitions to unknown 'billing'."
    },
    {
      "severity": "WARNING",
      "code": "UNREACHABLE_CONTAINERS",
      "message": "Unreachable containers: ['faq', 'support_entry']"
    }
  ]
}

This one report catches all three bugs I planted:

1. UNKNOWN_TARGET — the billing typo. A transition to a step that isn't in the graph.
2. CYCLE_WITH_REENTRY_BLOCKED — faq transitions back to support_entry, forming a loop faq → support_entry → faq, but support_entry declared allow_reentry: false. The workflow contradicts itself: the topology requires re-entering a step the config forbids re-entering.
3. The entrypoint inference falling apart. Because I removed the entrypoints block, the analyzer had to infer the root from graph shape — and the only step nothing points to is orphaned_step, so it wrongly became the "entry." Everything else is now unreachable from that inferred root, hence the UNREACHABLE_CONTAINERS warning. This is a great argument for always declaring entrypoints explicitly: it removes the guessing.

The severity split matters. An ERROR is a contradiction that makes the workflow structurally invalid — it blocks. A WARNING is a smell worth surfacing but not necessarily fatal. Cycles are a nice example of the distinction: a loop through steps that all permit re-entry is a WARNING (it can legitimately terminate via max_invocations), while a loop through a step that forbids re-entry is an ERROR (it can't).

Doing it in Python

The CLI is a thin wrapper. If you're building the workflow programmatically — or want validation inside your own app's startup path — the same checks are one function call.

from router.engine import WorkflowEngine
from router.models.config import ContainerConfig, WorkflowConfig
from router.models.enums import InteractionState
from router.models.metadata import InteractionMetadata
from router.validation.config_validator import validate_workflow_config

cfg = WorkflowConfig(
    containers={
        "support_entry": ContainerConfig(
            allow_transitions=["faq", "REFUSE"],
            allow_reentry=False,
            max_invocations=3,
        ),
        "faq": ContainerConfig(
            allow_transitions=["REFUSE"],
            allow_reentry=True,
            max_invocations=5,
        ),
    },
    entrypoints=["support_entry"],
)

validate_workflow_config(cfg)   # raises ConfigValidationError if the graph is invalid
print("Config valid ✓")

validate_workflow_config raises ConfigValidationError on any hard error, so the natural place to call it is at load time — the workflow simply refuses to start if its shape is wrong. That's the whole idea: an invalid state is made unrepresentable at runtime because it was rejected before runtime began.

Once the config is validated, evaluating a step is deterministic — the same metadata always yields the same decision:

engine = WorkflowEngine(cfg)

metadata = InteractionMetadata(
    container="support_entry",
    previous_state=InteractionState.PROCEED,
    transition_history=[],
    invocation_depth={"support_entry": 1},
    requested_action="faq",
    trace_id="demo-001",
)

print(engine.evaluate(metadata))

EvaluationResult(state=<InteractionState.PROCEED: 'PROCEED'>, container='support_entry', reason=None, trace_id='demo-001')

The engine checks the request against the validated rules: is support_entry a real step? Under its invocation ceiling? Is faq in its allow_transitions? All yes, so the result is PROCEED. Ask for a transition that isn't allowed and you'd get a REFUSE with a machine-readable reason like INVALID_TRANSITION — no exceptions thrown mid-flight, just an explicit, inspectable decision.

The part worth stealing: error messages that point at the fix

Go back and look at that first failure message:

Container 'support_entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}.

It would have been easier to write ValueError: invalid transition. It would also have been useless. A good validation error answers three questions without making the reader go digging: where is the problem (support_entry), what is wrong (billing isn't a valid target), and what would be right (a real step name, or a terminal state). The analyze codes do the same job for tooling — UNKNOWN_TARGET, CYCLE_WITH_REENTRY_BLOCKED, ORPHAN_CONTAINER are greppable, stable identifiers you can build dashboards or CI rules around.

If you take one habit away from this, let it be that one. Validation that only says no makes people resent your tool. Validation that says no, here, because of this, and here's what valid looks like makes people trust it. The effort is a few extra f-strings; the payoff is every future debugging session.

Wrapping up

Structural bugs in an LLM workflow — dead ends, phantom transitions, non-terminating loops — are present the instant the config is written. There's no reason to let them surface at runtime. By modeling the workflow as an explicit graph and validating its topology at load time, you convert a whole category of production incidents into a fast, boring failure at startup with a message that tells you exactly what to fix.

Everything here runs against the open-source llm-workflow-router (MIT). Try validate and analyze on your own configs — and if you're building anything where an AI system hands off between steps, put the validation call in your startup path before you need it.