Why the Supervisor Must Be Deterministic: Building a Multi-Agent Claims Triage System on AWS

A multi-agent system looks elegant on a whiteboard. Three specialist agents, a supervisor routing between them, a human approval gate at the end. The hard part is not making agents call each other. it is making sure the system does not break when an LLM returns something unexpected, a budget runs out, or a human rejects a recommendation.

I built Verdikt, an open-source insurance claims triage system on AWS. The core engineering idea turned out to be counterintuitive: the supervisor should not be intelligent. It should be a state machine.

The Problem With Intelligent Supervisors

My first instinct was to use an LLM as the orchestrator. Have the supervisor read the claim, decide which agent to call next, interpret results, and route rework requests. It felt natural — LLMs are good at reasoning about text.

Then I thought about retries. If the Reviewer returns a rework request, the supervisor needs to route back to the Extractor or Investigator, increment an attempt counter, check whether the budget is exhausted, and regenerate an attempt ID. An LLM could do this once. Doing it correctly across six attempts, with budget checks before every dispatch, while maintaining consistent state — that is a different problem.

The real issue is that constraint enforcement and LLM reasoning do not mix well. An LLM might route a rework to the wrong agent. It might forget to check the budget. It might hallucinate a successful completion when an agent actually failed. These are not hypothetical failure modes — they are the kinds of bugs that appear under load and disappear in demos.

I chose AWS Step Functions as the supervisor. It is a state machine, not a model. It enforces constraints deterministically.

Architecture: State Machine Meets Stateless Agents

The system has five components that matter:

Step Functions (the Supervisor) owns all control flow. It initializes each run, dispatches agents, checks budgets, routes rework, and pauses for human approval. It never calls an LLM.

Three Lambda agents (Extractor, Investigator, Reviewer) are stateless. Each receives an envelope containing the claim documents and prior results, calls Amazon Bedrock, validates the output against a Zod schema, persists the result to DynamoDB, and reports completion back to Step Functions via SendTaskSuccess or SendTaskFailure.

DynamoDB stores everything: 15 tables covering claims, runs, attempts, results, budgets, snapshots, events, and reference data. The schema is derived from Zod contracts, not the other way around.

Amazon Bedrock provides the inference. All three agents use Claude 3 Haiku by default, though the model is configurable per deployment.

API Gateway exposes a single endpoint for human approval decisions.

The flow for a single claim:

Claim submitted
  → InitializeRun (attemptCount: 0, budgetRemaining: $2.00)
  → Extract phase (increment → check step → check budget → dispatch → Bedrock → validate → persist)
  → Investigate phase (same pattern)
  → Review phase (same pattern)
  → ReviewRouter
      → rework? → loop back to Extractor or Investigator
      → recommendation? → ApprovalGate (pause)
  → Human approves/rejects via POST /approval
  → Completed or BudgetExceededFail

Every state transition is explicit. Every constraint check is a Choice state, not a prompt instruction.

The Contract Boundary

The most important architectural decision is where agents stop thinking and the state machine starts routing.

Each agent receives an AgentEnvelope — a Zod-validated structure containing the claim documents, the run ID, the attempt ID, and prior agent results. The agent's job is to process this envelope, call Bedrock, and return a typed result. It does not decide what happens next.

The results are schema-validated:

• ExtractionResult: structured facts with provenance (which document, which section)
• InvestigationResult: a verdict (proceed, reject, flag_for_review) with policy and prior-claim references
• ReviewResult: a discriminated union — either a ReworkRequest (targeting extractor or investigator) or a Recommendation (approve, reject, or escalate)

The ReviewRouter in Step Functions reads $.reviewerResult.outcome and $.reviewerResult.rework.targetAgent to decide the next state. No LLM interpretation needed.

This works because the contract is enforced at two levels. Zod validates at runtime inside each Lambda. JSON Schema files (generated from the same Zod definitions) are available for Step Functions input/output validation. If an agent returns something outside the schema, it fails immediately — not three steps later.

Budget Enforcement: Two Layers, Same Answer

Cost control is enforced at two levels, and they agree.

Step Functions checks $.budgetRemaining > 0 before every agent dispatch. If the budget is exhausted, the state machine transitions to BudgetExceededFail without invoking the agent. This is the primary guard.

BudgetStore provides reservation-based tracking inside each agent. Before a Bedrock call, the agent reserves 25% of the ceiling ($0.50). After the call, it records actual usage and adjusts. The reservation is atomic — a conditional DynamoDB update that fails fast if the budget is gone.

The constants are conservative: $2.00 ceiling per run, 6 maximum attempts. These are not arbitrary. A typical claim goes through three phases (extract, investigate, review). With rework, you might double that. Six attempts gives enough room for one rework cycle per phase without letting a pathological case burn unlimited Bedrock tokens.

The Step Functions-level check prevents the agent from even starting. The BudgetStore-level check prevents overspend within an agent. Together they create a hard ceiling that no combination of LLM hallucinations or retry storms can breach.

The Approval Gate: Pausing a State Machine

The human approval mechanism is where Step Functions' WAIT_FOR_TASK_TOKEN integration pattern becomes valuable.

After the Reviewer produces a Recommendation, the state machine transitions to ApprovalGate. This state invokes a Lambda that stores the recommendation in DynamoDB and returns the recommendationId. The workflow then pauses — the task token is held by Step Functions, waiting for an external signal.

A human calls POST /approval with the recommendationId, a decision (approve or reject), an actor identifier, and a timestamp. The approval Lambda validates the request, checks for duplicate approvals (returns 409 if already decided), records the decision, and then calls SendTaskSuccess to resume the workflow or SendTaskFailure to terminate it.

On approval, the system creates a SimulatedAction record — a no-op write-back that logs what a real claims-system integration would do. This is the boundary between the triage system and whatever downstream system would actually process the payment.

The 24-hour timeout on the approval gate is a practical constraint. If no human responds within a day, the workflow times out. For a production system, you would probably want escalation paths or deadline extensions, but for a reference architecture, a hard timeout prevents orphaned runs.

What I Would Change

The snapshot and replay mechanism is partially implemented. The SnapshotStore can persist a checkpoint of a TriageRun's full state after each successful attempt, and a replay handler can resume from a snapshot. But the Step Functions state machine does not yet integrate snapshot creation into its flow. Right now, a failed run means starting over.

The agent Lambdas lack unit tests. The contracts and budget store are tested, but the actual Bedrock integration, prompt construction, and result validation inside each agent are not. For a reference architecture this is acceptable. For production, I would mock Bedrock responses and test each agent's behavior against edge cases — malformed LLM output, schema validation failures, timeout handling.

The SQS handoff pattern works but adds latency. Each agent is invoked via Step Functions' LambdaInvoke with WAIT_FOR_TASK_TOKEN, but the actual invocation goes through SQS for retry semantics. That indirection means a failed agent takes longer to surface than a direct Lambda invocation would. The trade-off is worth it for durability, but I would profile the end-to-end latency before deploying this at scale.

The One Sentence Summary

A multi-agent system is only as reliable as its weakest coordination mechanism, and LLMs are not coordination mechanisms — they are reasoning engines. Put the reasoning in the agents and the coordination in a state machine.

The full source is at github.com/apLanka/verdikt. The CDK stack, the Zod contracts, and the Step Functions definition are the three files worth reading first.