DEV Community

Cover image for Three Agent Runs Passed. One Missing Handoff Broke the Workflow
Raju Dandigam
Raju Dandigam

Posted on

Three Agent Runs Passed. One Missing Handoff Broke the Workflow

My triage agent extracted a refund request and its order reference.

The refund specialist said the reference was missing.

The escalation agent then opened a generic support ticket.

All three runs completed successfully.

Viewed separately, each trace told a plausible story. Viewed as one user journey, they contradicted one another. The failure lived between the green runs.

That is the uncomfortable part of multi-agent debugging: a handoff can lose the one field that mattered while every component reports success.

The timestamp trap

The tempting investigation is:

  1. find the triage trace;
  2. search for a specialist trace a few milliseconds later;
  3. assume the next escalation belongs to the same request.

That works in a demo with one user. It breaks under queues, retries, parallel sessions, worker concurrency, and clock differences.

Time proximity is useful for discovery. It is weak evidence of causality.

10:00:00.100 triage completed
10:00:00.102 triage completed
10:00:00.110 specialist started
10:00:00.111 specialist started

Which triage run produced which specialist run?
The timestamps cannot answer.
Enter fullscreen mode Exit fullscreen mode

The relationship needs an identity that survives the boundary.

Give the journey an explicit spine

For a cross-run workflow, I want four small pieces of metadata:

  • sessionId: the user journey or workflow instance;
  • workflowName: the reusable workflow type;
  • handoffFrom and handoffTo: the declared edge; and
  • retryOf plus attempt when a run is a retry.

Here is a synthetic TypeScript example verified against agent-inspect@6.17.6:

import { inspectRun, step } from "agent-inspect";

const traceDir = ".agent-inspect";
const sessionId = "sess-support-042";

const triage = await inspectRun(
  "triage-agent",
  async () => {
    return step("extract-request", async () => ({
      category: "refund",
      orderRef: "internal-value",
    }));
  },
  {
    traceDir,
    metadata: {
      sessionId,
      workflowName: "support-refund",
      handoffFrom: "triage-agent",
      handoffTo: "refund-specialist",
    },
  },
);

// The bug: orderRef disappears while building the handoff payload.
const specialistInput = { category: triage.category };

const specialist = await inspectRun(
  "refund-specialist",
  async () => {
    return step(
      "read-handoff",
      async () => ({
        needsEscalation: !("orderRef" in specialistInput),
      }),
      {
        metadata: {
          receivedFields: Object.keys(specialistInput),
          orderRefPresent: "orderRef" in specialistInput,
        },
      },
    );
  },
  {
    traceDir,
    metadata: {
      sessionId,
      workflowName: "support-refund",
      handoffFrom: "refund-specialist",
      handoffTo: "escalation-agent",
    },
  },
);

if (specialist.needsEscalation) {
  await inspectRun(
    "escalation-agent",
    async () => {
      await step("open-ticket", async () => undefined);
    },
    {
      traceDir,
      metadata: { sessionId, workflowName: "support-refund" },
    },
  );
}
Enter fullscreen mode Exit fullscreen mode

Notice what the trace does not store: the order reference itself. The field name and presence bit are enough to locate the broken boundary.

Inspect one session instead of three files

List grouped workflow activity:

npx agent-inspect sessions --dir .agent-inspect
Enter fullscreen mode Exit fullscreen mode

Then inspect the declared handoffs and each run's timeline:

npx agent-inspect session sess-support-042 \
  --dir .agent-inspect \
  --timeline \
  --diagnostics
Enter fullscreen mode Exit fullscreen mode

The session view first gives the investigation a bounded shape:

session: sess-support-042

triage-agent
  handoff: triage-agent -> refund-specialist
                     |
                     v
refund-specialist
  handoff: refund-specialist -> escalation-agent
                     |
                     v
escalation-agent

three runs: completed
Enter fullscreen mode Exit fullscreen mode

Then inspect the specialist run's bounded step metadata:

npx agent-inspect view <refund-specialist-run-id> \
  --dir .agent-inspect \
  --verbose
Enter fullscreen mode Exit fullscreen mode
read-handoff
  receivedFields: category
  orderRefPresent: false
Enter fullscreen mode Exit fullscreen mode

The escalation was not the original failure. The field disappeared when specialistInput was constructed.

That sounds obvious after the fact. Before the runs are correlated, it is easy to blame the last agent because it produced the visible symptom.

Correlation metadata is reviewable architecture

Explicit fields improve more than the trace viewer. They expose intended topology in code review.

A reviewer can ask:

  • Should these runs share a session?
  • Is the handoff target correct?
  • Which fields are required at this boundary?
  • Is this a retry of a specific run or merely another attempt?

AgentInspect refuses to manufacture a causal link from timestamps alone. If attempt is greater than one without retryOf, the relationship can be labeled correlated rather than explicit, with a diagnostic explaining the ambiguity.

That distinction matters. A convincing diagram with an invented edge is worse than an incomplete diagram that admits uncertainty.

Trace the boundary, but validate it too

Observability explains what executed. It should not be the first place a missing required field is discovered.

Add runtime validation before sending and after receiving:

type RefundHandoff = {
  category: "refund";
  orderRef: string;
};

function assertRefundHandoff(
  value: Record<string, unknown>,
): asserts value is RefundHandoff {
  if (value.category !== "refund") {
    throw new Error("Invalid refund category");
  }

  if (typeof value.orderRef !== "string" || value.orderRef.length === 0) {
    throw new Error("Missing orderRef at refund handoff");
  }
}
Enter fullscreen mode Exit fullscreen mode

The schema check prevents the bad handoff. The session trace shows which boundary and version actually executed when behavior still surprises you. Those are complementary jobs.

Record the contract version, not the payload

Multi-agent workflows evolve. The triage agent may emit orderRef today while a newer specialist expects orderId. Even when both fields are strings, that is a protocol change.

Add a bounded contract identifier to the step or run metadata:

metadata: {
  sessionId,
  workflowName: "support-refund",
  handoffFrom: "triage-agent",
  handoffTo: "refund-specialist",
  handoffContract: "refund-request/v2",
  receivedFields: ["category", "orderRef"],
}
Enter fullscreen mode Exit fullscreen mode

The identifier lets a reviewer ask whether producer and consumer used the same boundary definition without storing the business value itself. It also separates two failure classes that otherwise look identical:

missing field
  ├─ producer never emitted it
  ├─ transport dropped it
  ├─ mapper renamed it
  └─ consumer expected another contract version
Enter fullscreen mode Exit fullscreen mode

AgentInspect treats these values as metadata; it does not enforce refund-request/v2. Your schema validator or message layer still owns that guarantee.

What sessions do not solve

A session index is not a workflow engine. It does not deliver messages, guarantee field consumption, or validate every handoff schema. Current handoff-related TraceContract support is not a complete policy language.

Session identifiers can also become sensitive correlation data. Use random identifiers rather than business values, keep metadata bounded, and apply an appropriate redaction profile before sharing traces outside the team.

The pinned sessions and outcomes documentation describes the current boundary.

The first question is not “Which agent failed?”

For multi-agent systems, start with:

At which boundary did the state stop matching the workflow's promise?

One explicit session ID, two handoff endpoints, and a few safe field-presence checks can turn three green traces into one explainable failure.

Where does your multi-agent system validate a handoff: before sending, after receiving, or both?

Top comments (0)