Agent Handoff Contracts: A CI Gate for Multi-Stage Pipelines

The problem, in one number

Across 20,574 real coding-agent sessions in 1,639 repositories, agents claimed success they had not actually achieved in 22.58 percent of episodes (arXiv 2605.29442, May 2026).

If you chain agents across stages, that false claim does not stay put. It gets handed forward and the next stage builds on it.

Agents claim success they have not achieved in 22.58% of real sessions. Here is a 120-line Python gate that stops an unverified claim from crossing a stage boundary.

Decompose agentic delivery and you get roughly ten stages: context, spec, architecture, backend, middleware, frontend, integration, deployment, scale, recovery. Ten stages means nine handoffs. Every benchmark you have read measures a stage. None of them measures a handoff.

This post is the handoff gate I use. YAML contract, Python validator, GitHub Actions wiring, and an honest list of what it does not catch.

The contract

Four clauses per seam. One file per handoff, checked into the repo next to the code it describes.

`# seams/03_architecture-to-backend.yaml
handoff:
from: architecture
to: backend
claim: "Order events are applied exactly once per (order_id, version)."

evidence:
- kind: test
ref: tests/idempotency/test_replay.py::test_duplicate_delivery
ran: 2026-09-11T14:22:09Z
result: pass
- kind: policy_scan
ref: checkov --framework terraform infra/events
ran: 2026-09-11T14:23:40Z
result: pass

assumptions:
- id: A-114
text: "Upstream publisher retries at most 3 times within 60s."
owner: platform-eng
expires: 2026-12-01
verified: false

falsifier:
condition: "orders_duplicate_write_total > 0 over any 15m window"
signal: prom:orders_duplicate_write_total
pages: oncall-orders

signed_by: a.prasad`

claim is stated so it can be false. "Ingestion is robust" is not a claim.

evidence is an executable artifact with a timestamp. If the next stage cannot re-run it, it is not evidence.

assumptions are what the stage decided without being told. This clause exists because on ClarifyCodeBench (419 ambiguous tasks, six frontier models) the best rate of asking the key clarifying question was 0.30. Models will not ask you. Force the declaration instead.

falsifier is the observation that would prove the claim wrong, plus the human who gets paged.

The three rules

Everything the gate enforces reduces to three lines:

Evidence with a stale timestamp is not evidence.
An unverified assumption cannot cross a boundary unsigned.
A claim with no falsifier is not a claim.
The validator

Stdlib plus PyYAML. No framework, no service, no database.

"""
handoff.py - a CI gate for agent-to-agent handoffs.

Four clauses per seam: claim, evidence, assumptions, falsifier.
Three enforcement rules, and nothing else:

1. Evidence with a stale timestamp is not evidence.
2. An unverified assumption cannot cross a boundary unsigned.
3. A claim with no falsifier is not a claim.

Stdlib plus PyYAML. Exit 0 clean, 1 on any violation.

python handoff.py seams/*.yaml

"""
from future import annotations

import sys
from dataclasses import dataclass, field
from datetime import datetime, date, timedelta, timezone
from pathlib import Path

import yaml

MAX_EVIDENCE_AGE = timedelta(hours=24)
EVIDENCE_KINDS = {"test", "policy_scan", "query", "benchmark"}

@dataclass(frozen=True)
class Evidence:
kind: str
ref: str
ran: datetime
result: str

@dataclass(frozen=True)
class Assumption:
id: str
text: str
owner: str
expires: date
verified: bool = False

@dataclass(frozen=True)
class Falsifier:
condition: str
signal: str
pages: str

@dataclass(frozen=True)
class Handoff:
src: str
dst: str
claim: str
evidence: list[Evidence] = field(default_factory=list)
assumptions: list[Assumption] = field(default_factory=list)
falsifier: Falsifier | None = None
signed_by: str | None = None

@property
def seam(self) -> str:
    return f"{self.src} -> {self.dst}"

def _dt(v) -> datetime:
d = v if isinstance(v, datetime) else datetime.fromisoformat(str(v).replace("Z", "+00:00"))
return d if d.tzinfo else d.replace(tzinfo=timezone.utc)

def parse(doc: dict) -> Handoff:
h = doc["handoff"]
f = h.get("falsifier")
return Handoff(
src=h["from"],
dst=h["to"],
claim=h["claim"],
evidence=[Evidence(e["kind"], e["ref"], _dt(e["ran"]), e["result"])
for e in h.get("evidence", [])],
assumptions=[Assumption(a["id"], a["text"], a["owner"],
a["expires"], bool(a.get("verified", False)))
for a in h.get("assumptions", [])],
falsifier=Falsifier(f["condition"], f["signal"], f["pages"]) if f else None,
signed_by=h.get("signed_by"),
)

def validate(h: Handoff, now: datetime | None = None) -> list[str]:
"""Return a list of violations. Empty list means the handoff may cross."""
now = now or datetime.now(timezone.utc)
today = now.date()
v: list[str] = []

# Rule 0. A claim has to be capable of being false.
if not h.claim or len(h.claim.split()) < 5:
    v.append("claim is absent or too vague to be falsified")

# Rule 1. Evidence with a stale timestamp is not evidence.
if not h.evidence:
    v.append("no evidence attached")
for e in h.evidence:
    if e.kind not in EVIDENCE_KINDS:
        v.append(f"evidence '{e.ref}' has unknown kind '{e.kind}'")
    if e.result != "pass":
        v.append(f"evidence '{e.ref}' did not pass (result={e.result})")
    age = now - e.ran
    if age > MAX_EVIDENCE_AGE:
        v.append(f"evidence '{e.ref}' is {age.days}d {age.seconds // 3600}h old, "
                 f"limit is {MAX_EVIDENCE_AGE}")

# Rule 2. An unverified assumption cannot cross a boundary unsigned.
for a in h.assumptions:
    if a.expires < today:
        v.append(f"assumption {a.id} expired on {a.expires}")
    if not a.verified and not h.signed_by:
        v.append(f"assumption {a.id} is unverified and the handoff is unsigned")

# Rule 3. A claim with no falsifier is not a claim.
if h.falsifier is None:
    v.append("no falsifier: nothing here can be proven wrong in production")
elif not h.falsifier.pages:
    v.append("falsifier names no one to page")

return v

def expand(patterns: list[str]) -> list[Path]:
files: list[Path] = []
for pat in patterns:
files.extend(sorted(Path().glob(pat)) if any(c in pat for c in "*?[")
else [Path(pat)])
return files

def main(patterns: list[str]) -> int:
failed = 0
for f in expand(patterns):
h = parse(yaml.safe_load(f.read_text()))
problems = validate(h)
if problems:
failed += 1
print(f"BLOCKED {h.seam} ({f})")
for x in problems:
print(f" {x}")
else:
print(f"ok {h.seam} ({f})")
return 1 if failed else 0

if name == "main":
sys.exit(main(sys.argv[1:] or ["seams/*.yaml"]))

Running it

$ python handoff.py "seams/*.yaml"
ok architecture -> backend (seams/03_architecture-to-backend.yaml)
BLOCKED integration -> deployment (seams/07_integration-to-deployment.yaml)
evidence 'terraform validate infra/events' is 9d 0h old, limit is 1 day, 0:00:00
assumption A-220 expired on 2026-08-01
assumption A-220 is unverified and the handoff is unsigned
no falsifier: nothing here can be proven wrong in production
$ echo $?
1

The blocked seam is a real pattern, not a contrived one. In an August 2026 text-to-Terraform study, a model reached 77.8 percent on terraform validate with zero Checkov compliance (arXiv 2608.02672). Syntactically perfect, structurally indefensible. A gate that only reads result: pass would have let that through. A gate that also checks freshness and demands a falsifier does not.

Wiring it into CI

`# .github/workflows/handoff.yml
name: handoff gate
on: [pull_request]

jobs:
seams:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install pyyaml
- name: Validate every seam
run: python handoff.py "seams/*.yaml"`

Two configuration decisions worth making deliberately.

MAX_EVIDENCE_AGE defaults to 24 hours. Set it to whatever your slowest evidence-producing job takes, plus headroom. Too tight and you will re-run suites for no reason; too loose and the rule stops meaning anything.

signed_by should be a real identity your CI can attribute, not a free-text string. Wire it to the commit author or an OIDC subject. A signature nobody can trace is a comment.

Measuring whether it worked

Add a metric, not a vibe.

Seam defect rate is the share of handoffs whose claim was later contradicted downstream. Count a claim as falsified when a later stage, a test, a reviewer, or production disagreed with it.

Two-week protocol, no new tooling:

Week 1. Change nothing. Log the claim at every handoff and whether anything downstream contradicted it. That is your baseline. I would expect most teams between 15 and 30 percent. Under 5 percent usually means your logging is missing failures.
Week 2. Add clauses 3 and 4 (assumptions and falsifier) to your three busiest seams. Skip the evidence harness for now, it is the expensive one. Measure the same rate, plus where the false claim was caught: at the boundary, or three stages later.

Decide the success bar first. Mine is a third fewer seam defects and detection moving to the boundary. Decide the failure bar too: if the ledgers fill with boilerplate and nothing moves, drop it.

What this does not catch

Rule 0 in the validator checks that a claim is at least five words. That is a heuristic and it is weak. "The events module is production ready" passes it and is a terrible claim. No static check can tell you whether a sentence is falsifiable; a human reviewing the contract has to. If you want the gate to do more here, the honest options are a lint list of banned vague words or a required link to an acceptance criterion, not a cleverer regex.

It also does not verify that the evidence actually tests the claim. ref is a string. Pointing it at an unrelated passing test satisfies the gate. Reviewers still matter; the gate just stops the boring failures so reviewers can spend attention on the interesting one.

And it adds friction. That is the point, but it is a real cost and you should expect pushback in week one. In my experience the vague-claim problem is the one that survives every gate I have built, and I am not sure it is solvable in code at all.

Why bother

Two measurements convinced me this is worth the friction.

The first is a comparison at matched compute. Stanford researchers held thinking tokens constant across four models and compared single-agent against sequential multi-agent: 0.418 against 0.379 at a 1,000-token budget, 0.427 against 0.386 at 5,000 (arXiv 2604.02460, 2 April 2026). Adding specialist agents per stage does not help. Adding structure at the boundaries between them might.

The second is where the failures actually live. Berkeley's MAST taxonomy annotated 1,600+ traces across seven frameworks with inter-annotator agreement of 0.88 and found fourteen failure modes in three categories. Two of the three are specification issues and inter-agent misalignment (arXiv 2503.13657).

The stages are getting better on their own. The seams are not, because nobody is scored on them.

Full code, including the two example seam files: github.com/[your-repo]/handoff-contracts

If you run the two-week test, I would like to know your number.