Cap the Retry: A Fail-Closed Loop Budget Checklist

If an agent can retry a write, three numbers must live in the repo before you merge: a hard iteration cap, a wall-clock cap, and a side-effect budget. Missing any one of them is a fail-closed event, not a warning. Soft limits in a README do not count.

You already review the prompt. You already argue about the model. That is not the production problem. The production problem is an unbounded loop that talks to a queue, a billing API, or a ticket system and then tries again.

Why this checklist exists

Agent loops fail in a boring way. They do not always hallucinate a function name. They retry.

A tool call times out. The process dies mid-JSON. The orchestrator restarts the same step. You now have two refunds, two deploys, or two “urgent” issues with the same title. The model did not need to be clever. It needed a ceiling.

If you cannot point at the ceiling in git, you do not merge.

What “unbounded” looks like in a PR

Scan the diff for these shapes. Any one of them is enough to block.

• while True / for _ in range(999) around a tool dispatcher
• retries with no idempotency key on a non-GET side effect
• a timeout that only lives in a chat message, not in config
• a “max steps” comment that the runtime never reads
• a cron or webhook that re-enters the same agent with no run id

You do not need a research paper to classify this. You need a file the CI job can parse.

The merge rule

Treat loop configuration as a production contract. The PR must add or update that contract. The checker must fail closed. Reviewers do not get to “remember to watch it in staging.”

Copy this as the team rule:

1. Iteration cap — integer, required, runtime-enforced, below the team ceiling.
2. Wall-clock cap — seconds, required, includes tool I/O, not just model time.
3. Side-effect budget — per non-idempotent tool, required, counted on attempt not on success.
4. Retry identity — every retry of a write carries a stable idempotency key.
5. Dry-run fixture — one recorded transcript that hits the caps without touching prod.

If the evidence is missing, the gate returns non-zero. Warnings are for linters. This is a merge gate.

Gate 1: Iteration cap

An iteration is one model turn plus the tool batch it emitted. Say that in the contract. Do not let “step” mean three different things in three files.

Fail closed when:

• max_iterations is absent, zero, or above the team ceiling
• the runtime reads a different key than the contract (MAX_STEPS vs max_iterations)
• the cap is only applied in “strict mode”

Evidence you attach to the PR:

• the contract file path
• the runtime log line that prints the remaining iterations
• a unit test that asserts the loop stops on the cap, not on an exception

A cap that is not tested is a comment.

Gate 2: Wall-clock cap

Model latency is not the budget. Tool I/O is the budget. A “30s LLM timeout” with a five-minute HTTP client is how you page the on-call.

Fail closed when:

• there is no max_wall_clock_seconds
• the HTTP client timeout is greater than the remaining wall clock
• a child process can outlive the parent agent

You want one clock. Process start is t0. Every tool call checks now - t0. When the remaining time cannot cover the next client timeout, you stop. You do not start a write you cannot bound.

Gate 3: Side-effect budget

Reads can be chatty. Writes cannot. Classify every tool as read, idempotent_write, or unsafe_write. Only the last two consume the side-effect budget, and unsafe_write consumes it on attempt.

That last clause matters. A timeout after the server applied the write still spent the budget. If you count only HTTP 200, you will double-apply on retry.

Fail closed when:

• a tool that creates, charges, pages, deploys, or emails is not classified
• unsafe_write has no budget
• the budget resets inside a single run
• classification lives in prose instead of the contract

Gate 4: Idempotency keys on retries

Retries without identity are duplicate work. You generate the key before the first attempt and reuse it. The key is a function of run_id + step_name + input_hash. It is not a random UUID minted inside the retry loop.

Fail closed when:

• unsafe_write retries with a fresh UUID
• the provider’s idempotency header is optional
• a crashed worker restarts the step with a new run_id and no recovery map

If the downstream API has no idempotency support, the tool is not retryable. Put it behind a human approval or a single-shot executor. Do not pretend a sleep-and-repeat loop is safe.

Gate 5: Dry-run fixture must pass

A contract you never execute is a wish. You keep one fixture that drives the real loop code against recorded tool responses. The fixture must prove three stops: iteration cap, wall-clock cap, and side-effect budget. All three. One happy-path transcript is not enough.

Fail closed when:

• there is no fixture path in the contract
• the fixture talks to a live network
• only the happy path is asserted
• the fixture is skipped on CI because “it needs a GPU”

The dry-run can run on a small CPU box. That is the point.

Copy-paste contract

Commit this as agent/loop-budget.yml. Keep it boring. Boring parses.

# agent/loop-budget.yml
version: 1
team_ceilings:
  max_iterations: 8
  max_wall_clock_seconds: 90
  max_unsafe_writes: 2
run:
  max_iterations: 6
  max_wall_clock_seconds: 60
  max_unsafe_writes: 1
  dry_run_fixture: tests/fixtures/loop_budget_dryrun.jsonl
tools:
  search_docs:
    class: read
  upsert_ticket:
    class: idempotent_write
    idempotency_header: Idempotency-Key
  charge_customer:
    class: unsafe_write
    retryable: false
  page_oncall:
    class: unsafe_write
    retryable: false

If a new tool lands without a class, the checker fails. Unknown is not read.

The checker

This script is the artifact. Run it locally, then in CI. It does not call a model. It refuses to guess.

#!/usr/bin/env python3
"""Fail closed if the agent loop contract is missing, over-ceiling, or untested."""
from __future__ import annotations

import json
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    sys.stderr.write("pip install pyyaml\n")
    sys.exit  = 2
    raise SystemExit(2)

ROOT = Path(__file__).resolve().parents[1]
CONTRACT = ROOT / "agent" / "loop-budget.yml"
ALLOWED = {"read", "idempotent_write", "unsafe_write"}


def die(msg: str) -> None:
    sys.stderr.write(f"FAIL-CLOSED: {msg}\n")
    raise SystemExit(1)


def load_contract() -> dict:
    if not CONTRACT.is_file():
        die(f"missing {CONTRACT}")
    data = yaml.safe_load(CONTRACT.read_text()) or {}
    if data.get("version") != 1:
        die("unsupported contract version")
    return data


def require_int(node: dict, key: str, hi: int) -> int:
    if key not in node:
        die(f"missing {key}")
    val = node[key]
    if not isinstance(val, int) or isinstance(val, bool) or val < 1:
        die(f"{key} must be a positive int")
    if val > hi:
        die(f"{key}={val} exceeds team ceiling {hi}")
    return val


def check_tools(tools: dict) -> None:
    if not tools:
        die("tools map is empty")
    for name, spec in tools.items():
        if not isinstance(spec, dict):
            die(f"tool {name} is not a map")
        klass = spec.get("class")
        if klass not in ALLOWED:
            die(f"tool {name} has unknown class {klass!r}")
        if klass == "idempotent_write" and not spec.get("idempotency_header"):
            die(f"tool {name} retries writes without an idempotency header")
        if klass == "unsafe_write" and spec.get("retryable", False):
            die(f"tool {name} is unsafe_write and must not be retryable")


def check_fixture(path_str: str, tools: dict) -> None:
    path = ROOT / path_str
    if not path.is_file():
        die(f"dry-run fixture missing: {path}")
    stops = {"iteration_cap": 0, "wall_clock_cap": 0, "side_effect_budget": 0}
    unsafe = {n for n, s in tools.items() if s.get("class") == "unsafe_write"}
    for line_no, line in enumerate(path.read_text().splitlines(), 1):
        if not line.strip():
            continue
        try:
            ev = json.loads(line)
        except json.JSONDecodeError as exc:
            die(f"fixture line {line_no}: {exc}")
        kind = ev.get("stop_reason")
        if kind in stops:
            stops[kind] += 1
        if ev.get("network") is True:
            die(f"fixture line {line_no} sets network=true")
        tool = ev.get("tool")
        if tool in unsafe and ev.get("attempted") and ev.get("retry_with_new_key"):
            die(f"fixture line {line_no} retries unsafe_write with a new key")
    missing = [k for k, n in stops.items() if n < 1]
    if missing:
        die("fixture never demonstrated stops: " + ", ".join(missing))


def main() -> None:
    data = load_contract()
    ceilings = data.get("team_ceilings") or {}
    run = data.get("run") or {}
    tools = data.get("tools") or {}
    hi_iter = require_int(ceilings, "max_iterations", 10**6)
    hi_wall = require_int(ceilings, "max_wall_clock_seconds", 10**6)
    hi_writes = require_int(ceilings, "max_unsafe_writes", 10**6)
    require_int(run, "max_iterations", hi_iter)
    require_int(run, "max_wall_clock_seconds", hi_wall)
    require_int(run, "max_unsafe_writes", hi_writes)
    fixture = run.get("dry_run_fixture")
    if not fixture:
        die("run.dry_run_fixture is required")
    check_tools(tools)
    check_fixture(str(fixture), tools)
    print("loop-budget: ok")


if __name__ == "__main__":
    main()

Install and run:

pip install pyyaml
python tools/check_loop_budget.py
echo $?
# 0 means the contract and fixture both exist and close the loop.

Label the snippet as a starting checker, not a production agent. It does not execute tools. It only refuses missing evidence.

Fixture shape

Keep the dry-run as JSONL so you can grep it in review.

{"stop_reason":"iteration_cap","iterations":6,"network":false}
{"stop_reason":"wall_clock_cap","elapsed_ms":60010,"network":false}
{"stop_reason":"side_effect_budget","tool":"charge_customer","attempted":true,"retry_with_new_key":false,"network":false}

Three lines. Three stops. If a later PR deletes a line to “clean up noise,” CI goes red. That is the desired failure.

CI gate

Wire it so main cannot move without the checker. Example GitHub Actions job:

name: loop-budget
on:
  pull_request:
    paths:
      - "agent/**"
      - "tools/check_loop_budget.py"
      - "tests/fixtures/loop_budget_dryrun.jsonl"
jobs:
  fail-closed:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pyyaml
      - run: python tools/check_loop_budget.py

Path filters are optional. If your agents live all over the repo, drop the paths: block and run it on every PR. Cheap checks should be loud.

Decision table

Use this in review. If two people disagree, they are arguing the table, not vibes.

Signal in the PR	Evidence required	Merge if missing?
New tool that writes	class + budget impact	No
Retry wrapper added	Stable idempotency key	No
Timeout only on the model client	Wall-clock covers tools	No
max_iterations raised	Still <= team_ceilings + fixture updated	No
Fixture talks to the network	Recorded responses only	No
Unsafe write marked retryable	Redesign or human approval	No
Docs say “be careful”	None. Docs are not a gate	No

Print the table in the PR template if your team skips YAML. The checker still wins.

Where a free model box fits

You still need a place to exercise the real loop against the fixture when the unit checker is not enough. That is a dry-run host, not a production identity.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

If you want a throwaway environment that is not your laptop and not prod, MonkeyCode’s free model access and free server option can host that dry-run. Keep the same contract file. Point the runtime at recorded tools first. Only then let a free model fill the “what would the next thought be” slot. Do not point that box at live billing APIs. The gate above exists so a convenient model host cannot become an unbounded worker.

That is optional. The checklist stands if you never touch the product.

What this does not prove

This gate does not prove the agent is correct. It does not prove the model followed policy. It does not prove the tool schemas are right. It only proves the loop cannot run forever, cannot ignore the clock, and cannot retry unsafe writes as if they were reads.

It also does not replace:

• authz on the tools themselves
• egress allowlists
• secret scanning of prompts and traces
• a rollback plan for the writes that did land

If you already have those gates, keep them. Stack this one next to them. Do not fold everything into a single “agent linter” that nobody trusts.

Who should skip this

Do not use this approach if any of the following is true:

• the “agent” is a single model call with no tools and no retry
• a cluster autoscaler already kills the job at a hard wall clock and the job cannot emit side effects after SIGTERM
• you are in a notebook proving an idea, with credentials that cannot reach prod
• every write already goes through a human approval queue with no automatic retry

If you are in that last bucket, a YAML contract is ceremony. Stay there until someone adds a retry.

Ship the ceiling

Put agent/loop-budget.yml in the PR that introduces the loop, not in a follow-up. Run the checker on that PR. Attach the fixture that demonstrates all three stops. If a reviewer says “we will monitor it,” that is a no.

Unbounded retries are not an AI problem. They are a missing number in git. Cap the retry, or do not merge.