I Scored Retry Loops. Boundedness Was Optional.

Boundedness failed first. Not the model. Not the schema. The ceiling. I stopped asking whether an agent was "actually thinking" and started asking a dumber question that still draws blood. Does this while ever stop if the tool hangs?

That question is unromantic. It is also the one that keeps a shared box from turning into a heat lamp. I wanted a number I could re-run on Monday, not a vibe from a chat transcript that already flattered me.

Most agent retry code I read is a loop wearing a trench coat. You know the coat. try, except, sleep, continue, maybe a comment that says "be resilient." Resilience without a budget is just an unbounded bill. Did the author cap attempts? Did they cap wall time? Or did they only cap their optimism?

I built a scorer. Not a leaderboard. A lint with opinions. It reads Python, walks the AST, and grades a retry helper the way I wish code review would: boundedness first, poetry never. The artifact is the point. If you strip every product name out of this article, you should still be able to save the files and get the same integers.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

When I need a candidate loop I do not want to hand-write at 11pm, I ask a free model through MonkeyCode. Then I drop the output into fixtures/ and refuse to trust it until the scorer prints a score. The free server option is not a mascot for that step. It is the second half of the experiment, the half where time.sleep meets scheduling noise. My laptop is a terrible witness. It is too fast, and it likes me.

The rubric I actually encode

I do not grade "quality of reasoning." I grade whether a stop condition exists in the artifact you were about to deploy. Those are different papers. One of them is a keynote. The other one is a unit test.

A retry helper starts at zero. I add points for an attempt cap, a wall-clock timeout, backoff, jitter, and an idempotency key traveling with the request. I subtract points for while True with no break budget, and for sleep that cannot see a deadline. Is that harsh? Good. A loop that can outlive your patience is not robust. It is a fuse with the coating peeled off.

Save this as loop_hygiene.py. It is the whole static half.

#!/usr/bin/env python3
"""Score retry-loop hygiene from Python source. Fixture-calibrated, not a bake-off."""
from __future__ import annotations

import ast
import json
import sys
from dataclasses import asdict, dataclass, field
from pathlib import Path

ATTEMPT_NAMES = {"max_attempts", "max_retries", "retries", "attempts", "n_tries"}
TIMEOUT_NAMES = {"timeout", "deadline", "max_seconds", "wall_timeout", "budget_s"}
BACKOFF_NAMES = {"backoff", "backoff_s", "delay", "base_delay"}
JITTER_NAMES = {"jitter", "jitter_s", "jitter_ratio"}
IDEM_NAMES = {"idempotency_key", "idempotency", "request_id", "dedupe_key"}


@dataclass
class LoopScore:
    path: str
    has_max_attempts: bool = False
    has_timeout: bool = False
    has_backoff: bool = False
    has_jitter: bool = False
    has_idempotency: bool = False
    unbounded_while: bool = False
    sleep_without_budget: bool = False
    score: int = 0
    notes: list[str] = field(default_factory=list)


class HygieneVisitor(ast.NodeVisitor):
    def __init__(self) -> None:
        self.names: set[str] = set()
        self.unbounded_while = False
        self.sleep_calls = 0
        self.breaks_in_loop = 0

    def visit_Name(self, node: ast.Name) -> None:
        self.names.add(node.id)
        self.generic_visit(node)

    def visit_While(self, node: ast.While) -> None:
        constant_true = (
            isinstance(node.test, ast.Constant) and node.test.value is True
        ) or (
            isinstance(node.test, ast.Constant) and node.test.value == 1
        )
        if constant_true:
            self.unbounded_while = True
        for child in ast.walk(node):
            if isinstance(child, ast.Break):
                self.breaks_in_loop += 1
        self.generic_visit(node)

    def visit_Call(self, node: ast.Call) -> None:
        func = node.func
        name = ""
        if isinstance(func, ast.Attribute):
            name = func.attr
        elif isinstance(func, ast.Name):
            name = func.id
        if name in {"sleep", "usleep"}:
            self.sleep_calls += 1
        self.generic_visit(node)


def score_source(path: Path, src: str) -> LoopScore:
    tree = ast.parse(src)
    v = HygieneVisitor()
    v.visit(tree)
    row = LoopScore(path=str(path))
    row.has_max_attempts = bool(ATTEMPT_NAMES & v.names)
    row.has_timeout = bool(TIMEOUT_NAMES & v.names)
    row.has_backoff = bool(BACKOFF_NAMES & v.names)
    row.has_jitter = bool(JITTER_NAMES & v.names)
    row.has_idempotency = bool(IDEM_NAMES & v.names)
    row.unbounded_while = v.unbounded_while and v.breaks_in_loop == 0
    budget = row.has_max_attempts or row.has_timeout
    row.sleep_without_budget = v.sleep_calls > 0 and not budget

    n = 0
    if row.has_max_attempts:
        n += 2
        row.notes.append("+2 attempt cap")
    if row.has_timeout:
        n += 2
        row.notes.append("+2 wall clock")
    if row.has_backoff:
        n += 1
        row.notes.append("+1 backoff")
    if row.has_jitter:
        n += 1
        row.notes.append("+1 jitter")
    if row.has_idempotency:
        n += 2
        row.notes.append("+2 idempotency key")
    if row.unbounded_while:
        n -= 3
        row.notes.append("-3 unbounded while")
    if row.sleep_without_budget:
        n -= 2
        row.notes.append("-2 sleep with no ceiling")
    row.score = n
    return row


def main(argv: list[str]) -> int:
    root = Path(argv[1] if len(argv) > 1 else "fixtures")
    rows = []
    for path in sorted(root.glob("*.py")):
        rows.append(score_source(path, path.read_text(encoding="utf-8")))
    print(json.dumps([asdict(r) for r in rows], indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Run it like a test, not like a demo.

mkdir -p fixtures
python3 loop_hygiene.py fixtures/

If that command prints nothing, you have an empty folder, not a passing grade. Empty is not resilient either.

Four fixtures, four numbers I can defend

I calibrated the scorer on four files I wrote on purpose. This is not a claim about any named model. It is a claim that the ruler does not flop when I bend it. Two fixtures are the "looks fine in a PR" loops a chat window emits when you say make it robust. Two are loops I would actually leave under a cron that can page me.

fixtures/a_trench_coat.py is the coat.

import time

def call_tool(url):
    while True:
        try:
            return fetch(url)
        except Exception:
            time.sleep(1)

fixtures/b_range_but_naked.py looks bounded until you notice the request can be charged twice.

import time

def call_tool(url):
    max_attempts = 5
    for _ in range(max_attempts):
        try:
            return fetch(url)
        except Exception:
            time.sleep(0.5)
    raise RuntimeError("gave up")

fixtures/c_budget.py finally admits time exists.

import random, time

def call_tool(url, timeout=8.0):
    max_attempts = 5
    backoff = 0.2
    deadline = time.monotonic() + timeout
    for attempt in range(max_attempts):
        if time.monotonic() >= deadline:
            raise TimeoutError("wall clock")
        try:
            return fetch(url)
        except Exception:
            jitter = random.random() * 0.05
            sleep_for = min(backoff, max(0.0, deadline - time.monotonic()))
            time.sleep(sleep_for + jitter)
            backoff *= 2
    raise RuntimeError("attempts exhausted")

fixtures/d_idempotent.py is the one I would actually argue for in review.

import random, time, uuid

def call_tool(url, timeout=8.0, idempotency_key=None):
    max_attempts = 5
    backoff = 0.2
    jitter = 0.05
    deadline = time.monotonic() + timeout
    key = idempotency_key or str(uuid.uuid4())
    for attempt in range(max_attempts):
        if time.monotonic() >= deadline:
            raise TimeoutError("wall clock")
        try:
            return fetch(url, headers={"Idempotency-Key": key})
        except Exception:
            sleep_for = min(backoff, max(0.0, deadline - time.monotonic()))
            time.sleep(sleep_for + random.random() * jitter)
            backoff *= 2
    raise RuntimeError("attempts exhausted")

On my machine, against those four files, the scorer prints scores of -5, 2, 6, 8. Read that again. The trench coat is not "a bit sloppy." It is negative. The range-loop that reviewers like is a 2 because it remembered max_attempts and then immediately forgot time, jitter, and duplicate side effects. The budgeted loop is a 6. The idempotent one is an 8. That spread is the experiment working. If your ruler cannot tell a fuse from a space heater, you are collecting anecdotes.

Would I ship a 2? Not on a box I do not babysit. Would I call an 8 production-ready? Still no. An AST visitor cannot see a non-idempotent POST hiding behind a pretty header name. The number is a gate, not a medal.

The prompt I actually paste

When I bother a free model, I do not ask it to "write a robust agent." That prompt is how you get the trench coat. I ask for a function with a contract the scorer can see.

Write a Python function call_tool(url, timeout=8.0, idempotency_key=None)
that retries a failing fetch(). Requirements:
- max_attempts is an int cap
- timeout is a wall-clock budget using time.monotonic()
- exponential backoff plus jitter
- send Idempotency-Key on every attempt
- no while True
Return only the function.

Then I save whatever came back as fixtures/model_candidate.py and run the same command. I do not read the prose around the code first. The integer first. If that sounds rude, ask yourself why a retry helper deserves manners that a unit test does not.

Did the model add jitter, or did it just sleep(1) in a nicer jacket? Did it name timeout and then never consult a clock? Names are cheap. The visitor is cheaper. That is the whole trick.

The half my laptop keeps lying about

Static scores catch missing ceilings. They do not catch a ceiling that exists on paper and then loses a fight with real delay. So I added a dynamic probe. It is deliberately ugly. A mock tool returns after 0.25s, then 1s, then 3s, then it never returns. The loop under test has to respect max_attempts and a wall clock, and the process has to exit.

#!/usr/bin/env python3
"""Dynamic probe. Label: run this; do not treat my laptop timings as yours."""
import threading
import time
from http.server import BaseHTTPRequestHandler, HTTPServer

DELAYS = [0.25, 1.0, 3.0, 999.0]
HITS = {"n": 0}

class SlowTool(BaseHTTPRequestHandler):
    def do_GET(self):
        i = min(HITS["n"], len(DELAYS) - 1)
        HITS["n"] += 1
        time.sleep(DELAYS[i])
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"ok")

    def log_message(self, fmt, *args):
        return

def main() -> None:
    server = HTTPServer(("127.0.0.1", 8765), SlowTool)
    t = threading.Thread(target=server.serve_forever, daemon=True)
    t.start()
    deadline = time.monotonic() + 10.0
    # Import YOUR candidate here and call it against http://127.0.0.1:8765/
    # If this process is still alive past deadline, the loop has no ceiling.
    while time.monotonic() < deadline:
        time.sleep(0.05)
    server.shutdown()
    print({"hits": HITS["n"], "exited_before_watchdog": True})

if __name__ == "__main__":
    main()

On my laptop the 3s delay is a polite pause. On a shared free server, that pause collides with other people's jobs and whatever watchdog the box actually has. If your stop condition is "I got bored watching the spinner," the server will not get bored for you. That is why the free server belongs in the method instead of in a slogan. Latency is a witness. A quiet SSD is a friend of the defendant.

I do not publish a bake-off table for that half, because I will not invent a model name, a hardware spec, or a permanence claim. The protocol is the result. You run the candidate against the mock. Either the process exits before the 10s watchdog, or you just watched an unbounded loop spend someone else's electricity. Which outcome did you want to be surprised by, the integer or production?

What this does not measure

This does not measure whether a model is "better at coding than most developers." I have no idea how you would even sample that sentence without lying to yourself. It does not measure MCP servers, community knowledge bases, or whether agents are secretly if-statements. Plenty of if-statements are honest. The dishonest ones are the loops that refuse to be if-statements with a counter.

The AST scorer is a flashlight. It will miss retries hidden in exec, in a decorator I did not name, or in a subprocess that relaunches itself because someone thought that was architecture. Free model output shifts. Free servers are not SLOs. If you need pinned model identity, reserved CPUs, or a vendor contract that names latency, do not use this workflow. Use a paid, named endpoint and a box you can SSH into without guessing who else is on the metal.

Who should skip it: anyone shipping billing, healthcare, or a retry around a non-idempotent charge. A hygiene score of 8 is not a PCI audit. Also skip it if you will not read the generated loop. A scorer you ignore is just another dashboard, and dashboards do not page you until the invoice does.

I still run the scorer before I argue about "agent quality." Boundedness is optional in chat. It is not optional in a process table. If you want the dynamic half to be ugly on a machine that is not your laptop, I parked that probe on MonkeyCode's free server and let the delays stay dishonest. Steal the harness first. Fight me on the rubric second.