Budget the Loop or Don't Dispatch: A Fail-Closed Tool-Call Checklist

If a model can invoke tools, an open-ended loop is a production bug, not a feature. You fail closed on iteration count, wall-clock budget, per-tool timeout, allowlist, and result schema—or you do not dispatch the first call.

Tool-calling demos make the happy path look finished. The model picks a function. Your runtime executes it. The model “thinks” again. That path hides failure modes unit tests never see: a mutating endpoint retried, a search tool hung, JSON that is almost valid, a loop that burns the afternoon because nobody set max_steps.

You do not need another agent framework to stop that. You need gates that can refuse the next call, plus evidence that they actually fired.

The problem you actually ship

A tool loop is a state machine you did not write. Each step can call a side-effecting API, block on a network you do not own, return text that is not the schema you documented, or ask for another step. Forever.

Green local traces do not prove any of that is bounded. A checklist that cannot fail the run is commentary. It is not a control.

Fail-open here looks friendly. The model gets “one more try.” The hung tool gets a longer timeout. Invalid JSON gets pasted into the next prompt as “context.” That is how a demo becomes an unbounded distributed job.

Four gates, then evidence

Copy these. If a gate cannot produce evidence, it is not a gate. Do not reorder them to be nice to the model.

1. Iteration cap

Gate: steps_used < MAX_STEPS before every model turn and every tool dispatch.

Evidence: a structured log line with run_id, step, and max_steps.

Fail closed: when step >= MAX_STEPS, skip the model, skip tools, return a terminal budget_exceeded error. No hidden extra turn.

2. Wall-clock budget

Gate: now - started_at < MAX_WALL_MS on a monotonic clock.

Evidence: start timestamp on the run record, plus the remaining budget at each step.

Fail closed: expire the run even if the model is mid-token. Cancel in-flight tools you own. Do not start new ones.

3. Per-tool timeout and allowlist

Gate: tool name is in a pinned allowlist, and the call is wrapped in a hard timeout shorter than the remaining wall budget.

Evidence: allowlist version or hash in the run record; timeout value per tool name.

Fail closed: unknown tool name is a hard error, not a prompt to invent another function. Timeout returns a typed error. The model may see that error only if budget remains.

4. Result schema

Gate: tool output is parsed against an explicit schema before it re-enters the model context.

Evidence: schema id, parse ok/fail, byte size after truncation.

Fail closed: invalid JSON, missing required keys, or oversized payloads never reach the next prompt. Raw text is not “close enough.”

Check budget first. Then allowlist. Then invoke under timeout. Then schema. Unknown tool beats timeout. Budget beats schema. You are not scoring elegance. You are stopping the loop.

Copy-paste checklist for the PR that binds tools

Use this on the change that wires tools into a runtime, not on the notebook that demoed them.

1. Named run — every dispatch carries a run_id. No anonymous loops in prod.
2. Hard MAX_STEPS — integer, committed in the same change as the tool bind. Not a dashboard default someone can raise without review.
3. Hard MAX_WALL_MS — includes model time and tool time.
4. Tool allowlist — exact names. Not a regex over the whole registry.
5. Per-tool timeout — shorter than remaining wall budget at dispatch time.
6. Failed steps still count — a timeout or schema miss increments the step counter. Otherwise retries are free and unbounded.
7. Schema on the way back — parse before concatenate. Truncate before parse if you must bound memory.
8. Terminal error types — budget_exceeded, tool_timeout, unknown_tool, schema_invalid stay distinct so operators can grep them.
9. No silent continue — catching a gate error and stuffing it into the prompt as a normal tool result is a bypass. Do not ship it.
10. Human resume is a new run — a retry gets a new run_id and a recorded parent. It is not a hidden extra iteration on the dead run.

Print the evidence or the run does not ship. A green UI screenshot is not evidence.

Example evidence line you can demand in logs:

{
  "run_id": "run-20260923-01",
  "step": 3,
  "max_steps": 8,
  "wall_ms_used": 4120,
  "max_wall_ms": 15000,
  "tool": "search",
  "allowlist_version": "tools-v4",
  "schema_id": "search.hits.v1",
  "gate": "ok"
}

If that object cannot be produced, the gate is theater.

Artifact: a fail-closed dispatcher you can run

The following is a proposal you can execute locally. It is not production telemetry. It does not call a vendor. It shows gate order and fail-closed errors.

# tool_loop_gate.py
from __future__ import annotations

import json
import time
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, Mapping

class GateError(Exception):
    def __init__(self, code: str, message: str) -> None:
        super().__init__(message)
        self.code = code

@dataclass
class ToolSpec:
    name: str
    timeout_s: float
    schema_required_keys: tuple[str, ...]

@dataclass
class RunBudget:
    run_id: str
    max_steps: int
    max_wall_s: float
    started_monotonic: float = field(default_factory=time.monotonic)
    steps_used: int = 0

    def remaining_wall_s(self) -> float:
        return self.max_wall_s - (time.monotonic() - self.started_monotonic)

    def assert_can_step(self) -> None:
        if self.steps_used >= self.max_steps:
            raise GateError(
                "budget_exceeded",
                f"{self.run_id}: max_steps={self.max_steps}",
            )
        if self.remaining_wall_s() <= 0:
            raise GateError(
                "budget_exceeded",
                f"{self.run_id}: wall clock exhausted",
            )

def parse_tool_result(raw: str, spec: ToolSpec) -> Dict[str, Any]:
    try:
        data = json.loads(raw)
    except json.JSONDecodeError as exc:
        raise GateError("schema_invalid", f"{spec.name}: not json ({exc})") from exc
    if not isinstance(data, dict):
        raise GateError("schema_invalid", f"{spec.name}: expected object")
    missing = [k for k in spec.schema_required_keys if k not in data]
    if missing:
        raise GateError("schema_invalid", f"{spec.name}: missing {missing}")
    return data

def dispatch(
    budget: RunBudget,
    allowlist: Mapping[str, ToolSpec],
    tool_name: str,
    invoke: Callable[[], str],
) -> Dict[str, Any]:
    budget.assert_can_step()
    spec = allowlist.get(tool_name)
    if spec is None:
        raise GateError("unknown_tool", f"not in allowlist: {tool_name}")

    timeout = min(spec.timeout_s, max(budget.remaining_wall_s(), 0.0))
    if timeout <= 0:
        raise GateError("budget_exceeded", "no time left for tool")

    # Proposal only: replace with a real kill (subprocess, worker, sandbox).
    deadline = time.monotonic() + timeout
    raw = invoke()
    if time.monotonic() > deadline:
        budget.steps_used += 1
        raise GateError("tool_timeout", f"{tool_name}: exceeded {timeout}s")

    parsed = parse_tool_result(raw, spec)
    budget.steps_used += 1
    return parsed

Dry-run it:

python - <<'PY'
from tool_loop_gate import RunBudget, ToolSpec, dispatch, GateError

allow = {"search": ToolSpec("search", timeout_s=2.0, schema_required_keys=("hits",))}
budget = RunBudget(run_id="run-demo-1", max_steps=2, max_wall_s=5.0)

def ok():
    return '{"hits": []}'

print(dispatch(budget, allow, "search", ok))

try:
    dispatch(budget, allow, "shell", ok)
except GateError as e:
    print(e.code, str(e))

budget.steps_used = 2
try:
    dispatch(budget, allow, "search", ok)
except GateError as e:
    print(e.code, str(e))
PY

You should see one parsed object, then unknown_tool, then budget_exceeded. If a second tool still runs after the cap, the gate is not fail closed.

The timeout in this snippet is a check after return. That is labeled on purpose. A shippable timeout must kill the worker. Do not take the post-hoc check to production by itself.

Decision table

Condition	Evidence you store	Action
tool_name missing from allowlist	name + allowlist version	raise unknown_tool; do not offer extra tools
steps_used >= MAX_STEPS	step counters	raise budget_exceeded; end run
wall clock exhausted	start + now	cancel in-flight work; same budget error
tool exceeds timeout	tool, timeout_s	raise tool_timeout; count the step
payload not JSON / missing keys	schema id	raise schema_invalid; do not stuff raw text into context
all gates pass	run_id, step, latency	increment step; continue

If two rows could apply, pick the stricter one. Do not let the model vote.

Proposed test plan (unexecuted)

Treat these as tests you still have to run in CI. They are not results.

1. Allowlist miss — dispatch rm when the allowlist is {search}. Expect unknown_tool and zero side effects.
2. Step cap — MAX_STEPS=1, two sequential tools. The second call never invokes invoke().
3. Failed steps consume budget — first tool returns invalid JSON; second dispatch is refused when MAX_STEPS=1.
4. Wall clock — inject a clock that jumps past max_wall_s between steps. Expect budget_exceeded.
5. Schema isolation — return not-json from a tool. The next model prompt must not contain that string.
6. Timeout ownership — hang the tool past timeout_s. The worker is killed, not joined.

Wire assertions to process exit codes. A skipped test is an open gate.

Bypasses that look like product behavior

Watch for these in review. They reopen the loop while sounding reasonable.

• Catching GateError and sending str(error) back as a normal tool payload. That teaches the model to keep talking.
• Counting only successful tools. Timeouts then become free retries.
• Raising MAX_STEPS in a config map to “unblock” an incident. That is how the unbounded loop returns.
• Sharing one budget across tenants or across users in the same process. One runaway run should not steal another run’s steps; it also should not inherit a huge shared cap.
• Letting the model add a tool name to the allowlist mid-run. The allowlist is an operator artifact. It is not a conversation.

If you need a longer job, start a new run with a parent id. Do not mutate the cap on a live run.

Where a scratch model path fits

You will want a non-prod loop to prove the gates fire. Do not use the production model endpoint for that. Fake tools, tiny prompts, and the dispatcher above are enough to see unknown_tool and budget_exceeded on purpose.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you need a throwaway place to exercise the same gate against a model that is not your production path, MonkeyCode's free model access and free server option can host that dry run. Keep production credentials out of it. The checklist does not depend on that environment.

Limitations — who should not use this

This dispatcher is not an authorization system. An allowlist of tool names does not replace user-scoped auth, rate limits, or audit logs.

It does not measure answer quality. A loop that stops in three steps can still be wrong. Pair quality checks with a separate eval path; do not overload this runtime gate with scoring.

It is the wrong shape for batch ETL and for human-in-the-loop research that is supposed to run for hours. Those jobs need a workflow engine with checkpoints, not a request-scoped MAX_STEPS=8.

Do not use a post-return timeout as your only kill switch. Do not treat this list as a substitute for idempotency on mutating tools. If a tool can move money, inventory, or identity, bounding the loop is necessary and still insufficient. You still need authz, an idempotency key, and a human approval path you can name in the run record.

Teams that already dispatch tools through a workflow engine with SLAs, kill switches, and stored step evidence can skip this snippet. Use their controls. Do not run a second unbounded loop beside them “just for the agent.”

Ship the bound, not the demo

Tool calling is easy to demo and easy to leave unbounded. Put the four gates in the runtime that actually dispatches. Store the evidence. Fail closed.

If a PR binds a new tool without MAX_STEPS, wall clock, allowlist, timeout, and schema, you do not merge it. The model can wait.