Field Notes: 48 Hours to Prove Your Agent Loop Terminates

Two days is enough to find out whether an agent loop stops when the world refuses to cooperate. Not whether it is smart — whether it ends.

Here is the shape of the failure. A tool returns 200 OK, the model thanks you politely, calls the tool again, and the loop keeps paying for the same fact. Nothing errors. No stack trace, no alarm, no red row in the dashboard. The run just gets slower and more expensive until a human notices.

So the fence cannot live in the error handler, because there is no error. It has to live in the budget, and the budget has to be checked before each call rather than audited after the invoice.

Why step count is the budget you actually control

Token spend is a measurement; step count is a precondition. You learn your token total after the call returns, which means a single huge tool result can blow the budget before any check runs. Step count, wall clock, and repeated-output detection are all knowable before you spend anything on the next turn.

That ordering matters more than the numbers in it. Put the fence where the check is cheap.

# agent_budget.py — stdlib only, Python 3.11+
from __future__ import annotations

import hashlib
import time
from dataclasses import dataclass, field


class BudgetExceeded(RuntimeError):
    """Raised *before* a call is made, never after."""

    def __init__(self, reason: str) -> None:
        super().__init__(reason)
        self.reason = reason


@dataclass
class Limits:
    max_steps: int = 12
    max_prompt_tokens: int = 60_000
    max_completion_tokens: int = 8_000
    max_wall_seconds: float = 900.0
    max_repeats: int = 2          # identical tool results in a row


@dataclass
class Ledger:
    steps: int = 0
    prompt_tokens: int = 0
    completion_tokens: int = 0
    last_fingerprint: str = ""
    repeat_run: int = 0
    started: float = field(default_factory=time.monotonic)

    @property
    def wall_seconds(self) -> float:
        return time.monotonic() - self.started

    def record_tool_result(self, payload: str) -> None:
        fp = hashlib.sha256(payload.encode()).hexdigest()[:16]
        self.repeat_run = self.repeat_run + 1 if fp == self.last_fingerprint else 1
        self.last_fingerprint = fp

    def check(self, limits: Limits) -> None:
        if self.steps >= limits.max_steps:
            raise BudgetExceeded("max_steps")
        if self.prompt_tokens >= limits.max_prompt_tokens:
            raise BudgetExceeded("max_prompt_tokens")
        if self.completion_tokens >= limits.max_completion_tokens:
            raise BudgetExceeded("max_completion_tokens")
        if self.wall_seconds >= limits.max_wall_seconds:
            raise BudgetExceeded("max_wall_seconds")
        if self.repeat_run > limits.max_repeats:
            raise BudgetExceeded("identical_tool_results")

The loop below returns a stop reason instead of throwing. That is deliberate: an exhausted budget is a result, and you want it in the transcript next to everything else that happened.

from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import TimeoutError as FutureTimeout

_POOL = ThreadPoolExecutor(max_workers=4)


def run_tool(fn, args, timeout_s):
    future = _POOL.submit(fn, **args)
    try:
        return future.result(timeout=timeout_s)
    except FutureTimeout:
        return f"error: tool timed out after {timeout_s}s"
    except Exception as exc:  # a raising tool is data, not a crash
        return f"error: {type(exc).__name__}: {exc}"


def run_loop(model, registry, task, limits=Limits()):
    led, transcript = Ledger(), [{"role": "user", "content": task}]
    while True:
        try:
            led.check(limits)
        except BudgetExceeded as stop:
            return {"stop": stop.reason, "ledger": led, "transcript": transcript}

        call = model(transcript)      # {"tool": ..., "args": {...}, "usage": {...}}
        led.steps += 1
        led.prompt_tokens += call["usage"]["prompt"]
        led.completion_tokens += call["usage"]["completion"]

        if call.get("final"):
            return {"stop": "model_finished", "ledger": led, "transcript": transcript}

        name, args = call["tool"], call.get("args", {})
        fn = registry.get(name)
        result = (f"error: unknown tool {name!r}" if fn is None
                  else run_tool(fn, args, timeout_s=min(30.0, limits.max_wall_seconds)))
        led.record_tool_result(result)
        transcript.append({"role": "tool", "name": name, "content": result})

Four stub tools that break a naive loop

You do not need a real task to test the fence. You need tools that misbehave in the four ways tools actually misbehave. Register these in the same registry your production tools live in, and drive the loop with a scripted fake model so the test is deterministic and free.

Stub tool	What it simulates	Stop reason you should see
ok_forever() always returns "ok"	a healthy-looking no-op the model keeps re-calling	identical_tool_results
big_output() returns a few hundred KB of text	log dumps, DOM snapshots, wide query results	max_prompt_tokens
sleep(300)	a hung dependency or a dead endpoint	tool timeout, then max_wall_seconds if the model keeps asking
raise_after_write() writes a file, then raises	partial side effects behind a failed call	model_finished or any budget stop — but the file exists

The last row is the one people skip. A stop reason tells you the loop ended; it does not tell you the world is unchanged.

The 48-hour protocol

Block one, hours 0 to 2, is a unit test of the fence, not of the model. Use the scripted fake, register all four stubs, and confirm you get identical_tool_results in under a minute. If that does not fire, nothing downstream is worth running.

Block two, hours 2 to 12, swaps in a real model against one real task with the stubs still registered. Run it unattended. The question is narrow: is the stop reason one you predicted? An unknown tool stop usually means the model hallucinated a tool name and your registry silently absorbed it. That is a prompt bug wearing a budget costume.

Block three, hours 12 to 24, is the adversarial hour. Inject the hang and the huge output at the point where the loop is most confident, and check that the wall-clock fence fires before your patience does. This is also where you learn whether the tool timeout returns control to the loop or merely returns a string the model politely retries.

Block four, hours 24 to 48, is reconciliation. Replay the transcript against the ledger and check three invariants: step count equals the number of tool results, token totals match what the provider reported for the run, and no budget stop overwrote a final answer that already existed. That third invariant is the interesting one. A loop that had the correct answer at step 9 and kept going to step 12 is a product bug, not a budget bug, and only the transcript can tell you which you have.

What the timeout cannot do

future.result(timeout=...) stops waiting; it does not stop the thread. A CPU-bound tool keeps burning a core after the fence reports a timeout, and four of them will exhaust a small box while your ledger claims everything is fine. If your tools are long-running by nature, run them as subprocesses and kill the process group, or accept that the timeout is a reporting mechanism rather than a control.

The thread pool has the same edge. _POOL.submit without a bound on pending work is an unbounded queue wearing a bounded hat.

Where a free model and a free box change the economics

The harness is provider-agnostic; nothing above assumes a vendor. What makes a 48-hour run a habit instead of a budget decision is not paying for the clock. MonkeyCode's free model access and free server option are the two pieces that let you leave an adversarial loop running overnight and reconcile the ledger in the morning.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I am describing a workflow, not making performance claims. Model availability, request limits, and how long a free server stays up are set by the operator and change, so read the current terms before you plan around them — including any number quoted in a blog post, this one included. The limits in Limits above are deliberately conservative starting points for your fence, not measured thresholds for any provider.

Who should not use this approach

If a tool call mutates production state, a stop reason is not a rollback, and this harness gives you no transaction semantics. If your task legitimately needs hundreds of steps, the step fence is the wrong instrument — you want checkpointing and resumption, not a smaller number. And if you need residency guarantees or an SLA attached to the compute, a free tier is the wrong substrate by definition.

Workload	Reasonable fit for free model access + free server?
Harness and stub-tool development	Yes
Teaching, demos, internal walkthroughs	Yes
Short bounded tasks	Usually
Long unattended runs where an eviction loses state	Not without checkpointing
Anything touching production data, residency, or SLA terms	No

What I would repeat

Three things survive the 48 hours. Check the budget before the call, not after. Fingerprint tool results, because repetition is the failure mode that looks most like success. And reconcile the transcript against the ledger at the end — the fence tells you the loop stopped, and only the reconciliation tells you whether it stopped at the right place.

If you want to falsify your own loop without provisioning anything first, the free server option is the cheapest way to start the clock and let the stubs do the arguing.