Hard-Stop the Loop Before You Touch Free Inference

You should hard-stop an agent before you touch free inference. The prompt cannot kill a retry storm. A missing price signal is the real bug.

Take a side

This is not an argument against free models. It is an argument against unbounded agent loops. You must put the brake in your own client.

Developers still paste stop rules into system prompts. The model can ignore every one of those rules. Your process will not notice until files change.

You keep calling because nothing in the loop stings. Cheap inference hides the cost of another retry. That hidden cost shows up as a dirty worktree.

Why the failure mode shifted

Paid APIs taught you to fear long loops. A rising bill was a crude circuit breaker. You felt the damage in the same working hour.

Free model access removes that pain on purpose. A free server option removes the pain faster. You will retry because each retry looks free.

Agent posts this week keep celebrating extra autonomy. The hard part is still forcing the loop to halt. Assumption-heavy agents burn steps when nothing is priced.

You do not need another glossary of agent terms. You need a kill switch that lives outside the model.

Cheap generation is not cheap cleanup

Industry talk this week worries about AI-made technical debt. Debt arrives faster when a loop can keep writing. Your write budget is a debt budget in disguise.

You do not need a new architecture sermon. You need fewer unattended writes per session. Architecture opinions cannot outrun a loop with no ceiling.

Three budgets that actually halt a loop

One budget is never enough for an agent. Split the cap across steps, writes, and time. Fail closed on the first ceiling you hit.

Step budget

Count every model call as one spent step. Count tool retries as steps, not as free extras. Eight steps is plenty for a local coding task.

Write budget

Reads can be noisy without wrecking the repo. Writes need a tiny and explicit ceiling. Two writes per task is a sane default.

Guessing can stay in the think path. Mutating the tree on a guess is not allowed.

Wall-clock budget

Tokens are not the only runaway resource. A stuck tool call can pin a free server. Forty-five seconds is enough to prove a local patch.

If the clock expires, you stop without extra pleas. Do not ask the model whether more time would help. The model will almost always say yes.

How retry storms start

The model fails a patch and asks to try again. Your wrapper says yes because the call looks cheap. The same broken tool schema comes back three times.

Nothing in the prompt notices the repeated failure. A step counter notices on the next checkpoint. That is why the counter is not optional.

Map every tool to an action class

Classify each tool before you expose it to the loop. If you cannot classify it, do not register the tool. Unclassified tools are how silent writes sneak in.

1. think produces a plan and must have no side effects.
2. read opens files, searches, or runs dry tests only.
3. write patches files, installs packages, or mutates state.
4. retry repeats a failed tool with the same intent.

Reads still consume steps, because steps are the loop. Only write increments the write ceiling. retry should get harsher as you near the step cap.

The gate belongs in code

Do not negotiate with the model about stopping. Raise an exception in your process and halt. The snippet below is a local, copy-paste gate.

Treat the helper as a starting module, not a vendor SDK.

# agent_budget.py
from __future__ import annotations

import json
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Literal


class BudgetExceeded(RuntimeError):
    pass


@dataclass
class AgentBudget:
    max_steps: int = 8
    max_writes: int = 2
    max_seconds: float = 45.0
    ledger_path: Path = Path('agent_ledger.jsonl')
    started_at: float = field(default_factory=time.monotonic)
    steps: int = 0
    writes: int = 0

    def checkpoint(
        self,
        action: Literal['think', 'read', 'write', 'retry'],
        detail: str,
    ) -> None:
        elapsed = time.monotonic() - self.started_at
        self.steps += 1
        if action == 'write':
            self.writes += 1
        record = {
            'step': self.steps,
            'action': action,
            'detail': detail,
            'writes': self.writes,
            'elapsed_s': round(elapsed, 3),
        }
        with self.ledger_path.open('a', encoding='utf-8') as fh:
            fh.write(json.dumps(record) + '\n')
        self._enforce(action, elapsed)

    def _enforce(self, action: str, elapsed: float) -> None:
        if self.steps > self.max_steps:
            raise BudgetExceeded(
                f'steps {self.steps} > {self.max_steps}'
            )
        if self.writes > self.max_writes:
            raise BudgetExceeded(
                f'writes {self.writes} > {self.max_writes}'
            )
        if elapsed > self.max_seconds:
            raise BudgetExceeded(
                f'elapsed {elapsed:.1f}s > {self.max_seconds}s'
            )
        if action == 'retry' and self.steps >= self.max_steps - 1:
            raise BudgetExceeded('retry blocked near step ceiling')

Wire the gate around every tool dispatch. Catch BudgetExceeded and refuse the pending side effect. Log the reason and do not ask the model for mercy.

# proposed loop sketch — run only after the tests below pass
from agent_budget import AgentBudget, BudgetExceeded


def run_task(budget: AgentBudget, tools: dict, intent: str) -> None:
    budget.checkpoint('think', intent[:120])
    try:
        result = tools['read'](intent)
        budget.checkpoint('read', 'tool:read')
        if not result.ok:
            budget.checkpoint('retry', 'tool:read')
            result = tools['read'](intent)
        if result.needs_write:
            budget.checkpoint('write', 'tool:write')
            tools['write'](result.patch)
    except BudgetExceeded as exc:
        raise SystemExit(f'fail closed: {exc}') from exc

What the ledger must record

If you cannot see the path, you cannot kill it. Append one JSON line for every client action. Keep the schema boring, stable, and secret-free.

Required fields:

• step: client counter, not a model claim
• action: think, read, write, or retry
• detail: short text with no secrets
• writes: write count so far
• elapsed_s: seconds since started_at

Do not log raw prompts if they contain secrets. Do not log tokens you did not actually measure. An honest ledger beats a decorative status dashboard.

{"step": 3, "action": "retry", "detail": "tool:read", "writes": 0, "elapsed_s": 4.812}

Tests you run before the first write

These tests are a checklist, not a benchmark. They do not prove a hosted model will behave. They prove your client will refuse to continue.

# test_agent_budget.py
import time
from pathlib import Path

import pytest

from agent_budget import AgentBudget, BudgetExceeded


def test_step_ceiling_fails_closed(tmp_path: Path) -> None:
    budget = AgentBudget(
        max_steps=2,
        max_writes=5,
        max_seconds=30,
        ledger_path=tmp_path / 'l.jsonl',
    )
    budget.checkpoint('read', 'a')
    budget.checkpoint('read', 'b')
    with pytest.raises(BudgetExceeded):
        budget.checkpoint('read', 'c')


def test_write_ceiling_is_stricter_than_reads(tmp_path: Path) -> None:
    budget = AgentBudget(
        max_steps=20,
        max_writes=1,
        max_seconds=30,
        ledger_path=tmp_path / 'l.jsonl',
    )
    budget.checkpoint('read', 'a')
    budget.checkpoint('write', 'b')
    with pytest.raises(BudgetExceeded):
        budget.checkpoint('write', 'c')


def test_retry_near_ceiling_is_blocked(tmp_path: Path) -> None:
    budget = AgentBudget(
        max_steps=3,
        max_writes=5,
        max_seconds=30,
        ledger_path=tmp_path / 'l.jsonl',
    )
    budget.checkpoint('read', 'a')
    budget.checkpoint('read', 'b')
    with pytest.raises(BudgetExceeded):
        budget.checkpoint('retry', 'b-again')


def test_wall_clock_fails_closed(
    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
    budget = AgentBudget(
        max_steps=50,
        max_writes=5,
        max_seconds=0.01,
        ledger_path=tmp_path / 'l.jsonl',
    )
    monkeypatch.setattr(
        time, 'monotonic', lambda: budget.started_at + 1.0
    )
    with pytest.raises(BudgetExceeded):
        budget.checkpoint('think', 'late')

Run the file with pytest before you enable writes. If a test fails, the loop stays in read-only mode. That is the whole point of a client-side brake.

Decision table

Read the table before you wire retries. Die on the first ceiling you actually hit.

Signal	Retry allowed	Client action
Read failed, steps left, no writes yet	Yes, once	Log retry, then read
Read failed, one step left	No	Raise BudgetExceeded
Patch failed after one write	No	Stop and open the ledger
Model asks for one more try	No	Ignore the plea
Wall clock past cap	No	Kill the process
Tool is unclassified	No	Do not register it

You retry only when the budget still has room. You never retry a write that already failed closed. Ambiguous model output is not a reason to spend steps.

Where a free model path fits

You still need somewhere to practice the gate. A local coding agent is useful if the brake is on.

MonkeyCode is an open-source project for coding agents. It offers free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use that path to rehearse the budget, not to skip it. The free access is relevant only as a place to iterate. Your kill switch still lives in the Python process.

Limitations

This gate estimates nothing about true billed tokens. It cannot see rate limits inside a hosted server. It will not protect you from a bad tool implementation.

JSONL logs can leak paths if you are sloppy. Wall-clock caps fight hung subprocesses, not all of them. You still need OS-level timeouts around shell tools.

Do not treat eight steps as a universal constant. Tiny refactors and wide migrations need different ceilings. Change the numbers in code, not in the prompt.

Free model access can change behavior without a local notice. A free server option is not an uptime contract. Pin your tests so a silent change still fails closed.

Who should skip this

Skip this if you already have provider-side hard quotas. Skip this if legal logging rules forbid local JSONL. Skip this for unattended bots that must page a human.

Multi-tenant products need real accounting, not a dataclass. This article is for a single developer loop. It is not a billing system and not a sandbox.

Close

Free inference is a gift that deletes your warning light. You should answer that gift with a harsher client. Pin steps, writes, and time before the first call.

After the tests pass, try MonkeyCode as a practice bench.