Retries Are a Token Multiplier: A Ledger-First Test Plan for Agent Budgets

An agent loop that succeeds on the first attempt looks cheap, and that is exactly why budget surprises happen. The expensive path is the one where the transport times out, the client retries, and the entire context is resent to a model that already answered. Those retried tokens do not produce a better answer; they only reproduce the same request. Most agent write-ups focus on capability, while the retry multiplier sits quietly outside the measurement.

Agent automation and agent security posts keep climbing the DEV front page this week, and almost none of them show a token ledger. That gap is the useful place to work. Below is an illustrative plan: what to instrument, what faults to inject deliberately, and how to verify that your stated budget matches what you actually spend.

Four places an agent token budget leaks

Before writing any code, name the leak you expect to measure. A budget you cannot attribute to a specific behaviour is a number you will re-derive by guesswork every month.

• Retried prompts. Each retry usually resends the full prompt, and depending on the provider a failed attempt may still consume prompt tokens. Check your provider's billing rules rather than assuming either behaviour.
• Tool output growth. If a tool returns raw JSON on the first attempt and raw JSON plus debug logs on the second, the next prompt is larger for reasons unrelated to the task.
• Truncated completions. A stream cut off by your client timeout can leave you billed for partial output and holding no usable answer.
• Fan-out. Ten parallel tasks that each retry twice behave like thirty calls, not the ten you planned for in the cost estimate.

Instrument first: a ledger, not a dashboard

A dashboard aggregates and hides individual attempts; a ledger keeps every attempt as a row so you can audit the retry path later. The smallest useful version records task, attempt number, prompt tokens, completion tokens, and outcome.

# Illustrative only: shapes depend on your client; this was not executed.
from dataclasses import dataclass, field

@dataclass
class Attempt:
    task_id: str
    attempt_no: int
    prompt_tokens: int
    completion_tokens: int
    outcome: str          # ok | timeout | http_429 | http_5xx | truncated

@dataclass
class Ledger:
    rows: list[Attempt] = field(default_factory=list)

    def record(self, a: Attempt) -> None:
        self.rows.append(a)

    def amplification(self, task_id: str) -> float:
        rows = [a for a in self.rows if a.task_id == task_id]
        ok = [a for a in rows if a.outcome == 'ok']
        spent = sum(a.prompt_tokens + a.completion_tokens for a in rows)
        if not ok:
            return float('inf')       # nothing usable; all spend is waste
        useful = ok[0].prompt_tokens + ok[0].completion_tokens
        return spent / useful

The amplification ratio is the single number worth arguing about. A value of 1.0 means retries cost you nothing measurable; 2.4 means the task cost two and a half times its successful call. If it is inf, you paid for attempts and shipped nothing.

A fault-injection test plan you can run in one sitting

This is a plan, not a report. Treat each step as something to observe and verify against your own ledger, and write down expected versus observed before you run anything.

1. Pin one workload. Choose a single prompt, a single tool call, and a fixed max_attempts. Record the token counts of a clean successful call as your baseline row.
2. Make faults deterministic. Wrap the transport in a seeded injector so attempt one raises a timeout and attempt two returns a rate-limit error. Random failures teach you nothing you can compare across runs.
3. Read the ledger. Verify that every injected failure produced a row, and check whether your client counted prompt tokens on attempts that never returned content.
4. Compute amplification. Compare the ratio to your estimate in step one. Any gap is either an accounting bug or a retry path you did not know you had.
5. Repeat with a cold dependency. Point the same client at an endpoint that may pause when idle, and watch whether first-byte latency alone triggers your timeout. Time-to-first-byte and generation time must be measured separately.
6. Verify the stop condition. Confirm which guard fires first — attempts, wall clock, or tokens — and that the ledger shows the exact attempt that crossed the line.

The injector, in about fifteen lines

# Illustrative fault injector: deterministic and seeded, no network egress.
import random

class FlakyTransport:
    def __init__(self, inner, seed=7, fail_plan=('timeout', 'http_429', None)):
        self.inner, self.rng = inner, random.Random(seed)
        self.plan, self.calls = list(fail_plan), 0

    def send(self, request):
        fault = self.plan[self.calls] if self.calls < len(self.plan) else None
        self.calls += 1
        if fault == 'timeout':
            raise TimeoutError('injected: first byte never arrived')
        if fault == 'http_429':
            raise RuntimeError('injected: 429')
        return self.inner.send(request)

Prerequisites are modest: Python 3.10 or newer, a client wrapper you control, and a place to persist the ledger rows so you can compare runs. The code above is illustrative and has not been executed; adapt the response shape to your provider's accounting fields.

The guard that makes a budget real

Attempt limits alone do not bound spend, because a single attempt can be large. A token guard placed next to the retry loop does bound it, and it gives you a clean failure mode instead of a monthly surprise.

# Illustrative. Your client and token accounting will differ.
class BudgetExceeded(RuntimeError):
    pass

def run_task(client, request, ledger, task_id, max_attempts=3, max_tokens=20_000):
    spent = 0
    for attempt_no in range(1, max_attempts + 1):
        try:
            resp = client.send(request)
        except (TimeoutError, RuntimeError) as err:
            cost = request.estimated_prompt_tokens   # count what a failure spends
            spent += cost
            ledger.record(Attempt(task_id, attempt_no, cost, 0, classify(err)))
            if spent > max_tokens:
                raise BudgetExceeded(f'{task_id}: {spent} tokens before any success') from err
            continue
        spent += resp.prompt_tokens + resp.completion_tokens
        ledger.record(Attempt(task_id, attempt_no, resp.prompt_tokens,
                              resp.completion_tokens, 'ok'))
        return resp
    raise BudgetExceeded(f'{task_id}: attempts exhausted')

classify(err) is a helper you write to map exceptions onto your outcome vocabulary. One boundary matters here: if the task has side effects, a retry needs an idempotency key on the write, because a token guard protects your budget and not your data.

Why a free endpoint changes the arithmetic

When nothing sleeps and nothing rate-limits, retries are rare enough to ignore. Free infrastructure inverts that: cold starts, idle pauses, and shared capacity make first-byte latency variable, and a client timeout that was fine against a warm service will now fire regularly. Each of those timeouts is a retry, and each retry is a resent prompt.

That is a feature for the plan above, not a defect to complain about. Step five deliberately uses an endpoint that may pause, because you want to see whether your timeout or your token guard fires first. What you should not do is read a cold-start retry as evidence about model quality, since the model often never saw the request at all.

Where MonkeyCode fits into this walk-through

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is described by its maintainers as an open source project that provides free model access and a free server option, which makes it a reasonable place to run steps one through four of the plan above. The free allowance is a token budget, and the exact current figure belongs in the project's own documentation rather than in a blog post that ages badly, so check the project's page for the number that applies when you read this.

Two honest constraints shaped that recommendation. First, a free server is the right target for fault injection precisely because it is not a latency guarantee, so do not treat your measurements there as production capacity numbers. Second, any free tier deserves a data review before you send real prompts, because your prompts are the input.

Decision table: free tier or paid capacity

Workload	Free tier / free server	Why
Learning an SDK, drafting prompts	Good fit	Failures cost patience, not contracts
Fault-injection rehearsal	Good fit	Cold starts supply realistic timeouts
Batch jobs, restartable and idempotent	Workable with a ledger	Retry amplification is measurable and bounded
User-facing requests with an SLA	Poor fit	Idle pauses become visible latency
Regulated or secret-bearing data	Review first	Data handling matters more than price
Long-running stateful services	Poor fit	A pausing server breaks state assumptions

Limitations, and who should skip this

The ledger measures what your client believes it spent, which may not equal what the provider bills; reconcile against usage records before quoting a ratio to anyone. The injector exercises your retry path, not your provider's real failure modes, and a deterministic plan can miss cascading failures entirely.

Skip this approach entirely if you need contractual latency, guaranteed throughput, or a data-processing agreement before any prompt leaves your network. Also skip it if a single failed write costs more than the tokens involved, because the guard protects budget and not correctness.

Start with one task and one fault plan, then compare the amplification ratio before and after you add the token guard. That comparison is the artifact worth keeping, and it survives whichever endpoint you end up paying for.