Retries Spend the Allowance: A Burn-Rate Ledger for Free Model Access

Illustrative scenario: An extraction job sends one prompt per record. Some records come back as malformed JSON. The wrapper retries up to three times, then marks the record done. The job finishes green and nobody asks how many attempts were billed.

That gap is the subject of this article. Free model access and a free server make attempts cheap to start. They do not make attempts free to repeat.

The operator of MonkeyCode describes free model access and a free server option for the project. Exact limits are not restated here. Quotas and terms change, so read the current terms before you rely on them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The allowance is a burn rate, not a budget line

Four properties decide how fast an allowance drains.

1. Consumption is per attempt, not per success.
2. A failed parse still bills the full prompt.
3. Every retry re-sends the whole context.
4. Agent loops multiply attempts per user action.

So a task that costs one unit in a design doc costs one plus failures in practice. Call that ratio amplification: billed tokens divided by useful tokens. Amplification is the number a feature list never shows.

Step 1: record one row per attempt

Aggregate later, never before. A wrapper that logs only successes cannot compute amplification.

# ledger.py — illustrative only; not executed for this article
from dataclasses import dataclass

@dataclass
class Attempt:
    task_id: str
    n: int                              # 1-based attempt number
    outcome: str                        # "ok" | "retryable" | "fatal"
    reported_tokens: int | None = None  # from provider usage field, if returned
    prompt_chars: int = 0
    completion_chars: int = 0


def billed(a: Attempt) -> int:
    if a.reported_tokens is not None:
        return a.reported_tokens
    return (a.prompt_chars + a.completion_chars) // 4  # rough estimate


def amplification(attempts: list[Attempt]) -> float:
    total = sum(billed(a) for a in attempts)
    useful = sum(billed(a) for a in attempts if a.outcome == "ok")
    if useful == 0:
        return float("inf")     # nothing succeeded; do not divide
    return total / useful

Two habits keep the rows honest.

• Write the row before branching on the result, not after.
• Give every task a caller-generated id, so retries from different processes join up.

The character-based fallback is an estimate. Mark it as one.

Step 2: gate a batch, not a task

One bad task proves nothing. The decision belongs at batch level, with a minimum sample.

# gate.py — illustrative only; thresholds are starting points, not findings
EXIT_RATIO = 2.5      # billed tokens per useful token, per task
EXIT_SHARE = 0.20     # share of tasks allowed to exceed EXIT_RATIO
MIN_TASKS = 30        # refuse to decide on a handful of tasks


def verdict(tasks: dict[str, list[Attempt]]) -> tuple[str, float]:
    if len(tasks) < MIN_TASKS:
        return "insufficient-data", 0.0
    over = [t for t in tasks.values() if amplification(t) > EXIT_RATIO]
    share = len(over) / len(tasks)
    return ("exit-free-lane" if share > EXIT_SHARE else "keep"), share

A reporting entry point makes the gate usable in CI.

# illustrative command shape, not a run
python -m ledger report runs/day-01.jsonl --ratio 2.5 --share 0.20
# verdict=exit-free-lane share=<value> tasks=<count>

Treat the printed numbers as placeholders in this article. The shape is the contribution; the values must come from your own traffic.

Step 3: verify the ledger against the provider

A ledger only counts what it sees. Retries outside the wrapper stay invisible.

Run this reconciliation once a week:

1. Pick one day of traffic and keep the raw rows.
2. Compare attempt counts against provider-side logs or usage metadata.
3. Compare token totals; if they diverge beyond the estimate band, trust reported tokens.
4. Re-run the verdict after every prompt or schema change.
5. Store the file outside the free server. Storage is the non-disposable part.

If step 2 disagrees, stop trusting the ledger and fix instrumentation first.

Anti-use rules: when to leave the free lane

The free lane is the wrong host for these workload traits.

Trait	Why the free lane misleads	Better move
Unvalidated structured output	Each parse failure bills the full prompt again	Validate locally; constrain decoding; add a schema check before retry
Long context re-sent per retry	Payload repeats on every attempt	Chunk, cache, or trim context before retrying
Batch backfill without keys	Duplicates are indistinguishable from retries	Deterministic idempotency key per record
Hard latency SLO	Retry budget consumes the deadline	Separate fast path with a paid lane
Multi-tenant fanout	One noisy tenant drains a shared allowance	Per-tenant budget and rejection at the edge
Side effects inside the retry	Duplicate emails, duplicate charges	Move the side effect after the confirmed success

Exit criteria, stated as triggers rather than vibes:

1. More than 20% of tasks sit above the amplification threshold for two weeks.
2. Any retry can repeat an external side effect.
3. The retry budget exceeds the latency target.
4. A task needs durable state across restarts.
5. One caller dominates the allowance draw.

Any single trigger is enough. Do not average them away.

Where the free server fits, and where it does not

A free server is a disposable compute host. That framing decides placement.

• Good fit: the nightly report job, a throwaway test runner, a demo endpoint.
• Poor fit: the only copy of the ledger, a cron that must fire, a queue with state.
• Poor fit: any inbound path where a missed request breaks a contract.

The usable rule is short. Treat compute as replaceable. Treat state as permanent and place it accordingly.

Limitations and who should skip this

• The ledger counts instrumented attempts only.
• Token counts may be estimates when usage metadata is absent.
• Free-tier terms can change; verify current limits before depending on them.
• Threshold values here are starting points, not measured recommendations.
• No billing-grade attribution is provided by this method.

Skip this approach if you need per-customer invoicing, regulated audit trails, or a hard uptime commitment. A paid lane with a contract is the honest answer there.

Closing

The ledger is cheap to build and uncomfortable to read. That is the point. If the numbers say the free lane fits, run one batch on it and keep the file somewhere durable — MonkeyCode's operator-stated free model access and free server are a reasonable first host for exactly that experiment.