Most agent sweeps on a free tier do not fail because the model was bad. They fail because two workers read the same remaining allowance, both decide there is room, and one of them commits capacity the other already promised away. This is an architecture review of that failure, written for engineers who run multi-step agent work on donated compute and donated tokens.
Example scenario (illustrative, not a report of a real run). A team wants a nightly evaluation sweep: forty agent steps, each calling a model a few times, all of it landing on a free server that may be reclaimed without notice. The model provider grants free access with a token allowance; the project also offers a free server option. Treat both grants as loans with expiry, not as balances you own.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. The environment above matches what the open-source MonkeyCode project advertises as free model access plus a free server option, and its stated free token allowance is currently described as ten million tokens. Both are time-sensitive commercial terms, so verify the current numbers and limits on the project's official page before you size a run against them.
The violating event order
Consider a lease with capacity 100 units, where each unit is one thousand tokens for arithmetic convenience. Worker A wants a step it estimates at 60 units; Worker B wants a step it estimates at 50. Both call a helper named remaining(), both see 100, both admit, and both start calling the model. No single read was wrong, and the sum still crosses the capacity before either step finishes.
The invariant a correct design must preserve is this: at every prefix of the event order, committed plus reserved capacity never exceeds the lease capacity, and any step holding a stale lease epoch must fail closed rather than commit. Checking a balance is a read; preserving that invariant requires a write that others can observe.
Model the two grants separately
A free server and a free token allowance fail in different ways, so give them separate leases rather than one merged budget object. The compute lease governs where a step runs and can be revoked by the platform; the token lease governs how much model work an admitted step may consume. A step is admitted only when it holds both, and it is committed only against the token lease it was admitted under.
admission
step request ───────────────► [compute lease: epoch c]
│ │
│ ├─ revoked ──► step aborts, artifact dropped
▼ │
[token lease: epoch t] ── reserve ──► RESERVED ── commit(actual) ──► COMMITTED
│ │
│ └─ release() ──► RELEASED
└─ epoch advanced (revocation / reset) ──► commits from epoch < t are rejected
The state machine has five states per step: PROPOSED, RESERVED, COMMITTED, RELEASED, and ABORTED. RESERVED is the only state that holds capacity without having produced a result, which makes it the state worth alarming on. A run that ends with any step still in RESERVED has lost capacity silently, and that is a gate failure rather than a cosmetic warning.
Step 1: define the accounting unit before writing code
Pick one unit, declare how it is measured, and record the measurement source next to every committed number. If your model provider reports usage per call, you can commit actuals; if it does not, your commits are estimates and your capacity check is approximate. State which one you have, because an unlabelled estimate passed off as an actual is how budget reports become fiction.
Set two numbers per step: an estimate used for reservation, and a hard cap that the step may not exceed. The cap is what protects the shared lease from a single runaway loop, and the estimate is what protects other steps from over-reservation.
Step 2: reserve with a compare-and-set on a fencing epoch
Reservation must be a single atomic operation on the lease owner, not a read followed by a write. Each lease carries a monotonically increasing epoch; advancing the epoch invalidates every outstanding reservation, which is exactly what you want when the platform reclaims the server or the allowance resets.
The following module is illustrative and was not executed for this article; it exists to make the transitions concrete and to be adapted, not copied as a library.
# Illustrative only — not executed, not a library release.
from dataclasses import dataclass
class LeaseError(Exception):
pass
@dataclass
class BudgetLease:
lease_id: str
epoch: int # fencing token, monotonic, bumped on revocation
capacity: int # total units this lease may admit
reserved: int = 0
committed: int = 0
closed: bool = False
def _remaining(self) -> int:
return self.capacity - self.committed - self.reserved
def reserve(self, step_id: str, estimate: int, cap: int) -> "Reservation":
# Must run as one critical section on the lease owner.
if self.closed:
raise LeaseError("lease closed")
if estimate <= 0 or cap < estimate:
raise LeaseError(f"bad budget for {step_id}")
if estimate > self._remaining():
raise LeaseError(f"would overcommit by {estimate - self._remaining()}")
self.reserved += estimate
return Reservation(self, step_id, estimate, cap)
@dataclass
class Reservation:
lease: BudgetLease
step_id: str
estimate: int
cap: int
epoch: int = None
state: str = "RESERVED"
def __post_init__(self) -> None:
self.epoch = self.lease.epoch
def _fenced(self) -> None:
if self.epoch != self.lease.epoch:
raise LeaseError(f"stale epoch on {self.step_id}")
def commit(self, actual: int) -> None:
self._fenced()
if self.state != "RESERVED":
return # idempotent replay of a duplicate commit
if actual > self.cap:
raise LeaseError(f"{self.step_id} exceeded its hard cap")
self.lease.reserved -= self.estimate
self.lease.committed += max(actual, 0)
self.state = "COMMITTED"
def release(self) -> None:
# Not fenced: releasing only ever frees capacity, so a stale holder
# cannot use it to claim budget. Still idempotent.
if self.state != "RESERVED":
return
self.lease.reserved -= self.estimate
self.state = "RELEASED"
The asymmetry between commit and release matters. Commit is a claim on capacity and must be fenced, while release only subtracts and is therefore safe from a stale holder. If your storage cannot express that asymmetry, put release behind a separate path that only the lease owner may call.
Step 3: make commit and release idempotent on step identity
Every commit and release is keyed by step_id, so a duplicate delivery changes state exactly once. This is the same requirement as any outbox, and it is worth repeating because a reclaimed free server often means the client retries a request whose first attempt already succeeded somewhere.
Store the reservation outcome before you publish the step's artifact. If the process dies between writing the artifact and committing the budget, you get an artifact with no accounting entry, which silently understates spend and hides a real failure.
Step 4: treat revocation as a normal transition
When the free server is reclaimed, aborting a step is correct behaviour, not an error page. Bump the lease epoch, let commits from old epochs fail closed, and mark the step ABORTED instead of retrying it into the same dead lease. Retries belong to the next epoch, with a fresh reservation.
Properties worth testing
Write these as assertions over a scripted event log rather than as prose in a design doc. Each one is checkable against the illustrative module above, and each one can be broken on purpose to confirm the test actually fires.
P1 no-overcommit : for every prefix of the event log,
committed + reserved <= capacity
P2 no-orphan : every admitted step ends COMMITTED, RELEASED, or ABORTED
P3 fenced-commit : a commit whose epoch < lease.epoch raises LeaseError
P4 replay-safety : applying commit() twice changes state and totals once
A useful fault schedule is short and adversarial: reserve 60, reserve 50, bump the epoch, then commit both in reverse order. A correct implementation accepts the first reservation, rejects the second with a remaining-capacity message, rejects both commits after the epoch bump, and leaves reserved at zero once the aborts release their holds.
Failure domains to review
| Failure | Blast radius | Detection | Design response |
|---|---|---|---|
| Two workers read the same remaining budget | Overspend; sweep halts mid-run | Sum of reservations exceeds capacity | Atomic reserve on the lease owner |
| Compute lease revoked mid-step | Partial artifact, wasted tokens | Epoch advanced before commit | Fail closed, mark ABORTED, do not reuse lease |
| Duplicate commit delivery | Double-counted spend | Step key appears twice in the ledger | Idempotent commit keyed by step_id
|
| Estimate drift on long steps | Reservation too small in practice | Actual exceeds cap repeatedly | Track estimate error and raise caps by a measured factor |
| Allowance reset mid-run | Old reservations become meaningless | Epoch bump from the provider side | Same fence as revocation |
Tradeoffs you are actually choosing
| Option | Latency cost | Overspend risk | Best fit |
|---|---|---|---|
| Reserve before every call | One atomic round trip per step | Bounded by reservation granularity | Shared allowance, concurrent workers |
| Optimistic spend, reconcile later | None | Unbounded until reconciliation | Single worker, generous quota |
| Shard the allowance per worker | None across shards | Idle capacity in quiet shards | Predictable, even workloads |
| Per-step hard cap only | None | Shared lease still overrun by many small steps | Trivial scripts with one worker |
Reservation granularity is the knob that decides whether this protocol helps or hurts. Reserving per model call protects the allowance tightly and adds a round trip to every call, while reserving per step is cheaper and lets one step hold capacity it never uses. Pick per step for long agent loops, and add a cap so a single step cannot monopolise the lease.
Who should not use this
If one worker runs one script against a comfortable allowance, the fencing layer is overhead with no failure to prevent. If your provider reports exact usage after the fact and your runs are short, reconciliation is simpler than reservation. And if you cannot tolerate rejected admissions at all, this protocol will annoy you, because failing closed is the behaviour that makes it work.
How to validate the conclusion yourself
- Implement the module above against a single-process lease owner, then against a shared store with a compare-and-set primitive.
- Drive the adversarial schedule and assert P1 through P4; break the fence on purpose and confirm P3 fails.
- Randomise interleavings with a small state-space explored exhaustively, which is affordable at two or three workers.
- Record the rejection rate and the estimate-versus-actual distribution across a real sweep, then decide whether per-step or per-call reservation fits your workload.
Acceptance rule for a sweep: it passes only if no step ends in RESERVED, committed never exceeds capacity at any prefix, and every artifact has exactly one accounting record. Anything else is a gate failure, regardless of how good the model outputs looked.
So here is the question to carry back to your own system: which interleaving of reservation, revocation, and duplicate commit breaks your current allowance check, and should the system reject the step, replay it under a new epoch, or compensate after the fact?
Top comments (0)