Enforce the budget inside a synchronous admission gate immediately before every billable external call, using an atomic reservation against the agent's prepaid balance. The deciding constraint is whether the system must honor a hard spend ceiling or preserve traffic when its estimate is wrong: a hard ceiling necessarily refuses work once no safe reservation can be made.
This is the architecture decision record I would want for an unattended developer-tool agent, where a planning loop can call models and tools without a person watching the meter. Node.js and Python are implementation details here. The correctness boundary belongs in a shared account service, not in either loop's process memory, because concurrent workers must contend on one balance and leave one audit trail.
Where should an autonomous AI agent loop enforce its budget limit?
Put enforcement at the last common point before dispatch, after the request has a tenant, run, operation, and conservative maximum cost, but before credentials can authorize the external call. The gate should perform one atomic state transition: reserve the maximum charge or refuse the operation. A check performed earlier in the planner is useful for user feedback, yet it cannot be authoritative; another worker can consume the remaining balance between that check and dispatch.
The credential boundary matters. If a worker can bypass admission and use a provider secret directly, the ceiling is merely advisory. Give the dispatcher access to the secret, keep it out of planner state and logs, and require a valid reservation identifier on the internal dispatch path. OWASP's secrets-management guidance is relevant to that separation: credentials need controlled access, rotation, and auditable handling rather than casual distribution through application configuration.
Three invariants define the design. First, available funds equal funded balance minus open reservations minus settled charges; the exact schema may differ, but the arithmetic cannot. Second, the same operation key can create at most one reservation and one settlement effect, even after retries. Third, every refusal, reservation, release, and settlement records the account, run, operation key, amount, currency or usage unit, policy version, and timestamp.
Keep that third one.
An exactly-once mindset does not mean pretending the network delivers exactly once. It means making repeated delivery converge on one financial effect through idempotency, while retaining enough evidence to reconcile the internal ledger with the eventual provider record. Compliance obligations vary by jurisdiction and data classification, so retention periods, operator access, and whether prompts may appear in audit records require review by the relevant security and legal owners; an architecture diagram cannot settle those limits.
Invariants and failure boundaries
The reservation is a liability hold, not a final charge. Suppose a prepaid account has 1,000 abstract usage units, run A reserves 300, and run B concurrently requests 800. The store must serialize those attempts so only a valid combination succeeds; reading 1,000 in both workers and subtracting later can authorize 1,100. This example is intentionally unit-based because converting tokens, tool calls, and time into money before provider reconciliation can imply precision the system does not possess.
Estimation therefore needs an explicit policy. Reserve a defensible upper bound for the next operation, settle the actual attributable amount when it becomes known, and release the remainder. If actual usage can exceed the reservation, choose in advance whether the dispatcher constrains the request to the reserved maximum or permits an overdraft. A hard prepaid ceiling requires the former. There is no clever accounting trick that offers both unrestricted completion and a guaranteed cap.
Retries cross several failure boundaries. If the client loses a response after a reservation commits, retrying with the same operation key must return that reservation rather than hold funds twice. If dispatch never begins, expire or explicitly release the hold under a documented policy. If dispatch begins but its final usage record is delayed, keep the hold open and reconcile later; releasing it immediately would let a second call spend funds that may already be owed. The policy also needs a maximum reservation age and an escalation path, but I'm not sure there is a universally correct duration: it depends on the longest legitimate operation and how quickly authoritative usage arrives.
Refusal should be a typed domain outcome, not an ambiguous transport failure. The loop can then stop, request funding, reduce the next operation's maximum, or switch to a locally permitted action without retrying the same unaffordable call forever. Record that decision too. An unattended loop that retries budget refusals is still a runaway loop, only quieter.
Comparing enforcement locations
The options differ mainly in race behavior, bypass resistance, and operational cost. This table treats enforcement as a correctness control; a planner-side estimate can still be layered on top for a faster user experience.
| Location | Hard ceiling under concurrency | Main benefit | Limitation and valid use case |
|---|---|---|---|
| In-memory loop counter | No | Minimal latency and simple local tests | Suitable only for a single, non-restarted worker with a disposable soft budget |
| Planner or job queue | Not by itself | Can reject obviously unaffordable runs early | Keep it as a preview when later steps have variable cost; another path may still spend |
| Shared account admission gate | Yes, with atomic storage | One policy and audit stream across Node.js, Python, and other workers | Adds a synchronous dependency and can refuse traffic when funds or safe estimates are insufficient |
| Post-usage billing job | No | Reconciliation is naturally based on observed usage | Use it for invoiced credit arrangements where temporary exposure is contractually acceptable |
The shared gate is the decision for a prepaid, unattended agent. The catch is real: putting a dependency on the critical path means its availability and latency become dispatch concerns, and conservative reservations may reject a call that would have completed cheaply. Teams choosing a soft alert threshold, contractual credit line, or best-effort batch workload should stick with asynchronous metering plus alerts instead of presenting a hard cap they do not actually enforce.
Critical path in Go
The following Go sketch keeps vendor calls behind a generic dispatcher. Reserve must be transactional in its implementation: insert the idempotency record and decrement available capacity as one atomic operation. The example uses integer units so there is no floating-point balance arithmetic.
package admission
import (
"context"
"errors"
"time"
)
var ErrBudgetExceeded = errors.New("budget limit exceeded")
type Operation struct {
AccountID string
RunID string
Key string
MaxUnits int64
PolicyID string
}
type Reservation struct {
ID string
OperationKey string
HeldUnits int64
ExpiresAt time.Time
}
type Usage struct {
Units int64
}
type Ledger interface {
Reserve(ctx context.Context, op Operation) (Reservation, error)
Settle(ctx context.Context, reservationID string, usedUnits int64) error
Release(ctx context.Context, reservationID string) error
}
type Dispatcher interface {
Call(ctx context.Context, reservationID string, op Operation) (Usage, error)
}
type Gate struct {
ledger Ledger
dispatcher Dispatcher
}
func (g Gate) Execute(ctx context.Context, op Operation) (Usage, error) {
reservation, err := g.ledger.Reserve(ctx, op)
if err != nil {
return Usage{}, err
}
usage, err := g.dispatcher.Call(ctx, reservation.ID, op)
if err != nil {
// The reconciler decides release versus settlement from dispatch evidence.
return Usage{}, err
}
if err := g.ledger.Settle(ctx, reservation.ID, usage.Units); err != nil {
return Usage{}, err
}
return usage, nil
}
Production code needs more state than the short path shows. Model reservations as a state machine such as held, dispatched, settled, released, with legal transitions enforced in storage. Do not release inside the generic dispatch-error branch: a canceled client context does not prove that the external operation was never accepted. A reconciler can inspect durable dispatch evidence and apply an idempotent transition, while an outbox or equivalent commit-coupled event mechanism carries ledger changes to analytics without making analytics part of admission. Use the same operation key from the first reservation attempt through every retry. In Node.js or Python, preserve that rule even if cancellation, exception, and task APIs differ. A generated key on each retry defeats idempotency; a key reused for two genuinely different calls incorrectly merges their financial effects. The useful shape is usually account ID plus run ID plus stable step ID plus attempt purpose, canonicalized and stored under a uniqueness constraint. Test the races, not just the happy path: start two reservations against a balance that can satisfy only one and assert that exactly one succeeds, replay reserve and settle calls with the same key, then cancel after durable dispatch but before the response and verify that reconciliation neither releases an owed charge nor settles it twice. Finally, rotate the dispatch credential and confirm that planners cannot read it; cost control and secret isolation fail together when every worker holds unrestricted credentials.
Race it deliberately.
Rejected option and operating decision
The rejected design is a per-loop counter that subtracts estimated cost after each response. It is appealing because it requires no shared service and keeps a Node.js or Python prototype compact. It is also too late for a hard cap: the expensive call has already been authorized, concurrent loops do not see one another, and a process restart can erase local state.
That design still has a valid use case. Keep it for a soft development guard when one engineer runs one ephemeral loop, the credentials themselves are tightly constrained, and exceeding the local estimate has no material financial or compliance consequence. Promote enforcement to shared admission before adding parallel workers, unattended schedules, or a prepaid customer promise.
For operations, alert on reservation refusals, aging holds, reconciliation lag, and divergence between settled internal usage and authoritative external records. Avoid treating a rising refusal count as an infrastructure incident by default; it may show the ceiling doing its job. The run record should expose which policy refused the call without exposing the secret or sensitive prompt content.
The final rule is deliberately strict: no reservation, no external call. Accept the refused traffic that follows from a hard ceiling, or document that the system provides alerts and credit exposure instead. Calling a post-usage notification a budget limit makes the dashboard comforting, but it does not keep an autonomous agent's prepaid balance from running out unattended.
Top comments (0)