An AI agent can fail without crashing, throwing an exception, or producing an obviously wrong answer. It can simply keep doing valid work for far too long.
That is a different failure mode from the ones most software teams are used to. A retry loop can keep calling a paid API. A coding agent can keep launching test environments. A background worker can keep generating images, embeddings, logs, or storage objects. Every individual action may be permitted, yet the overall run becomes economically unsafe.
This is why hard budget caps are starting to look less like billing features and more like runtime safety controls.
Simon Willison made the case directly on October 3: usage-priced services need hard limits that stop work instead of merely sending an alert after a threshold is crossed. Google Cloud has moved in the same direction with Spend Caps, which can pause eligible services after a configured budget is exceeded. The important shift is architectural: cost is becoming something software can enforce, not merely something finance reviews later.
For agent systems, that distinction matters even more because agents can create new work on their own.
A Budget Alert Is Not a Control Boundary
Traditional cloud budgets are usually observational. They answer questions such as:
- How much has this project spent?
- Is spending above the expected trend?
Those are useful questions, but none of them stop execution.
An agent needs a control boundary. If a run is allowed to consume at most 500 units of a metered resource, then unit 501 should fail closed. The exact unit may be API credits, GPU minutes, database writes, browser actions, tokens, or a normalized internal cost score.
The basic distinction is simple:
soft budget:
observe -> alert -> keep running
hard budget:
observe -> compare -> deny next action
The second model belongs on the execution path itself.
Google Cloud's current Spend Caps preview makes this idea concrete. Its documentation says eligible service usage can be paused when the configured cap is enforced, while resources remain intact. Google also warns that enforcement is not instantaneous because estimated costs are involved, which is an important reminder: a provider-side cap is useful, but an application should still maintain its own tighter limits.
Agents Turn Small Mistakes Into Repeated Actions
A normal application usually has a fairly stable request shape. Users click buttons, jobs enter queues, and traffic rises or falls within recognizable patterns.
Agents can create a feedback loop.
A planner decides that a task failed. The retry policy creates another attempt. The new attempt spawns a browser session. The browser session calls an image service. The image service output fails validation. The planner decides to try again with a revised prompt. Nothing in that sequence is necessarily broken.
The problem is multiplication.
One bad decision can become twenty valid paid actions. A second recovery rule can turn those twenty into a hundred. If the only cost control is an email, the system can continue while nobody is watching.
This is why an agent budget should be treated like a memory limit or a request deadline. It is not an accounting preference. It is a condition for continued execution.
Put the Budget Check Before the Tool Call
The safest place for a budget gate is immediately before a side-effecting or metered action.
A simple implementation can keep the policy outside individual tools:
from dataclasses import dataclass
@dataclass
class Budget:
limit: int
used: int = 0
def reserve(self, units: int) -> None:
if units <= 0:
raise ValueError("units must be positive")
if self.used + units > self.limit:
raise RuntimeError("hard budget exceeded")
self.used += units
budget = Budget(limit=500)
def call_metered_tool(tool, estimated_units, **kwargs):
budget.reserve(estimated_units)
return tool(**kwargs)
This example is intentionally small. The important property is ordering: reserve first, execute second.
If the budget check happens after the tool call, it is only telemetry. If a retry path can bypass the wrapper, it is not a hard cap. If a child agent receives a fresh counter instead of sharing the parent budget, spawning becomes a way to escape the limit.
The budget object therefore belongs at the same architectural level as authentication, authorization, and cancellation.
Use More Than One Budget Dimension
Money is not the only resource that needs a ceiling.
A useful agent runtime can enforce several independent dimensions:
run_budget:
max_tool_calls: 80
max_browser_writes: 12
max_external_api_units: 500
max_wall_clock_seconds: 1200
max_child_agents: 3
max_retries_per_step: 2
These limits solve different problems.
A wall-clock limit catches work that stalls without spending much. A tool-call limit catches rapid loops. A browser-write limit protects external accounts from repeated side effects. A retry limit prevents one failing step from consuming the entire run. A child-agent limit prevents recursive delegation from turning one task into an uncontrolled tree.
The dimensions should be independent because a run can be safe in one dimension and dangerous in another.
For example, a cheap API can still generate thousands of unwanted external objects. A browser automation task can cost almost nothing in infrastructure while repeatedly posting, liking, or modifying data. Financial cost alone would miss the real failure.
Reservations Are Better Than Post-Hoc Accounting
Many metered operations do not reveal their final cost until they finish. That creates a race: several concurrent workers can all see remaining budget and begin expensive work at the same time.
Reservations fix that.
Before starting an operation, reserve the maximum or a conservative estimate. After it finishes, reconcile the reservation with actual usage.
class Ledger:
def __init__(self, limit):
self.limit = limit
self.committed = 0
def reserve(self, estimate):
if self.committed + estimate > self.limit:
return None
self.committed += estimate
return {"reserved": estimate}
def settle(self, ticket, actual):
reserved = ticket["reserved"]
self.committed += actual - reserved
A production implementation needs atomic storage, but the model is what matters. Concurrency should not create free budget.
This is similar to inventory systems. Two buyers should not both be able to purchase the last item because they read the same stock count before either write completes. Two agents should not both spend the same remaining allowance because they checked it at the same time.
Provider Caps and Application Caps Solve Different Problems
Provider-side limits are valuable because they are outside the application. If the application has a bug, the provider can still stop further billable usage.
Application-side limits are valuable because they understand intent.
A cloud provider sees requests. Your runtime sees tasks, users, sessions, tools, retries, and side effects.
That suggests a layered design:
- A provider cap limits worst-case account exposure.
- A project cap limits one application.
- A user cap limits one tenant or operator.
- A run cap limits one agent execution.
- A step cap limits retries and side effects inside one workflow.
Google Cloud's Spend Caps are an example of the first two layers for eligible services. The feature can pause usage after the configured threshold is reached, and Google documents alert points below the cap as well. But an agent runtime can act earlier because it knows whether a request is part of a retry storm or a legitimate new task.
The two controls should reinforce each other rather than compete.
Budget Exhaustion Must Be a Normal Terminal State
Many agent systems treat every interrupted task as an error that deserves another attempt. That is dangerous when the interruption is the budget system doing its job.
Budget exhaustion should be a first-class terminal state:
SUCCESS
FAILURE
BLOCKED
CANCELLED
BUDGET_EXHAUSTED
A scheduler receiving BUDGET_EXHAUSTED should not automatically retry the same plan with a fresh allowance. That would turn the budget into a delay instead of a boundary.
The terminal record should include enough evidence to explain what happened:
- configured limit;
- consumed amount;
- last successful step;
- action that was denied;
- retry counts;
- child runs;
- external side effects already completed.
That makes recovery explicit. A human or higher-level policy can choose to grant more budget, reduce scope, or stop permanently.
Retries Need Their Own Economics
Retries are one of the easiest ways to accidentally defeat a cap.
Suppose an image generation step fails quality validation. The system may reasonably try a second prompt. But if each retry receives the original full run allowance, the cost model is fictional.
Retries should spend from the same parent budget and usually become stricter as attempts accumulate.
One useful policy is:
retry_limits = {
"network_timeout": 2,
"rate_limit": 1,
"validation_failure": 1,
"authentication_failure": 0,
"permission_denied": 0,
}
The important part is not the exact numbers. It is that retry permission is based on failure class, and every retry still consumes the original run budget.
This also improves behavior quality. An agent that knows it has one remaining attempt is more likely to gather evidence before acting than one that can keep trying indefinitely.
A Hard Cap Should Fail Closed Without Destroying State
Stopping spend should not mean destroying work.
Google Cloud explicitly describes its Spend Caps as non-destructive: eligible usage can be paused while data and resources remain. Agent runtimes should follow the same principle.
When a hard cap is reached:
- stop new metered actions;
- preserve logs and receipts;
- keep already-created drafts or artifacts;
- do not roll back verified external side effects blindly;
- record the exact continuation point;
- require an explicit new allowance before resuming.
This produces a resumable system instead of a disposable one.
It also separates two concerns that are often mixed together: stopping new risk and cleaning up old state. The first should happen immediately. The second should be deliberate and evidence-based.
Budget Policy Belongs in the Runtime, Not in Prompts
A prompt can tell an agent to be frugal. That is not enforcement.
Prompts influence decisions. Runtime gates constrain actions.
The safest architecture assumes that the model may misunderstand a limit, forget it after context compression, delegate to another process, or choose a plan whose cost estimate is wrong. A hard budget should still hold.
That means the policy must live outside the model loop:
model proposes action
|
v
policy checks permission
|
v
budget reserves capacity
|
v
tool executes
|
v
ledger settles actual usage
The model can participate by estimating cost and selecting cheaper plans, but it should not be the final authority over whether the action is allowed.
Cost Is Becoming Part of Correctness
The broader lesson is that correctness for autonomous software has changed.
A task is not correct merely because it eventually reaches the requested output. It also has to respect limits while getting there: time, external writes, retries, permissions, and spend.
That is why the current push toward real spend caps matters. Simon Willison's argument for default hard limits is not only a billing recommendation. It points toward a runtime design principle for agentic systems: every autonomous loop needs a maximum amount of damage it is allowed to cause before the infrastructure stops it.
Google Cloud's July 2026 Spend Caps preview shows that cloud platforms are beginning to expose this control at the billing layer. Agent frameworks should not wait for every provider to do the same.
A reliable agent should know what it is allowed to spend, reserve that capacity before acting, share the same ledger across retries and child workers, and stop cleanly when the allowance is gone.
That is not pessimism about autonomous systems. It is the mechanism that makes unattended execution practical.
Sources:
- Simon Willison, "We're going to need default hard budget caps on pretty much everything"
- Google Cloud, "New early anomalies and spend caps on Google Cloud Budgets"
- Google Cloud documentation, "Manage spend cap budgets"
Originally published on Dispatch.
Top comments (1)
Your point about individually permitted actions becoming economically unsafe gets sharper when agents spawn more agents. Do those children consume the originating run’s remaining budget, or does each get a fresh allowance?