DEV Community

Khoushik Reddy Ananth
Khoushik Reddy Ananth

Posted on

Cap what an AI agent can spend before it runs, not after the bill arrives

Most teams find out what an agent cost by reading the invoice. By then the loop that ran all weekend has already spent the money.

The fix is boring and it works: decide what a task is allowed to cost before it starts, and make the agent stop itself when it gets there. Here is the pattern I use, in plain Python, no framework required.

The four steps

  1. Estimate what the next model call will cost.
  2. Reserve that amount against the task's budget. If it doesn't fit, don't make the call.
  3. Run the call.
  4. Settle: replace the estimate with the real cost from the response's usage numbers.

Every call goes through those four steps. Nothing calls the model directly.

A budget object

from dataclasses import dataclass, field


class BudgetExceeded(Exception):
    pass


@dataclass
class TaskBudget:
    task_id: str
    owner: str            # a named person, not a team alias
    limit_usd: float      # set before the task starts
    spent_usd: float = 0.0
    reserved_usd: float = 0.0
    log: list = field(default_factory=list)

    def remaining(self) -> float:
        return self.limit_usd - self.spent_usd - self.reserved_usd

    def reserve(self, amount: float) -> None:
        if amount > self.remaining():
            raise BudgetExceeded(
                f"{self.task_id}: needs ${amount:.4f}, "
                f"${self.remaining():.4f} left of ${self.limit_usd:.2f}. "
                f"Escalating to {self.owner}."
            )
        self.reserved_usd += amount

    def settle(self, reserved: float, actual: float, note: str) -> None:
        self.reserved_usd -= reserved
        self.spent_usd += actual
        self.log.append({"note": note, "estimated": reserved, "actual": actual})
Enter fullscreen mode Exit fullscreen mode

Two fields matter more than the maths: owner and log. When the budget runs out, a person is told, and there is a record of where the money went.

Estimating a call

You don't need a perfect estimate. You need one that is never too low. Count the prompt, assume the reply uses the full max_tokens, and price it with the rates for the model you actually call.

# Fill these in from your provider's current price page.
# Prices change; keep them in config, not in code.
PRICES_PER_MTOK = {
    "small-model": {"in": 0.0, "out": 0.0},
    "large-model": {"in": 0.0, "out": 0.0},
}


def estimate_cost(model: str, prompt_tokens: int, max_tokens: int) -> float:
    p = PRICES_PER_MTOK[model]
    return (prompt_tokens * p["in"] + max_tokens * p["out"]) / 1_000_000


def actual_cost(model: str, usage_in: int, usage_out: int) -> float:
    p = PRICES_PER_MTOK[model]
    return (usage_in * p["in"] + usage_out * p["out"]) / 1_000_000
Enter fullscreen mode Exit fullscreen mode

Assuming the full max_tokens makes the estimate pessimistic on purpose. Settling afterwards gives the unused part back.

The guarded call

def guarded_call(budget: TaskBudget, model: str, prompt: str,
                 max_tokens: int, call_model, count_tokens):
    est = estimate_cost(model, count_tokens(prompt), max_tokens)
    budget.reserve(est)                      # raises before any spend
    try:
        reply, usage_in, usage_out = call_model(model, prompt, max_tokens)
    except Exception:
        budget.settle(est, 0.0, f"{model}: call failed")
        raise
    budget.settle(est, actual_cost(model, usage_in, usage_out), f"{model}: ok")
    return reply
Enter fullscreen mode Exit fullscreen mode

call_model and count_tokens are whatever your stack already uses. The guard doesn't care which provider or framework sits behind them.

Using it in an agent loop

budget = TaskBudget(task_id="refund-4821", owner="priya@ops", limit_usd=0.50)
done, next_prompt = False, first_prompt

try:
    while not done:
        step = guarded_call(budget, "small-model", next_prompt, 400,
                            call_model, count_tokens)
        done, next_prompt = handle(step)
except BudgetExceeded as e:
    hand_to_person(budget.owner, reason=str(e), history=budget.log)
Enter fullscreen mode Exit fullscreen mode

The loop can't spend past the cap, because the reservation fails before the call is made, not after.

Three things I learned the hard way

Cap per task, not per month. A monthly cap tells you on the 28th that something went wrong on the 3rd. A per-task cap stops the one runaway case and lets everything else keep running.

Route before you spend. Most of the savings come from not calling the large model at all. Work with a right answer (a lookup, a format check) goes to a script. Routine work goes to a small model. The large model only gets the steps where judgment matters. The budget makes that visible, because every log line says which model spent what.

Hitting the cap is a handoff, not a crash. When the budget runs out, the task goes to its named owner with the history attached. A person decides whether it's worth more money. The agent never raises its own limit.

What this doesn't cover

This is the core of it. In production you also want the limits held outside the agent's own code (so an agent can't edit its own budget), one shared record across all agents, and cost per task reported next to the business result it produced, so you can judge spend and value in the same meeting.

How LeafCraft can help

LeafCraft is the platform where AI agents and people work as one team. Every agent has a named owner, a written job and a spending limit set before each task runs, and every step lands on one record you can audit. When an agent hits its limit, the work hands off to the right person instead of quietly spending more. It runs in your own environment and works with the agents you already use. If AI cost control is on your list for next quarter, take a look at https://leafcraft.ai.

Khoushik Reddy Ananth, founder of LeafCraft, the Hybrid Intelligent Enterprise platform.

Top comments (0)