DEV Community

Zira
Zira

Posted on

Your AI Agent Has a Budget. Can You Explain Every Token and Tool Call?

#devops #agents #automation #testing

An agent can be “within budget” while still being impossible to operate: the dashboard shows aggregate tokens, but nobody can answer which retry, tool call, browser session, or tenant consumed them.

The fix is to treat cost as a durable control-plane record, not a number copied from a provider dashboard.

Define the unit you will charge

Pick one unit and keep it stable across model and runtime changes. A useful default is a run:

  • run_id, parent run, tenant, and workflow version
  • model/provider and request class
  • input, output, cached-input, and reasoning-token counts when available
  • tool name, attempt number, and estimated tool cost
  • browser/runtime seconds and external API charges
  • outcome: SUCCEEDED, FAILED, CANCELLED, or UNKNOWN

Do not derive cost from the final transcript. A timed-out request may have been accepted upstream, and a tool can create a charge before the agent receives its response.

Record intent before dispatch

Write an immutable dispatch-intent row before making a model or tool request. Store a stable request key and a normalized payload hash so a retry cannot silently become a second billable operation.

CREATE TABLE usage_events (
  event_id TEXT PRIMARY KEY,
  run_id TEXT NOT NULL,
  request_key TEXT NOT NULL,
  kind TEXT NOT NULL, -- model, tool, browser, api
  attempt INTEGER NOT NULL,
  state TEXT NOT NULL, -- INTENDED, SUCCEEDED, FAILED, UNKNOWN
  input_units INTEGER,
  output_units INTEGER,
  estimated_micros INTEGER,
  observed_micros INTEGER,
  payload_sha256 TEXT NOT NULL,
  created_at TEXT NOT NULL
);

CREATE UNIQUE INDEX one_intent_per_request
  ON usage_events(request_key, kind);
Enter fullscreen mode Exit fullscreen mode

The unique key is not a substitute for provider-side idempotency. It prevents your own dispatcher from creating duplicate intent records; reconciliation still has to determine whether an UNKNOWN request was accepted upstream.

Enforce a budget before the call

Keep three values separate:

  1. Reserved: worst-case cost admitted for the next operation.
  2. Observed: confirmed usage returned by the provider or tool.
  3. Unresolved: cost attached to an ambiguous request that must be reconciled.

Admission should fail closed when observed + reserved + unresolved exceeds the run or tenant limit. Release a reservation only after the operation reaches a terminal state. Otherwise a burst of concurrent workers can all pass a stale balance check.

A simple reservation transaction is:

BEGIN
  lock budget(run_id)
  reject if observed + reserved + unresolved + next_reservation > limit
  increment reserved by next_reservation
  insert INTENDED usage event with request_key
COMMIT
send request
Enter fullscreen mode Exit fullscreen mode

Never hold the database transaction open across the network call. The intent row is the handoff boundary.

Test the failure states

A cost system is not tested by checking a successful token count. Inject failures at these boundaries:

Failure point Required result
before intent commit no dispatch occurs
after intent, before send intent remains retryable
after upstream accepts, before response mark UNKNOWN; do not blindly resend
after response, before usage commit replay the response or reconcile the request key
worker crash during reservation reservation is leased and recoverable
concurrent workers only one reservation wins for a request key
provider price change historical events retain the price version used

Then run the same workflow after a process restart. Compare the event ledger with provider usage and runtime logs. Differences should produce an explicit reconciliation task, not a manually edited total.

A practical rollout checklist

  • Generate run_id and request keys at the control-plane boundary.
  • Persist intent before every billable model, tool, browser, or API operation.
  • Track reserved, observed, and unresolved cost independently.
  • Attribute usage to tenant, workflow, model, tool, and attempt.
  • Store price and estimator versions with each event.
  • Put hard limits at admission, not only in a dashboard.
  • Make UNKNOWN visible and reconcile it before retrying mutations.
  • Alert on reservation age, unresolved cost, and estimate-versus-observed drift.

This makes cost a debuggable property of the run. You can explain a bill, stop runaway work before the next call, and change providers without losing the accounting trail.

Top comments (0)