A metered invoice should come from settled usage events, while an agent budget needs an estimate before admitting a call. For an edtech service billing each school for its tutoring agent, the least complex workable design is a reservation against a school-level budget followed by a reconciled usage ledger. Keep the invoice ledger longer than the request payloads. TL;DR: a pre-call cost estimate controls exposure; a post-hoc usage report determines the charge. Neither can substitute for the other.
What actually fills the bill?
Start with the storage and reconciliation bill, not a per-token price. In an illustrative workload of 10,000 agent calls per day, retaining 20 KB of request and response material per call for 30 days means roughly 6 GB before replication and indexing. A 200-byte accounting event per call over the same period is about 60 MB. These are planning assumptions, not observed traffic or provider prices. The dominant term in this example is raw material retained for investigation; trimming it changes the storage footprint far more than shaving fields from the accounting event. The accounting event stays because deleting evidence behind a school invoice would make a dispute harder to resolve.
That calculation does not say to discard payloads immediately. A delivery-style failure matters here: a tutoring workflow can retry after a timeout even when the first model call completed. The same ambiguity that makes an OTP retry dangerous makes usage attribution dangerous. Two completed calls may be real usage; two deliveries of one usage report are not two charges. Capture a stable tenant identifier, logical attempt identifier, provider request identifier when available, and the provider's reported input and output units. Keep the identifiers and the provenance of each correction in a durable ledger, with access controls and retention policy appropriate to the billing record.
Should pre-call cost estimates or post-hoc usage reports determine school invoices?
No. Before dispatch, estimate an upper bound using the selected model's configured rates, input size, allowed output ceiling, and any bounded tool calls. Reserve that amount atomically against the school's remaining allowance. This is admission control: if several tutoring sessions start together, a read-then-write balance check can admit all of them against the same remaining balance. A reservation must have an expiration and an explicit state, so a process crash does not strand capacity forever. For unbounded tools or unknown rates, refuse the bounded-budget path or apply a separately approved limit; a guessed zero is not a limit.
After completion, use the reported usage units and the rate version applicable to that call to settle the reservation. Release the unused portion. If the final report exceeds the reservation, record the overage instead of silently dropping the excess or altering the historical estimate. An absent usage report is a pending reconciliation case, not an invoice line with zero units. Rate versions matter: applying a new rate card to last week's units changes an old invoice without changing the underlying work.
Here is the narrow accounting calculation in Python. The caller supplies rates and units from its own approved configuration and usage report; no vendor-specific field names or live prices are implied.
from decimal import Decimal
def amount(input_units: int, output_units: int,
input_rate: Decimal, output_rate: Decimal) -> Decimal:
if min(input_units, output_units) < 0:
raise ValueError("usage units must be nonnegative")
if min(input_rate, output_rate) < 0:
raise ValueError("rates must be nonnegative")
return input_rate * input_units + output_rate * output_units
def reserved_amount(input_bound: int, output_limit: int,
input_rate: Decimal, output_rate: Decimal) -> Decimal:
return amount(input_bound, output_limit, input_rate, output_rate)
This function is not the reservation transaction. The transaction has to enforce a uniqueness key for each logical attempt and update the remaining budget consistently under concurrency. A second event with the same event identifier should be harmless; a genuinely new call from the same retrying session must remain distinguishable. Precision matters too: use decimal arithmetic and define a single invoice rounding policy, rather than rounding each small call and accumulating a different total.
Which records belong to a school?
Attribution is the governing constraint. Resolve the school from the authenticated application context before a call, and bind that identity to the reservation. Do not accept a tenant identifier supplied only by an agent's generated tool arguments. Keep the bound identity on the settlement event even if the student changes classes or an account is renamed later. The event should carry an immutable school key, attempt key, usage units, rate-version key, timestamps, and a status such as pending, settled, or corrected. A human-readable school name belongs in a separate, versioned mapping.
There is a practical compliance reason to avoid copying whole prompts into billing records: student text may be sensitive, while usage counts and opaque identifiers usually suffice to reproduce the arithmetic. Store credentials outside those records and rotate them without rewriting the invoice trail. The OWASP secrets guidance covers access control, rotation, and lifecycle handling for secrets; the billing ledger should refer to a credential identity only where that reference is operationally necessary.
Reconciliation needs a deliberate late-arrival rule. A report arriving after an invoice is issued should generate a traceable adjustment under a defined cutoff policy, not an in-place edit to a finalized line. Compare admitted attempts, completed requests, reported units, and settled ledger entries by school and period. Alert on missing reports, duplicate identifiers, negative corrections, and persistent gaps between reserved and settled amounts. For deployment, run a shadow ledger against existing invoice totals before making it authoritative; test simultaneous admissions, lost responses, duplicate callbacks, rate changes, and invoice cutoff boundaries with fixtures containing two schools. That last case catches cross-tenant attribution errors that a single-school test cannot.
What do we stop keeping?
Keep compact accounting events and correction history for the applicable invoice and dispute window. Delete full request and response bodies after a shorter, documented investigation window unless a specific retention obligation requires them. The 30-day payload window above is an example for capacity planning, not a universal retention rule. This choice reduces the dominant storage term and limits unnecessary exposure of student content. It has a cost: after payload deletion, an engineer may be unable to reconstruct exactly why a particular agent call consumed its reported units. The invoice can still be audited from its usage evidence, but the conversational cause of that usage may no longer be recoverable. Choose that trade-off with the billing and privacy owners, then test the deletion job as carefully as the settlement path.
Further reading
References:
- OWASP, Secrets Management Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- Python documentation,
decimalmodule: https://docs.python.org/3/library/decimal.html
Top comments (0)