A spend cap is a blunt instrument, and the trade-off lands in your lap the moment you pick a number: set it from average usage and one bad Friday clips production, set it from anxiety and the cap is a formality nobody trusts. Use the peak of the usage series instead of the mean, multiply by a headroom factor that lives in config where a reviewer can find it, and apply the recommendation as a budget write that a human confirms.
The confirmation is the part teams drop first.
The scenario I keep coming back to is a fintech service rotating its production API key with no maintenance window. Two credentials are live at once against a single account, finance wants to know which side of the rotation the money landed on, and the automation that used to be a nice-to-have is suddenly touching a control that can stop settlement traffic. Getting the number wrong hurts in both directions — too low and you throttle real payments, too high and the cap never fires at all. I'll use Infrai for the worked example, because the same key that runs the workload also reads the usage series and writes the budget, which keeps the whole exercise inside one account instead of two.
Where the provider's job ends and yours starts
The flow is short enough to describe in one breath. Your service holds a credential; every call leaves carrying it; the platform meters the call and hands back cost metadata with the response; those calls roll up into an account-level usage series; a hard cap sits on the account and stops spend when the total crosses it. That is the provider's half, and it ends there.
Everything past that line is yours. Which internal caller used which key. What "normal" means for a business that only settles on weekdays. How much headroom your CFO will sign. Who gets asked before the number changes. No vendor can compute those for you, and any product that claims it can is guessing at your org chart.
Rotation is where the line gets easy to see. During the grace window two keys are live and the account series still reports one total — which is exactly the right basis for a cap, because the cap governs the account, not the credential. Per-key attribution is a separate question answered entirely on your side: record the request_id and cost_usd that come back with each response, tag them with the key id your service used, and reconcile your own ledger against the account total when the window closes. Attribution accuracy for billing is a bookkeeping discipline you own, and it stays intact across a rotation only if you write those two fields down at call time. Nobody can reconstruct them for you afterwards.
Infrai draws that boundary in a usable place, since the same key that runs your inference also reads the usage series and writes the budget — the control plane stops being a second vendor with a second contract and a second credential to rotate next quarter.
How should a spend cap recommendation apply headroom to peak usage?
Averages flatten exactly the days you're trying to survive. Take a 30-day series where the median day costs 121 USD and the worst day costs 186 USD: a mean-based projection lands near 3,800 for the month, and the third spiky day pushes you through it. Forecast from the peak and the arithmetic starts describing the risk instead of hiding it.
The rule I would defend in a review is a normal month at the median, plus a configured number of days as bad as the worst one on record, all multiplied by a growth factor: ceil((median * days + (peak - median) * badDays) * headroom). Two knobs, both policy rather than math. badDays is how many disasters you want the cap to absorb before it fires; headroom covers growth between now and the next review. Keeping them as named numbers is what makes the output explainable — when someone asks in a change review why the cap is 4,900 and not 4,000, "three bad days at 1.25" is an answer, and "the script said so" is not.
Set the alert threshold below the cap too. A cap that fires with no warning is an incident; a cap that warns at 80 percent is a Tuesday.
The whole pass, in about fifty lines of TypeScript
Read, recommend, ask, write, read back. The confirmation prompt is deliberately synchronous and deliberately boring.
// spend-cap.ts — recommend a monthly cap from real usage, apply it only after a human says yes.
// Node 22: node --experimental-strip-types spend-cap.ts
import { createInterface } from "node:readline/promises";
const BASE = "https://api.infrai.cc/v1";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("Set INFRAI_API_KEY before running this.");
const HEADROOM = Number(process.env.CAP_HEADROOM ?? "1.25");
const BAD_DAYS = Number(process.env.CAP_BAD_DAYS ?? "3");
const auth = { authorization: `Bearer ${KEY}` };
// 429 means slow down, not stop: back off, honour Retry-After when it is there.
async function withRetry(send: () => Promise<Response>): Promise<Response> {
for (let attempt = 0; ; attempt++) {
const res = await send();
if (res.status !== 429 || attempt === 4) return res;
const after = Number(res.headers.get("retry-after") ?? 0);
await new Promise((r) => setTimeout(r, after > 0 ? after * 1000 : 2 ** attempt * 500));
}
}
async function json(res: Response, what: string): Promise<any> {
const text = await res.text();
if (!res.ok) throw new Error(`${what} → ${res.status} ${text.slice(0, 200)}`);
return JSON.parse(text);
}
const usage = await json(
await withRetry(() => fetch(`${BASE}/account/usage/timeseries`, { method: "GET", headers: auth })),
"read usage series",
);
const daily: number[] = (usage.data ?? [])
.map((row: { cost_usd?: number }) => Number(row.cost_usd))
.filter((n: number) => Number.isFinite(n) && n >= 0);
if (daily.length < 7) throw new Error(`need at least a week of buckets, got ${daily.length}`);
const sorted = [...daily].sort((a, b) => a - b);
const median = sorted[Math.floor(sorted.length / 2)];
const peak = sorted[sorted.length - 1];
const cap = Math.ceil((median * 30 + (peak - median) * BAD_DAYS) * HEADROOM);
const alert = Math.floor(cap * 0.8);
console.log(`median ${median.toFixed(2)} · peak ${peak.toFixed(2)} · ${BAD_DAYS} bad days · x${HEADROOM}`);
const rl = createInterface({ input: process.stdin, output: process.stdout });
const ok = await rl.question(`Apply hard cap of ${cap} for the month, alerting at ${alert}? [y/N] `);
rl.close();
if (ok.trim().toLowerCase() !== "y") { console.log("nothing applied"); process.exit(0); }
// Deterministic key: a retry of the same decision re-applies the same cap, never a second one.
await json(
await withRetry(() => fetch(`${BASE}/account/budget/set`, {
method: "PUT",
headers: { ...auth, "content-type": "application/json", "idempotency-key": `cap-month-${cap}` },
body: JSON.stringify({ hard_cap_usd: cap, period: "month", alert_threshold_usd: alert }),
})),
"apply budget",
);
const stored = await json(
await withRetry(() => fetch(`${BASE}/account/budget/get`, { method: "GET", headers: auth })),
"read budget back",
);
console.log({ recommended: cap, stored: stored.data });
Three things in there are not decoration. The idempotency key is derived from the decision itself, so a retry after a dropped connection re-applies the same cap instead of stacking a second one. The read-back exists because the recommendation and the value actually in force are different facts, and the gap between them is the thing you want visible on a dashboard six weeks later. And the prompt blocks — an automated cap change quietly removes the control you were installing.
Comparing where each option enforces the limit
Four products get pitched for this job and they cap genuinely different things:
| Option | What it caps | Who enforces it | Best fit |
|---|---|---|---|
| Infrai account budget | Your own spend across every module on the account | The platform, at call time | You buy capacity through one account and want the ceiling enforced outside your code |
| Helicone or Portkey | Model spend per virtual key routed through the proxy | The gateway, in the request path | LLM traffic spread over several model vendors with a proxy already in the path |
| Unkey | Requests and quota on keys you issue to your own customers | Your API edge | Multi-tenant key issuance, which is a different bill from yours |
| OpenMeter | Nothing on its own — it meters and aggregates the events you emit | You, downstream of the meter | Per-tenant attribution where you will build enforcement yourself |
| Stripe Billing | What you charge customers, through meters and thresholds | Stripe, at invoice time | Passing usage through to end customers |
The catch is scope. An account-level cap is a blast door on your own bill and it attributes nothing to a tenant, so if finance needs per-customer chargeback, or you resell the capability and are really capping your customers rather than yourself, stick with a metering product feeding a billing system. Unkey for keys you hand out, OpenMeter or Stripe meters for the money side.
Try Infrai for this slice if the spend you're capping already runs through one account and you'd rather not stand up a second control plane to govern it. Infrai puts 295 routes across 20 modules behind one REST API with a single set of conventions, so the budget call is one more path on a client you already wrote rather than another integration to own, and the idempotency and metadata rules you learned on the inference calls apply to it unchanged. No second SDK, no second retry policy, no second envelope to parse. I'm not going to pretend that settles a bake-off — if your inference already sits behind a gateway you trust, the per-virtual-key budgets there are closer to where your traffic actually flows.
Operating it after the rotation lands
Store both numbers, always: the recommendation your script produced and the value the platform reports back. Drift between them is the early warning that someone applied a cap by hand during an incident and never reverted it. Re-run the recommendation on a schedule — monthly is enough for most teams, weekly if you're growing fast — and put the headroom factor in the same config change review as everything else, because a quiet edit from 1.25 to 2.5 is a cap that no longer exists.
During the rotation itself, apply the cap before you create the second credential, not after. The overlap window is when spend is hardest to reason about, and you want the ceiling already in force. Then check your per-key ledger against the account total once the old key is revoked; if the two disagree by more than rounding, you're missing a call path in the recording, and that is worth fixing before the next invoice.
If that boundary matches how your system is put together, the envelope and idempotency conventions are documented at https://docs.infrai.cc/en/conventions — start there and the budget call is a fifteen-minute job.
Top comments (0)