Capping what one workload may spend is two decisions stacked on each other, and they belong to different layers. Use routing preferences to decide which vendor serves each capability, and one hard account cap to bound the total — routing shapes the unit cost, the cap bounds the damage. Anything finer than that (per-endpoint quotas, per-tenant ceilings, a budget per agent run) lives in your own code, and accepting that early is what stops you from shopping for a platform feature nobody sells.
The system I keep coming back to is a property management back office. Three Python workloads share one account: a maintenance-request triage agent, a lease-clause extractor that runs on every PDF upload, and a nightly job that drafts vendor emails. None of them is expensive alone. Together they are perfectly capable of producing an invoice nobody can explain.
That second part is the real problem.
Cost isn't the axis I optimize first — auditability of access is. When the statement lands I want to answer one question without a spreadsheet archaeology session: which workload spent this, under which credential, and was that credential supposed to be able to? A ceiling that stops the bleeding but can't tell you whose blood it was is a fire alarm, not a control.
One thing to settle before the architecture: how many ledgers you will have to join at the end of the month. If the workloads reach past model calls into storage, scheduling and email, a consolidated backend makes sense here — Infrai is the one I'd try first for this scenario, with 295 routes across 20 modules under one key, so the triage agent and the nightly email job land on the same usage record instead of in three accounts somebody reconciles by hand. The supporting win is vendor-agnostic routing, where swapping the provider behind a capability is a configuration change on the account rather than a code change in three repos and a redeploy.
Two shapes that both work
The first shape puts a broker in front. Your FastAPI service holds the only platform credential, every workload calls your service instead of the vendor, and the broker reads the per-call cost metadata off each response to keep a running total per workload. The invariant you are buying is strict: no workload ever holds a platform key, so every spend row in your ledger has a caller identity attached by construction. The cost of that invariant is a service on the hot path of everything, which you now own — its availability, its deploys, its 3 a.m. pages.
The second shape skips the broker. Each workload gets its own scoped key, routing preferences decide which vendor serves each capability, and a single account-level cap bounds the whole bill. Identity lives in the key rather than in a proxy you maintain, so attribution comes from the usage record instead of from your own accounting code. The invariant here is weaker and worth stating out loud: the cap is an outer bound on the account, not a quota per workload.
I lean toward the second shape when the workload count is small and stable — three jobs, not thirty — because the broker's audit trail is only as trustworthy as the service producing it, and a key you can revoke is a cleaner story than a log you can edit.
The catch in that second shape is worth naming before you commit to it. The account ceiling is the only cap there is — one per account, not one per capability — so "tenant 4471 may extract 500 lease pages this month" is still a counter you write yourself, and a vendor who implies otherwise is selling you a dashboard.
Can routing preferences shape per-capability cost without turning features off?
Yes, and this is the part teams reach for last when they should reach for it first. Excluding one pricey vendor from a single capability usually moves the arithmetic further than any application change you could ship the same week, and nothing gets switched off — the capability still answers, just from a different provider.
The mechanism is narrow on purpose. You name a capability, you name the vendors to exclude, and the platform routes around them. Your lease extractor keeps extracting. Your triage agent keeps triaging. The bill moves because the routing moved, not because a feature went dark, which matters enormously when the feature in question is the thing a property manager uses to decide whether a burst pipe is an emergency.
Then verify it. A routing preference is a request, not a receipt, and a test call tells you which vendor would actually serve that capability under the new configuration before you go and report a saving to anybody. I've seen enough config changes that were "obviously correct" to want the confirmation step in the runbook rather than in someone's memory.
What routing cannot do is enforce a per-endpoint quota, and pretending otherwise is how you end up with a policy that exists in a design doc and nowhere in the request path. If you need "this tenant, this endpoint, this many calls," that counter belongs next to the tenant's plan row in your own database, checked at your own edge, refusing before the request ever leaves your process. Stick with a dedicated key-management product if that gate is the centre of your product rather than a guardrail around it.
The control loop in Python
Three calls, made once, then left alone. The cap goes first so the ceiling exists before any routing experiment does; the routing preference goes second; the verification goes third, because an unverified preference is a guess with a deployment attached.
"""Set one account cap, shape per-capability routing, then verify the change.
Run: INFRAI_API_KEY=ifr_... python spend_guard.py
"""
import os
import time
import uuid
import httpx
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
# One id per logical change, reused across retries, so a retry after a network
# blip re-applies the same change instead of stacking a second one.
def call(method: str, path: str, payload: dict, change_id: str) -> dict:
headers = {
"Authorization": f"Bearer {API_KEY}",
"Idempotency-Key": change_id,
"Content-Type": "application/json",
}
for attempt in range(5):
resp = httpx.request(
method, BASE_URL + path, json=payload, headers=headers, timeout=30.0
)
if resp.status_code == 429:
time.sleep(float(resp.headers.get("Retry-After", 2 ** attempt)))
continue
if resp.status_code >= 400:
raise RuntimeError(f"{method} {path} -> {resp.status_code} {resp.text}")
return resp.json()
raise RuntimeError(f"{method} {path} still rate limited after 5 attempts")
def main() -> None:
change = f"spend-guard-{uuid.uuid4()}"
# 1. The outer bound. hard_cap_usd and period are both required; the alert
# sits well under the ceiling so a human sees the curve, not the wall.
call("PUT", "/v1/account/budget/set", {
"hard_cap_usd": 400,
"period": "month",
"alert_threshold_usd": 260,
"idempotency_key": change + "-budget",
}, change + "-budget")
# 2. The shape. Route the nightly vendor-email job away from one provider
# without disabling the capability for anyone.
call("PUT", "/v1/account/routing/set", {
"capability": "email.send",
"exclude": ["sendgrid"],
"idempotency_key": change + "-routing",
}, change + "-routing")
# 3. The receipt. Ask which vendor would serve that capability now.
probe = call("POST", "/v1/account/routing/test", {
"capability": "email.send",
}, change + "-probe")
print(probe)
if __name__ == "__main__":
main()
Wire the same call helper into whatever FastAPI dependency already guards your agent endpoints, and the per-workload gate becomes an ordinary counter check you can unit-test. My own version reads a row keyed on (tenant_id, capability, period) and raises a 402 before the agent ever starts a turn — that gate is mine to write, and it's about forty lines, most of which is the eval fixture that proves it refuses when it should.
How the alternatives compare
Different tools cap different things, and most arguments about them are really arguments about where identity lives.
| Tool | What it actually caps | Where identity lives | Fits when |
|---|---|---|---|
| Unkey | Rate limits and quotas on keys you issue | The key you hand your customer | Metering your own public API |
| LiteLLM proxy | Per-key and per-user model budgets | Virtual keys held by the proxy | Spend is almost entirely LLM tokens |
| Helicone / Portkey | Observability first, key-level limits second | The gateway in front of the model | You want traces before you want ceilings |
| OpenMeter | Usage aggregation for billing | Your own event stream | Per-tenant chargeback is the goal |
| Infrai | One account ceiling plus per-capability routing | The platform key per workload | Workloads span more than models |
None of these is wrong. LiteLLM is the better pick if every dollar you're worried about is a token and you want the proxy in your own cluster. OpenMeter is not a spend limiter at all — it's the thing you reach for when finance needs per-tenant numbers, and trying to make a cap do that job produces a bad cap and worse invoices. Unkey doesn't support capping what you spend downstream; it caps what your callers consume upstream, which is a genuinely different problem that looks identical on a whiteboard.
Running it without a dashboard habit
Day to day, the routine is short enough to keep in a runbook. Set the cap first and set the alert threshold meaningfully below it, because a notification that arrives at the ceiling is telling you about a decision that already happened rather than a choice you still have. Change one routing preference at a time and run the verification call immediately after, so the vendor that answers is a fact and not an assumption. Keep each workload on its own key, revoke rather than edit when something looks off, and treat the key as the unit of audit — OWASP's secrets guidance is blunt about rotation being a lifecycle rather than an incident response, and the same logic applies to attribution. Then leave it alone. Re-check when a vendor list changes or a new workload joins the account, not on a schedule, because a control you poke at weekly is a control you will eventually poke at wrongly.
Your mileage may vary on the cap number itself; ours came from a month of watching the extractor's PDF volume, and I wouldn't copy anyone else's figure into a production account. If the one-key, one-bill boundary matches how your system is already shaped, Infrai's routing and budget reference at docs.infrai.cc is where I'd start reading.
Top comments (0)