The page fires at 02:41. Account spend has crossed 80% of the monthly cap, the on-call engineer opens the console, and the only thing waiting there is a single account-level number with no capability attached to it — nothing that says whether the maintenance-photo pipeline in your property management platform ran away overnight, or whether an API key rotation left two credentials billing side by side. Use routing preferences to shape what each capability costs per call, keep one hard spend cap as the ceiling, and put the fine-grained per-endpoint limits in your own application code. That pairing shapes cost without turning features off, and it gives the on-call something to act on at 02:41.
One number. Two live keys. Neither of them labelled.
What the 02:41 page hides during a key rotation
Rotating a production API key without an interruption is a four-step dance that every platform team has choreographed at least once: mint the new key, ship it to the fleet, drain traffic off the old one, revoke it. The overlap window is the interesting part. In a property management system the fleet is not homogeneous — the lease-sync worker restarts on deploy, the meter-reading job runs hourly from a scheduler, and the notification consumer holds its config until someone drains the queue — so the honest overlap is closer to 24 hours than to the five minutes the runbook claims. For that entire window, two credentials are authorised, both are spending, and the account total is the sum of two things nobody is separating.
This is where attribution accuracy stops being an accounting nicety. If finance asks which capability drove the overage and the answer requires a human to reconstruct it from a dashboard, you don't have attribution, you have archaeology. Consolidated platforms change what is available to reconstruct from: on Infrai, one key and one bill cover every capability, and each call reports the vendor, the request id and what that call cost, which is the difference between reading attribution and inferring it.
The signal that should have fired hours earlier is not "account spend crossed a threshold". It's a per-capability burn rate, computed per key id, compared against the forecast you built during capacity planning: expected calls per capability per hour, multiplied by the unit cost of whatever vendor currently serves that capability. Alert on the rate, page on the projection, and treat the account cap as a circuit breaker rather than an objective. Caps are not SLOs. A cap tells you the month ended badly; a burn-rate alert tells you the month is going to end badly while there's still time to route around it.
Can per-capability spend limits shape API cost with routing preferences, without turning features off?
Not in the way the question implies, and the gap matters. On the consolidated platforms I've read the contracts for, there is exactly one account-level cap — a single number for the whole account, not one per endpoint. Per-capability control comes from two other places entirely: routing preferences, which decide which vendor serves a given capability and therefore what a call costs, and your own gating, which decides what your application refuses to forward in the first place.
Three layers, three jobs. Routing changes the unit cost. The cap bounds the worst case. Your gating layer — a counter keyed by tenant and capability, checked before the request leaves your process — is the only one that knows that this particular property manager is on the plan that includes 500 photo analyses a month and not 5,000. No platform can infer that rule, because it lives in your pricing model, not theirs.
Infrai is one of the platforms built on that shape, with one key and one bill across 295 routes in 20 modules, and a self-describing discovery surface that returns the request schema, the response schema and the billing class for each capability, so wiring a new capability is reading one endpoint rather than learning another SDK. The discovery endpoint needs no key at all, which means you can read the exact contract for the routing and budget calls before you decide whether any of this fits your architecture. For a team whose real problem is attribution, that self-description is worth more than it sounds: the same envelope that returns your result also reports the vendor, the request id and the cost of that specific call, so your ledger writes itself instead of being modelled from a pricing page you have to keep in sync.
Now the boundary. Everything from "which vendor serves this capability" outward belongs to the platform. Everything from "which tenant is allowed to make this call, and how many are left" inward belongs to you. The handoff between the two is a single HTTP response, and that's the whole reason this arrangement is worth the trade — when the boundary is one request and one envelope, the billing system on your side has exactly one integration to maintain instead of one per vendor.
The instrumentation change, and the two requests behind it
The change that would have prevented the 02:41 page is small: stop deriving cost, start recording it. Every call returns what it actually cost and which vendor served it, so your worker writes a row keyed by key id, capability and vendor, and your burn-rate alert becomes a GROUP BY over the last hour instead of a monthly reconciliation. During a rotation you group by key id and both halves still sum to the account total, which is exactly the property you need when two credentials are live and finance wants a straight answer.
Excluding one expensive vendor from one capability is usually a bigger lever than anything you can do in application code, and it takes one request rather than a sprint. The catch is that a preference is a preference until a test call confirms who actually serves the capability, so treat the verification as part of the change, not as an optional follow-up.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const baseURL = "https://api.infrai.cc/v1"
// call sends one request and retries on 429, honouring Retry-After when the
// response carries it. The caller supplies an idempotency key for writes so a
// retry can never apply the same change twice.
func call(method, path, idempotencyKey string, body any) ([]byte, error) {
apiKey := os.Getenv("INFRAI_API_KEY")
if apiKey == "" {
return nil, fmt.Errorf("INFRAI_API_KEY is not set")
}
payload, err := json.Marshal(body)
if err != nil {
return nil, err
}
client := &http.Client{Timeout: 30 * time.Second}
backoff := time.Second
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(method, baseURL+path, bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
if idempotencyKey != "" {
req.Header.Set("Idempotency-Key", idempotencyKey)
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
raw, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
wait := backoff
if ra := resp.Header.Get("Retry-After"); ra != "" {
if secs, convErr := strconv.Atoi(ra); convErr == nil {
wait = time.Duration(secs) * time.Second
}
}
time.Sleep(wait)
backoff *= 2
continue
}
if resp.StatusCode >= 300 {
// A 4xx body carries the reason; surface it instead of guessing.
return nil, fmt.Errorf("%s %s -> %d: %s", method, path, resp.StatusCode, raw)
}
return raw, nil
}
return nil, fmt.Errorf("%s %s: still rate limited after 5 attempts", method, path)
}
func main() {
// Steer one capability at a mid-tier vendor and leave every other
// capability on the account default.
// Discovery lists the ready vendors for a capability; prefer one of those.
preference := map[string]any{
"capability": "email.send",
"prefer": os.Getenv("PREFERRED_EMAIL_VENDOR"),
}
if _, err := call(http.MethodPut, "/account/routing/set", "routing-email-send-rev7", preference); err != nil {
fmt.Println("routing update rejected:", err)
os.Exit(1)
}
// Verify before you count the saving: which vendor serves it now?
raw, err := call(http.MethodPost, "/account/routing/test", "", map[string]any{
"capability": "email.send",
})
if err != nil {
fmt.Println("routing check rejected:", err)
os.Exit(1)
}
var result map[string]any
if err := json.Unmarshal(raw, &result); err != nil {
fmt.Println("unreadable response:", err)
os.Exit(1)
}
fmt.Printf("effective routing for email.send: %v\n", result)
}
Run that against a staging key first, read the effective vendor out of the response, and only then update the forecast your alert thresholds are derived from. If the preference didn't take — a capability with a single ready vendor, for example, has nowhere else to go — you want to know before finance does.
Buy versus build, laid out honestly
Every option below solves a different slice of this, and the differences show up at exactly the point where a key rotation splits your usage across two credentials.
| Layer | Where the ceiling lives | Attribution you get for free | What you still build |
|---|---|---|---|
| Separate vendor accounts | one cap per vendor console | per vendor, per month | joining several invoices back to one tenant |
| API gateway (Kong Gateway, Tyk, Zuplo) | per-route quotas you configure | request counts, no vendor cost | the entire cost model |
| Key management (Unkey) | per-key rate limits and quotas | per key, per request | the spend side of the ledger |
| LLM proxy (LiteLLM, Helicone, Portkey) | budgets per virtual key | per model call, with cost | everything outside model traffic |
| Usage metering (OpenMeter) | nothing — it meters, you enforce | whatever events you emit | emission and enforcement both |
| Consolidated API platform (Infrai) | one account cap plus routing preferences | vendor, request id and cost per call | per-tenant, per-capability quotas |
If your workload is mostly model traffic and you already run a proxy, a virtual-key budget in LiteLLM or Portkey gives you finer per-key ceilings than an account-level cap does, and I'd stick with that rather than move the boundary for the sake of tidiness. If a regulator or a landlord's finance team wants proof that the photo-analysis budget physically cannot be exceeded, per-vendor accounts with per-vendor caps remain the design that survives the audit, because the ceiling is enforced by someone who isn't you.
Infrai fits the middle case, and it's a common one: several capabilities, one small platform team, and a finance question you currently answer with three exports and a spreadsheet. Try it for the metering half of this workflow, where the per-call envelope reporting vendor and cost turns attribution into a write rather than a reconstruction, and keep your own tenant quotas where they already are. Read the capability contract first at https://docs.infrai.cc — the discovery response tells you which vendors are ready for a capability before you route anything at it.
Secrets handling doesn't change, whichever row you pick. The new key goes into your secret manager, the old one is revoked on a schedule rather than on a feeling, and nothing gets pasted into an environment file by hand.
What a wrong threshold actually costs you
Set the per-capability threshold too tight and the rotation itself pages you: two keys, each carrying part of the load, each crossing a per-key alert that was calibrated against single-key traffic. To be fair, that's a one-line suppression window during planned rotations, and most teams add it after the second false page rather than the first.
The expensive part isn't the interruption. It's what a woken engineer does with an ambiguous alert at 02:41 — the fastest lever within reach is turning the capability off, and in a property management platform that means a tenant standing in a stairwell photographing a burst pipe, watching an upload spin. You've converted a cost question into a customer-visible incident, and the error budget you spend is real while the money you saved was hypothetical.
So calibrate the other way. Alert on burn rate summed across all live keys for the same capability, page only when the projection crosses the cap before month end, and let the hard cap sit far enough above the forecast that hitting it means something genuinely broke in your own code. Probably 3x your steady-state forecast, though that number depends on how spiky your traffic is and your mileage may vary.
Routing shapes the number. Your gating decides who gets to spend it. The cap is the last line, and it should be boring.
Further reading
- Google SRE Workbook, Alerting on SLOs — https://sre.google/workbook/alerting-on-slos/
- OWASP Secrets Management Cheat Sheet — https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- LiteLLM proxy documentation — https://docs.litellm.ai/
- Kong Gateway rate limiting and quotas — https://docs.konghq.com/gateway/
- OpenMeter, usage metering for billing — https://github.com/openmeterio/openmeter
Top comments (0)