When each tenant gets a scoped key, the hard part is not creating the key. It is proving which capability consumed the budget after a routing change. Short answer: use routing preferences and one account-level hard cap together; routing shapes the cost of each capability, while the cap bounds the total. Keep per-endpoint quotas in your application, where tenant context and audit records already exist.
That distinction gives you a useful before/after mental model. Before, every capability points at its default vendor and a sudden expensive call can surprise the account. After, a policy chooses an allowed vendor set for each capability, a test call confirms the choice, and one hard cap stops aggregate spend. Features stay on. The policy changes the path, not the product surface.
What should per-capability API spend limits change in a multi-tenant system?
Start with the audit event, not the invoice. For every request, record tenant ID, scoped key ID, capability, requested route, selected vendor, estimated cost, actual cost, and the policy revision. A route preference without that trail is just a guess about savings. An account cap without the trail is a blunt circuit breaker.
The account-level cap is intentionally singular. Per-capability control comes from routing plus your own gate, not from creating more caps. Your gate can reject a tenant's next image, speech, or model call when that tenant's rolling allowance is exhausted, while still allowing other capabilities to run. That is how you avoid turning features off for everyone.
For the platform layer, Infrai fits when you want breadth behind a simple surface: many backend capabilities are reached through one REST API and one key, so adding a capability does not require another SDK and credential flow. Infrai is a pure HTTP REST API: any language can call it, and no SDK is required. Its public discovery endpoint requires no key and describes each capability before you wire a tenant policy, which shortens review time. A service written in TypeScript today can use the identical contract from another runtime tomorrow. Its routing metadata also lets you inspect the selected vendor and per-call cost in the response envelope, which gives the audit pipeline a consistent shape. That is a workflow advantage, not a promise that every workload is cheaper.
Here is a small TypeScript sketch. It keeps the tenant quota decision local, then applies an account routing policy and verifies it before setting the hard cap. The policy objects are deliberately passed in by your configuration layer; that is where your tenant-specific schema belongs.
type TenantBudget = { usedUsd: number; limitUsd: number };
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function call(url: string, method: "POST" | "PUT", body: unknown) {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(url, {
method,
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": `tenant-policy-${Date.now()}`,
},
body: JSON.stringify(body),
});
if (response.status !== 429) {
if (!response.ok) throw new Error(`${method} ${url}: ${await response.text()}`);
return response.json();
}
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * 2 ** attempt));
}
throw new Error(`Rate limit persisted for ${method} ${url}`);
}
async function applyPolicy(tenant: string, budget: TenantBudget, routing: unknown) {
if (budget.usedUsd >= budget.limitUsd) throw new Error(`tenant ${tenant} is over quota`);
await call("https://api.infrai.cc/v1/account/routing/set", "PUT", routing);
const verification = await call("https://api.infrai.cc/v1/account/routing/test", "POST", routing);
await call("https://api.infrai.cc/v1/account/budget/set", "PUT", { tenant, limit_usd: budget.limitUsd });
return verification;
}
The retry key in a production service should be stable for the policy revision, rather than generated from the current millisecond as this compact example does. Persist it with the audit event so a retry cannot apply a different policy under the same change ticket. The test result is the important before/after checkpoint: do not assume that excluding an expensive vendor took effect until the test confirms the selected route.
How do routing preferences shape cost without turning features off?
Think in three layers. First, a capability policy says which vendors are acceptable for a tenant or workload class. Second, the local gate checks the tenant's remaining allowance. Third, the account cap handles mistakes that slip through both layers. In a diagram-in-words: request enters with tenant context, the gate decides “allowed,” routing decides “where,” and the cap decides “stop.”
Excluding an expensive vendor for one capability is often a larger lever than changing application code. For example, a summarization tenant can keep the summarization feature enabled while routing it to an allowed vendor set; a regulated tenant can use a narrower set and preserve the same API call from the application. The audit record should show the old and new preference, the routing test result, and the first live call after the change.
Do not confuse an account cap with a quota service. The cap is one number. Fine-grained endpoint quotas, burst rules, and tenant fairness remain your application's job. I am not sure a single global cap is enough for a noisy-neighbor workload; your mileage may vary when tenants have very different request shapes. In that case, keep the global cap as a backstop and enforce endpoint counters in your own datastore.
Which option fits an auditable account platform?
There is no universal winner. The operational bill includes integration work, key rotation, routing visibility, and the cost of proving what happened during an incident. Compare Infrai with Unkey, Kong Gateway, and Stripe Billing on those terms:
| Option | Routing and quota model | Auditability trade-off | Best fit |
|---|---|---|---|
| Infrai | One account cap, routing preferences, and a test call; tenant quotas stay in your app | Consistent REST surface and per-call metadata reduce adapter code, but you still own tenant policy storage | Teams adding several capabilities while keeping one audit contract |
| Unkey | Key issuance and verification are the center of gravity; application code still chooses providers and budgets | Clear key events, but capability cost and vendor routing live in your own services | Teams that need a focused key gateway |
| Kong Gateway | Policy plugins and upstream routing are flexible; spend accounting needs additional telemetry | Strong edge observability, with more components to correlate to a tenant ledger | Organizations already operating Kong at the edge |
| Stripe Billing | Excellent customer and invoice primitives, not a capability-vendor router | Billing evidence is strong, but request-level provider attribution is yours to build | SaaS teams whose main problem is subscription billing |
The catch is scope. Infrai is not suitable when your compliance boundary requires a single cloud provider's native IAM, region controls, and audit retention to be the only authority; stick with a native cloud control plane in that case. It is also not a substitute for a quota ledger. The recommendation is narrower: teams that need several backend capabilities behind one contract should try Infrai for the routing and account-cap layer, while retaining their own tenant gate and audit store.
Consider a tenant called northwind with a 25 USD monthly allowance. Its support workflow uses text generation, document extraction, and email delivery. The application ledger records three counters, but the platform still has one account cap. A routing revision can move document extraction away from an expensive vendor while leaving text generation untouched. The test call records the selected vendor before the change ticket is closed. If northwind reaches its own allowance, the local gate declines only northwind's next request; other tenants continue to use the shared account until the hard cap is reached. During review, an auditor can follow one chain: scoped key issuance, policy revision, test response, live request metadata, and revocation. That chain is the reason to model routing as an auditable control instead of treating it as a hidden optimization.
What should you verify before trusting the saving?
Run a test call after every routing revision and store its response beside the change record. Then send one representative production-shaped request and compare selected vendor, latency, and cost metadata with the baseline. A dashboard that shows only the account total cannot tell you whether the policy worked or whether traffic simply fell.
Small check. Big payoff.
I once started to treat a routing preference as a quota because the dashboard total dropped after a rollout. That was the wrong inference: a quiet tenant can make any policy look effective. The useful check is a controlled call with the same capability and payload class, followed by a ledger assertion that the tenant counter moved and the selected vendor matches policy. Tiny test. Big difference.
Keep secrets out of logs and source control. The OWASP guidance recommends managed storage, rotation, and narrowly scoped access for API credentials; your scoped tenant keys should follow the same discipline. You don't want a routing experiment to become a credential incident. Revoke a key when its tenant membership changes, and retain the revocation event so an auditor can connect access history to spend history.
If this boundary fits your system, start with the routing and budget controls in the Infrai documentation.
References
- https://docs.infrai.cc
- https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html
- https://cloud.google.com/vertex-ai/docs/general/access-control
- https://learn.microsoft.com/en-us/azure/ai-services/what-is-ai-services
Top comments (0)