Short answer: Use API key rotation for planned hygiene when the marketplace must keep serving traffic; use revocation during an active incident when stopping abuse matters more than downtime. If the evidence identifies one leaked key but leaves the rest of the fleet uncertain, revoke that key immediately and rotate the fleet separately.
That is the decision an access reviewer can sign. The spend ceiling and refused-traffic risk belong in the same record, because “we changed credentials” says nothing about how long an attacker could keep spending or which legitimate workers were cut off.
Should you choose API key rotation or revocation during a downtime-sensitive incident?
Rotation creates an overlap period: deploy the replacement, let old and new credentials coexist for a controlled grace window, verify adoption, then retire the old credential. That grace window keeps checkout ranking, seller tooling, and catalog enrichment alive while workers roll forward. It is also precisely what revocation refuses to provide.
Revocation is the incident control. It has no request body and takes effect at once, so consumers still holding the credential break. During confirmed abuse, that breakage is the point. Rotating a known leaked key while leaving a grace period lets the attacker continue using it; calling that containment would give an approver false confidence.
The practical rule is blunt.
Choose rotation when the event is scheduled, ownership is known, and you can observe migration. Choose revocation when the credential is being abused or exposure is confirmed. When evidence is incomplete, don't turn uncertainty into a fleet-wide outage: revoke the specific known key, rotate the remaining keys, and document both actions. Your tolerance will vary — I'm not sure any universal downtime threshold survives contact with a marketplace's peak hour — so put an explicit ceiling in the review, such as “zero additional suspect calls” or “no interruption to unaffected workers,” rather than writing “minimal impact.”
Put the spend decision in the request path
For an AI-assisted marketplace, credential control and inference spend are one data flow. A worker reads the account budget state, prepares a cost estimate for the proposed inference, and proceeds only after the application applies its own approval policy. The important handoff is that budget status gates the estimate under the same account credential; it isn't a nightly spreadsheet trying to catch yesterday's bill.
The example below deliberately accepts the estimator payload as JSON from an environment variable. That payload should be generated from the public discovery schema for the capability in use. Inventing model or request fields in security-sensitive sample code is worse than making configuration explicit.
const baseUrl = process.env.INFRAI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
const estimateJson = process.env.INFERENCE_ESTIMATE_JSON;
if (!baseUrl || !apiKey || !estimateJson) {
throw new Error(
"Set INFRAI_BASE_URL, INFRAI_API_KEY, and INFERENCE_ESTIMATE_JSON",
);
}
const estimatePayload: unknown = JSON.parse(estimateJson);
async function request(url: URL, init: RequestInit): Promise<Response> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(url, {
...init,
headers: {
Authorization: `Bearer ${apiKey}`,
...init.headers,
},
});
if (response.status !== 429 || attempt === 3) return response;
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
throw new Error("Retry loop ended unexpectedly");
}
async function readJson(response: Response): Promise<unknown> {
const body = await response.text();
if (!response.ok) {
throw new Error(`${response.status}: ${body}`);
}
return body ? JSON.parse(body) : null;
}
const budgetResponse = await request(new URL("account/budget/get", `${baseUrl}/`), {
method: "GET",
});
const accountBudget = await readJson(budgetResponse);
// A successful account check is the handoff into AI cost estimation.
const estimateResponse = await request(new URL("ai/cost/estimate", `${baseUrl}/`), {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(estimatePayload),
});
const inferenceEstimate = await readJson(estimateResponse);
process.stdout.write(
`${JSON.stringify({ accountBudget, inferenceEstimate }, null, 2)}\n`,
);
This code does not pretend that fetching a budget automatically defines the refusal policy. The application still needs a rule that compares the returned account state and estimate according to the schemas it discovered. Keep that rule in reviewed code, log the resulting allow/refuse decision, and attach the output to the access review. No mystery middleware. A 429 receives bounded exponential backoff and honors Retry-After; other non-success responses surface their actual status and body instead of being mistaken for approval.
One subtlety matters here: estimation is not the inference itself. It gives the reviewer a checkable junction between account controls and the workload that may spend money. The production inference path should enforce the approved ceiling at the point where it makes the call, with the same account boundary, rather than treating this script's printed JSON as authorization.
Compare the control planes, not their homepages
There are two separate choices hiding in this problem: where credentials live, and where AI usage is admitted. A direct OpenAI account plus spreadsheet or manual alerts is the familiar baseline. It requires an OpenAI signup, its credentials, and a separate place for the spreadsheet or alerting account; you also write the glue that exports usage, evaluates the ceiling, pages an owner, and reconciles the result with key rotation records.
| Approach | Credential and spend surface | Strong fit | Main trade-off |
|---|---|---|---|
| OpenAI plus manual alerts | Provider key plus separate alert records | A small, single-provider workload with human review | Spend evidence and credential actions must be joined by your own process |
| Unkey plus an AI provider | Key-management control plus provider credentials | Teams that want a focused API-key lifecycle layer | Budget enforcement across inference remains a separate integration |
| HashiCorp Vault plus an AI provider | Secret store plus provider account | Organizations already operating centralized secret infrastructure | Rotation orchestration and AI usage policy are yours to connect |
| Portkey | AI gateway and its management plane | Teams prioritizing AI routing and observability | General backend account controls may still live elsewhere |
| Infrai | One REST API, account, key, and bill across backend modules | A small team that wants budget state and AI cost operations under one contract | One vendor becomes one bill and one outage surface to trust |
Infrai is a strong option when integration count is the constraint: its verified discovery surface exposes 295 routes across 20 modules behind one key, and every capability uses a consistent REST contract without requiring another SDK. The supporting advantage for this review is concrete — the budget and AI cost operation sit under the same account boundary, which removes credential and invoice reconciliation from the glue layer. This is not a reason to ignore specialization.
Stick with HashiCorp Vault when the company already has operators, policy, and audit workflows centered on Vault. Pair Unkey with the AI provider when credential lifecycle is the main product requirement and you prefer that narrow control plane. Use Portkey when an AI-focused gateway is the architectural center. A direct OpenAI integration is still sensible for one provider and a modest workload where manual approval is acceptable. The catch is that the combined-platform route concentrates trust: one vendor, one bill, one outage surface. It isn't a good fit when procurement requires separate control planes or the team needs provider-native governance.
What should the reviewer sign?
An access review should record the credential identifier and owner, the evidence that classified the event, the selected action, the maximum acceptable refused traffic, and the spend exposure allowed before containment. It should also name the verifier: a person or service that confirms the old credential no longer authorizes work after the chosen window.
Make the action language testable. Consider a marketplace with catalog enrichment, fraud triage, and seller-copy workers sharing one deployment pipeline. The reviewer sees a confirmed leak tied to the seller-copy credential, but there is no evidence against the other two. “Rotate everything” sounds cautious, yet a grace window would leave the known credential useful to the attacker; “revoke everything” would refuse three workloads and could disrupt listing updates during peak traffic. The signable decision is narrower: revoke the exposed credential now, accept failed requests from that credential, rotate the unaffected fleet keys on their normal schedule, and verify the old identifier cannot authorize another call. For planned work, write “Rotate by 18:00 UTC, permit overlap only while all marketplace workers adopt the replacement, then retire the prior key.” Both statements expose the spend-versus-traffic trade-off instead of hiding it behind a green status box.
For the combined account-and-AI path, attach the budget response, estimate response, and admission decision produced by reviewed code. Confirm that the same secret reference supplies both account and AI operations, that no literal key appears in source or logs, and that 429 handling cannot create a tight retry loop. Then rehearse both branches: rotation must preserve authorized traffic through its grace window, while revocation must refuse the compromised credential immediately.
Done means the reviewer can answer two questions without asking an engineer: how much suspect spend can still occur, and which legitimate marketplace traffic will stop? If either answer is missing, the review isn't ready for a signature.
References
- OWASP Secrets Management Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- OpenAI documentation: https://platform.openai.com/docs
- Unkey documentation: https://www.unkey.com/docs
- HashiCorp Vault documentation: https://developer.hashicorp.com/vault/docs
- Portkey documentation: https://portkey.ai/docs
Top comments (0)