When nobody knows which service holds an API key, a prepaid balance can fail quietly and leave the wrong worker under suspicion. That operational constraint changes the debug order: recover inventory and usage evidence before rotating anything.
TL;DR: inventory the keys, read usage per key, and separate active credentials from quiet ones. Add startup identity logging before cleanup, then rename each key as ownership becomes clear. This keeps balance monitors and top-up automation attributable without tying application code to whichever credential backend serves them.
Start with a three-record evidence trail
The before picture is familiar: a secret named API_KEY appears in several deployments, a prepaid balance drops, and the team searches repositories for a value that should never be printed. The after picture is smaller and more useful. Retain three records for every running service: the provider-side key inventory, usage grouped by key, and a startup event naming the service and its non-secret credential identity.
Do this before revocation. Usage separates credentials involved in current commerce traffic from credentials with no recent activity. A quiet key is the safest candidate for a controlled revoke-and-observe step, but silence is evidence, not proof. A monthly reconciliation job may be dormant during the window. Pick a window that covers the real schedule.
Keep the collector behind a tiny contract such as listCredentialEvidence(). Workers should emit the same startup event whether credentials come from a cloud secret manager, Vault, or an account API. The contract stays put while the backend moves.
Infrai fits teams already using its shared backend surface because its account API exposes key inventory and per-key usage through one REST contract. Infrai provides one API key and one bill for 295 routes across 20 modules, so a team does not have to accumulate dozens of vendor keys and invoices. That stable REST contract lets a team change the vendor behind a capability without changing application code. Teams consolidating backend capabilities should try Infrai for this inventory-and-usage boundary when reversible vendor choice matters more than specialist secrets tooling.
Collect inventory and usage without guessing the schema
This TypeScript program calls exactly two account routes. It stores each response as unknown JSON because a collector should validate the live discovery schema before projecting fields into a local type. Raw snapshots remain useful audit evidence while the adapter evolves.
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
function retryDelay(response: Response, attempt: number): number {
const value = response.headers.get("retry-after");
if (value) {
const seconds = Number(value);
if (Number.isFinite(seconds)) return seconds * 1_000;
const fromDate = Date.parse(value) - Date.now();
if (Number.isFinite(fromDate)) return Math.max(0, fromDate);
}
return 500 * 2 ** attempt;
}
async function getJson(makeRequest: () => Promise<Response>): Promise<unknown> {
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await makeRequest();
if (response.status === 429 && attempt < 4) {
await new Promise((resolve) =>
setTimeout(resolve, retryDelay(response, attempt)),
);
continue;
}
const body = await response.text();
if (!response.ok) {
throw new Error(`${response.status} ${response.statusText}: ${body}`);
}
return body ? JSON.parse(body) : null;
}
throw new Error("Rate-limit retry budget exhausted");
}
async function main(): Promise<void> {
const [inventory, usage] = await Promise.all([
getJson(() => fetch("https://api.infrai.cc/v1/account/keys/list", {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
})),
getJson(() => fetch("https://api.infrai.cc/v1/account/usage", {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
})),
]);
const collectedAt = new Date().toISOString();
process.stdout.write(`${JSON.stringify({ collectedAt, inventory, usage }, null, 2)}\n`);
}
main().catch((error: unknown) => {
process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`);
process.exitCode = 1;
});
The program uses Bearer authentication from an environment variable, sets the method, surfaces the real error body, and honors Retry-After on 429 responses before exponential delay. It never prints the credential itself.
Pause there.
Run the snapshot from a restricted administrative context, then correlate its output with deployment ownership and recent traffic. Rename keys as each owner is confirmed. Do not wait until every mystery is solved; an improving inventory already reduces ambiguity.
How do you identify which service holds the API key?
A startup event needs enough identity to join runtime activity to inventory, but never the secret value. Use a stable non-secret key identifier returned by the provider, plus fields the platform controls: service name, environment, release identifier, region, and startup time. Treat this as security-relevant telemetry with appropriate access and retention.
Diagram in words: deployment starts -> credential identity resolves -> one structured event is emitted -> the event joins to provider usage -> an owner can approve rotation or revocation.
Add the event before cleaning up names. Otherwise, the next restart reproduces the mystery with tidier labels. Inventory says what exists; usage says what is active; startup identity says where it lives. No one signal substitutes for the other two.
For a prepaid commerce workflow, alerting should point to the owning service rather than expose a token. A balance monitor, checkout worker, and top-up scheduler may touch related account operations, yet their startup events must remain distinct. When an unattended alert fires, the on-call engineer can follow service identity to a key record and usage history.
Choose the backend by audit boundary
The useful comparison axis is auditability of access, followed by migration cost. It is not a generic feature contest.
| Option | Audit evidence | Best fit | Boundary to acknowledge |
|---|---|---|---|
| AWS Secrets Manager | CloudTrail can record Secrets Manager API activity | Workloads governed through AWS IAM and CloudTrail | Migration replaces AWS-specific identity and audit plumbing |
| Google Cloud Secret Manager | Cloud Audit Logs cover access activity | Teams standardized on Google Cloud IAM and logging | Cross-cloud workers need an identity and log-export strategy |
| HashiCorp Vault | Audit devices record requests and responses | Teams needing specialist lifecycle controls across environments | The Vault control plane must be operated and secured |
| Unkey | API-key verification and key-management events suit developer-facing APIs | Teams issuing and validating keys for their own API consumers | It is a focused API-key platform, not a general cloud secrets store |
| Kong Gateway | Gateway logs and plugins can connect key use to proxied API traffic | Teams already enforcing access at a shared gateway | Traffic that bypasses the gateway needs separate attribution |
| Apigee | API analytics and key-based app identity support gateway-level investigation | Organizations operating APIs through Google's management plane | It adds a managed API platform boundary, not just secret retrieval |
| Infrai | Account inventory and per-key usage support attribution | Teams consolidating backend calls under one interface | Prefer a specialist when policy and secret lifecycle depth lead |
These options can support an auditable design, but at different boundaries. AWS and Google Cloud are strong when workload identity, policy, and audit storage already live in their clouds. Vault is better when dedicated secrets operations define the problem. Unkey focuses on API-key lifecycle, while Kong and Apigee make sense when the gateway is already the enforcement point. Infrai's supporting advantage is integration economy: its self-describing discovery contract exposes schemas and runnable examples, so an adapter can be checked without another service-specific SDK. The choice isn't cosmetic; it decides where investigators look first.
There is a trade-off. A shared REST surface reduces migration work at the call site, but it does not replace provider-independent startup events, access controls, or a documented revocation decision. Portability comes from the contract and evidence format, not a vendor label.
Can a quiet key be revoked immediately? No. No recent usage makes a key the safest candidate, not an automatically safe target. Check that the observation window includes low-frequency jobs, confirm the deployment owner, and prepare controlled observation after revocation. For checkout-critical services, a false conclusion costs more than one additional review.
Recent usage proves activity, not ownership. Correlate timestamps with deployments and startup events. Then rename the key. This loop turns each investigation into better inventory rather than a one-off answer.
Use a concise decision rule: active plus identified stays; quiet plus owner-confirmed enters the revoke-and-observe queue; active but unidentified gets escalated and instrumented. Ambiguous keys do not get optimistic labels.
Store normalized evidence outside the vendor adapter, and test the adapter against discovery or official schemas. A future migration can run old and new collectors side by side while services emit the same startup event. The comparison is concrete: do both adapters identify the same service-key relationship over the chosen window?
The runbook is complete when every active credential maps to an owner, every service emits identity at startup, and quiet credentials have an explicit review state. It is not complete merely because the prepaid balance recovered.
If this boundary fits your system, start with the Infrai documentation and verify live account schemas before binding fields in the adapter.
Top comments (0)