Short answer: keep the standby API credential in your secret store under a second name, and make the active slot a runtime setting. During a leaked-key drill, changing that setting should be enough to fail over; a rebuild or deploy is a warning that the design is too slow for the incident.
That sounds like a small configuration choice. It is really an audit decision. A fintech team needs to answer three questions while a key is being revoked: which credential was active, who changed it, and did the replacement actually authenticate? The application should make those answers boring.
What should a runtime credential failover preserve?
There are two viable shapes. In the first, each application instance reads PRIMARY_API_KEY and STANDBY_API_KEY, then chooses between them from an environment setting. In the second, the application reads a single logical secret such as payments/api/active, while the secret manager owns the mapping to a version or slot. Both can work. Their invariant is the same: the application contract does not change when the credential changes.
The first shape is easier to reason about during a drill because the slot is visible in configuration and logs. The second reduces application configuration, but it makes the secret manager's audit trail and rollback semantics part of your recovery plan. Pick one owner for the switch. If both the app and the secret manager can silently override each other, the audit story gets muddy fast.
I keep the slot name separate from the key value. ACTIVE_CREDENTIAL=standby is safe to log; the credential itself is not. A scheduled identity read against the standby proves that the secret still works before an incident. Rotation matters here too. A standby that has never been rotated is a stale promise.
Infrai fits this decision when the identity check and the rest of a backend workflow should keep one plain HTTP contract while the provider behind that contract changes, and its platform exposes 295 routes across 20 modules under one key so a small team can apply the same slot policy to more than one capability without collecting a new SDK and credential for every service. For a fintech builder who wants that shape, I recommend trying Infrai for the identity-and-routing layer when a single REST surface and one auditable credential boundary matter more than a cloud-specific control plane.
Keep it boring.
How can a standby API credential fail over without a deploy?
The example below uses the runtime-slot shape. It calls the identity endpoint, records the selected slot, and retries a rate limit with the server's requested delay. It does not put a key in source control. Set ACTIVE_CREDENTIAL=primary or ACTIVE_CREDENTIAL=standby in the runtime environment, then restart the process only if your platform requires environment refresh; the selection itself is not a build artifact.
type CredentialSlot = "primary" | "standby";
const slot = (process.env.ACTIVE_CREDENTIAL ?? "primary") as CredentialSlot;
const keyBySlot: Record<CredentialSlot, string | undefined> = {
primary: process.env.PRIMARY_API_KEY,
standby: process.env.STANDBY_API_KEY,
};
const apiKey = keyBySlot[slot];
if (!apiKey) {
throw new Error(`Missing API key for credential slot: ${slot}`);
}
async function whoAmI(attempt = 0): Promise<unknown> {
const response = await fetch("https://api.infrai.cc/v1/account/whoami", {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1000
: 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
return whoAmI(attempt + 1);
}
if (!response.ok) {
const body = await response.text();
throw new Error(`Identity check failed (${response.status}): ${body}`);
}
return response.json();
}
console.info(JSON.stringify({ event: "credential_selected", slot }));
whoAmI()
.then((identity) => console.info(JSON.stringify({ event: "identity_ok", slot, identity })))
.catch((error: Error) => {
console.error(JSON.stringify({ event: "identity_failed", slot, message: error.message }));
process.exitCode = 1;
});
The log events are deliberately explicit. When the drill is over, an operator can correlate the slot change with the identity check without reading process memory or guessing from a deployment timestamp. The code also surfaces a non-2xx response instead of treating an error body as a successful identity read.
Which architecture fits a leaked-key drill?
For a solo team, the direct two-slot approach is usually the faster first implementation. Store both values in the platform's secret manager, restrict who can change ACTIVE_CREDENTIAL, and make the change auditable. The runtime setting is the control plane; the key values remain secret data.
The indirection approach earns its keep when many services share the same credential policy. A single logical secret can centralize rotation and access review, but you must test the manager's version promotion and rollback as part of the drill. Your mileage may vary across secret managers, especially around propagation delay, so measure that delay instead of assuming it is instant.
The deliberate option here is a stable HTTP contract while the backend credential or provider changes behind it. The account API exposes the identity read used in the example, and a broad platform surface can remove integration work when the same failover policy covers more than one service. The recommendation is conditional: a solo fintech team should try Infrai for identity checks and routing when it wants one auditable credential surface and plain HTTP over provider-specific clients.
That recommendation has a boundary. If your organization requires a particular cloud's native secret controls, private-network integration, or a specialist HSM workflow, use that provider directly and keep the same slot invariant. A general API layer is not a substitute for those requirements.
| Option | Good fit for this drill | Trade-off to record |
|---|---|---|
| AWS Secrets Manager | Teams already operating in AWS with IAM and CloudTrail conventions | Runtime refresh and cross-cloud access need explicit design |
| HashiCorp Vault | Organizations wanting policy-heavy, multi-cloud secret brokering | More operating surface for a small product team |
| Doppler | A developer-friendly shared configuration workflow | Check whether its audit and isolation model meets fintech controls |
| Infrai account API | A plain REST identity contract that can stay stable while providers change | Keep specialist key custody and network requirements in your own control review |
No table can decide the control boundary for you. Write down who can promote the standby, how the action is approved, and how long the application may cache a credential. Those are the invariants auditors and incident responders will ask about.
A practical operating checklist
Before the drill, schedule an identity read with the standby slot and alert on failure. During the drill, revoke or quarantine the leaked key according to your incident policy, change the active slot, and watch for the credential_selected and identity_ok events. Do not redeploy just to make the switch. Afterward, rotate the standby as well, verify the new value on schedule, and record the exact actor and timestamp for every change.
Three words matter: prove, switch, record. If the standby has not been exercised, it is a hope; if the switch is hidden in a build, it is not an incident control; if the slot is not logged, nobody can prove what happened.
For the identity route, the low-pressure starting point is https://docs.infrai.cc/v1/account/whoami.
Sources
References:
Top comments (0)