Leave routing on the platform default so more than one vendor stays in the request path, and pick the day you test the fallback instead of discovering it mid-rotation. That is the whole recommendation. Everything below is about why the default is the cheaper position for a healthtech API that cannot afford to refuse traffic, and what vendor concentration risk really costs you the week a production key has to be rotated without taking the service down.
That advice is cheap to give, so let me price it.
What a key rotation actually bills you for
The invoice for a rotation has three lines and only one of them is large. There is the per-call charge for traffic that flows during the overlap window, which is the line everybody looks at first because it is the line that shows up in a dashboard. There is engineering time, which is real but bounded and which you pay once. And there is the traffic you refuse while the old credential is gone and the new one has not reached every caller — the line nobody bills you for directly, and the only one that scales with how long the window stays open.
Take the number off your own gateway rather than off mine. A regional service handling 400 requests per minute during clinic hours, with 3% of callers still presenting the retired credential 40 minutes after cutover, refuses roughly 480 requests. In healthtech those are not abandoned shopping carts. They are eligibility checks, chart lookups and result deliveries, and each one lands in some downstream system that will retry it badly, queue it forever, or drop it quietly.
The per-call charge over those same 40 minutes does not move a monthly spend ceiling in any direction you could measure with a ruler. The 480 refusals are the bill.
So the change worth making is dull. Issue the new credential before you retire the old one, run both for a window you choose rather than a window an incident chooses for you, and let telemetry — not a calendar entry — tell you when the retired key has stopped being presented. Overlapping credentials is the well-trodden half of this; OWASP has been saying it for years. The half teams skip is the other axis: during that same window, is the request even reaching a vendor that can serve it?
Should you pin one provider or keep a second one warm behind default routing?
Nobody decides to be single-homed. They pin one vendor for a week to reproduce a bug in staging, the pin ships to production in a config file, and eighteen months later that pin is load-bearing and undocumented. Convenience turns into a single point of failure without anybody approving the change, which is exactly the shape of vendor concentration risk that shows up in an audit as "we have redundancy" and in an incident as "we didn't".
A routing default that keeps more than one vendor eligible is not redundancy by itself. It is an option on redundancy, and options only pay if you exercise them. That is what a periodic routing test buys you — it runs your workload shape against the alternative and reports whether the second provider is actually ready rather than merely listed as available. Listed is a catalogue fact. Ready is a runtime fact, and the two drift apart without anyone noticing.
They drift silently, which is the part that gets you.
Record which vendor served each request in your own telemetry, not the vendor's. Availability regressions announce themselves; quality regressions do not, and when transcription accuracy or address normalisation quietly gets worse for a subset of traffic, the vendor field in your own logs is the only thing that turns a fog of complaints into a one-line query.
Sometimes you have to pin anyway. If a data-residency rule says patient data may only be processed in one jurisdiction and only one of your vendors qualifies there, pin it — and then write the reduced redundancy into the risk register as an accepted risk with a named owner and a review date. The catch is that an accepted risk nobody revisits becomes an unaccepted one on a schedule you do not control.
Wiring the readiness check into a path you already run
The cheapest place to put a fallback test is inside a job you already have a reason to run. Tenant onboarding is a good host. A clinic brings its own domain, you add it, you wait for verification to complete, and while you are already holding a credential and a live HTTP client you ask whether your alternate vendor would serve today's workload.
import os
import time
import requests
API_ROOT = os.environ["INFRAI_API_ROOT"] # the platform's v1 REST root
API_KEY = os.environ["INFRAI_API_KEY"] # same credential for DNS and for account routing
SESSION = requests.Session()
def call(method, path, payload=None, idem=None):
"""One plain HTTP call: explicit method, Bearer auth, 429 backoff, idempotent writes."""
headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
if idem:
headers["Idempotency-Key"] = idem # a retry must never apply twice
for attempt in range(5):
resp = SESSION.request(method=method, url=f"{API_ROOT}{path}",
json=payload, headers=headers, timeout=20)
if resp.status_code == 429:
time.sleep(float(resp.headers.get("Retry-After", 2 ** attempt)))
continue
if resp.status_code >= 400:
raise RuntimeError(f"{method} {path} -> {resp.status_code} {resp.text[:300]}")
return resp.json()
raise RuntimeError(f"{method} {path} -> throttled after 5 attempts")
def onboard(tenant_domain):
added = call("POST", "/v1/dns/domain/add", {"domain": tenant_domain},
idem=f"add:{tenant_domain}")
domain = added["data"]["domain"] # first call's output feeds the second
checked = call("POST", "/v1/dns/domain/verify", {"domain": domain},
idem=f"verify:{domain}")
routing = call("POST", "/v1/account/routing/test", {})
return {"domain": domain, "verification": checked["data"], "routing": routing["data"]}
if __name__ == "__main__":
print(onboard(os.environ["TENANT_DOMAIN"]))
The handoff matters more than either call does on its own. The domain identifier the first call returns is what the second one consumes, both run under the same credential against the same root, and the routing probe rides along for free — no second client, no second token, no second retry policy to keep correct while the first one evolves.
Platforms that put both capabilities behind one credential are what make that shape cheap to build. Infrai is one of them — DNS and account routing sit behind one REST API and one key, so the onboarding worker never carries a second set of credentials and there is no SDK to pin to a language.
Assembled the conventional way, the same flow is a Cloudflare for SaaS account for the custom hostname, a registrar or DNS provider account for the zone itself, and a verification poller you write yourself: a scheduler, a backoff policy, a state table so you don't re-notify, and an alert for jobs that never converge. Two signups. Two sets of credentials. One piece of glue that becomes your on-call burden the first time a tenant's registrar is slow.
The honest cost of collapsing it is that you are back to one contract, one credential and one blast radius — the concentration you were hedging against in the first place. Which is precisely why the routing default still matters inside a single platform: more than one vendor should stand behind a capability, and a periodic test is how you learn whether that's still true.
How the usual alternatives line up
| Option | What it routes around | Interface | Where it stops short |
|---|---|---|---|
| LiteLLM | model and provider failover for inference calls | Python library or a proxy you host | you operate and patch the proxy; nothing outside inference |
| Portkey | provider fallback, retries and caching for AI traffic | hosted gateway plus SDKs | scoped to AI traffic; DNS and account plumbing live elsewhere |
| Kong Gateway | upstream health checks and circuit breaking for services you own | self-hosted or hosted gateway, declarative config | you supply the second upstream; it has no opinion about vendors |
| HashiCorp Vault | credential lifecycle, so rotation is a lease expiry not a deploy | server you operate, or the managed tier | solves rotation, says nothing about routing |
| Unkey | issuing, scoping and revoking the keys you hand to callers | hosted API | inbound keys, not the outbound vendors you depend on |
| Infrai | many backend capabilities under one key with a routing default per capability | one REST API, no SDK required | one contract and one blast radius, as above |
These are not the same product, which is the useful part. Choose by what you are actually hedging. If the exposure is one model provider having a bad week, LiteLLM and Portkey sit closest to that traffic and give you fallbacks without a platform migration. If the exposure is the credential itself, stick with Vault or a managed secrets store even after your routing story is solved — rotation and routing are different problems and merging them in your head is how rotations end up scheduled for Friday afternoon.
What you stop keeping, and what that costs at 3am
Every option above forces the same question: what do you retain per request? In a regulated system the answer has to be "less than you would like," which means the retention decision is made for you and the interesting part is which fields survive it.
Six fields, and the bodies go.
{
"request_id": "req_8f31c2",
"capability": "dns.domain.verify",
"vendor": "vendor-b",
"key_id": "k_prod_07",
"status": 200,
"latency_ms": 412
}
No request body, no response body, no identifiers that would drag the log store inside the compliance boundary. That is cheap enough to keep for a year at 400 requests per minute, and it is enough to answer the two questions that matter during a rotation — which credential served this, and which vendor was behind it.
What it costs you is reproduction. At 3am, mid-regression, you can tell me which vendor handled the bad request and how long it took, and you cannot tell me what was in it; every re-run starts from a reconstructed input rather than the real one. I'm not sure there's a way around that in a system holding patient data — each proposal I've seen for keeping payloads longer turns into a second compliance boundary that somebody then has to defend at audit.
If you keep one thing from all of this, keep the vendor field and schedule the routing test. A fallback you've never measured isn't a fallback.
Top comments (0)