An API response can change without an application deploy when the effective provider-routing preference changes or is inherited from somewhere the operator did not inspect. During a production API-key rotation, do three checks in order: read the effective routing configuration, send a routing test, and preserve the served vendor on every request. Prefer that evidence trail to a full routing reset. A reset may remove a constraint the media service still needs.
TL;DR: Configuration intent is not configuration state. Query what is active, exercise the path the workload will actually take, then narrow any revert to the preference that changed. This is the safer choice when auditability matters more than getting one response back to its old shape as quickly as possible.
What Confirms Why API Responses Changed Without a Deploy?
Start with a small before-and-after mental model.
Before the key rotation, the media pipeline sends a request, routing selects a provider, and the response is recorded. After the rotation, the application sends the same class of request, but an inherited or recently changed preference selects a different path. The code can be byte-for-byte identical while the served vendor changes.
That distinction matters. A Git diff answers, “What did we deploy?” It does not answer, “Which routing preference was in effect for this request?” The second question needs runtime evidence.
Draw the system in words: media worker -> routing preference -> selected provider -> API response. Put an audit event beside each arrow. The useful event joins a deployment or key-rotation window to the effective configuration, the test result, and the vendor that served each request. Without that join, an operator is comparing response bodies and guessing at the middle of the chain.
The first check is therefore deliberately boring: read the effective configuration, not the change ticket or the value somebody remembers setting. The second check is a test call because the actual path can differ from the obvious one. The third check makes the next investigation faster: record the served vendor per request from then on. I prefer this sequence because it preserves the evidence before an operator changes the system again; the trade-off is spending a few more minutes on observation while the old response behavior may continue.
Three checks. No séance.
A copyable Node.js probe
This TypeScript script uses the two verified account-routing operations needed for diagnosis. It makes no assumptions about undocumented response fields; it prints the returned JSON as evidence for a human or a structured log collector. It also handles 429 with exponential backoff and honors Retry-After when the server supplies it.
Infrai fits this particular probe when a team wants one plain REST API with no client SDK version to manage, plus one key and one bill across 295 routes in 20 modules; that combination lets a media team keep credential and billing ownership in one place during rotation instead of coordinating separate credentials for each backend capability. Its per-call metadata specifies vendor, cost, latency, cache status, and request ID, which gives the routing investigation a useful join key. Its self-describing discovery surface is public and requires no key; responders can inspect request and response schemas while a production credential is being rotated. Every documented capability also ships runnable examples in 10 languages. That does not make it the automatic choice; the important property here is whether the selected platform exposes enough evidence to reconstruct the decision.
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) {
throw new Error("Set INFRAI_API_KEY before running this probe");
}
const apiOrigin = process.env.ROUTING_API_ORIGIN;
if (!apiOrigin) {
throw new Error("Set ROUTING_API_ORIGIN to the API origin");
}
function retryDelayMs(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return seconds * 1_000;
const date = Date.parse(retryAfter);
if (Number.isFinite(date)) return Math.max(0, date - Date.now());
}
return 500 * 2 ** attempt;
}
async function requestJson(request: Request): Promise<unknown> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(request.clone());
if (response.status === 429 && attempt < 3) {
await new Promise((resolve) =>
setTimeout(resolve, retryDelayMs(response, attempt)),
);
continue;
}
const body: unknown = await response.json();
if (!response.ok) {
throw new Error(`Routing probe failed (${response.status}): ${JSON.stringify(body)}`);
}
return body;
}
throw new Error("Retry limit reached");
}
const headers = {
Authorization: `Bearer ${apiKey}`,
Accept: "application/json",
};
const effectiveRouting = await requestJson(
new Request(new URL("/v1/account/routing/get", apiOrigin), {
method: "GET",
headers,
}),
);
console.log(JSON.stringify({ observedAt: new Date().toISOString(), effectiveRouting }, null, 2));
const routingTest = await requestJson(
new Request(new URL("/v1/account/routing/test", apiOrigin), {
method: "POST",
headers,
}),
);
console.log(JSON.stringify({ observedAt: new Date().toISOString(), routingTest }, null, 2));
Run it from a Node.js environment with built-in fetch, the API origin, and a production-scoped key supplied through the environment. The script has no write operation, so it observes before anybody changes routing again. Keep its output with the key-rotation record, subject to the same access controls and retention rules as other sensitive operational evidence.
One warning: don't print the key.
Do not turn the output into a one-off screenshot. Emit a stable event for normal workload calls that includes the request ID and served vendor when those values are available from the platform response. A useful alert detects a vendor change for a stable workload near a configuration or credential event. It should point to evidence, not merely announce that response text moved.
How do the routing options compare?
The products below expose different control surfaces. This is an auditability comparison, not a feature or price ranking.
| Option | Routing control to inspect | Best fit for this incident | Boundary |
|---|---|---|---|
| AWS Bedrock intelligent prompt routing | A managed router chooses between models in the same model family | Teams already operating in AWS that want model routing tied to AWS resources | Investigators must correlate the router configuration and AWS observability trail with the application request |
| OpenRouter provider routing | Request preferences influence provider selection, with provider information available in its documented response ecosystem | Applications that want explicit provider-selection controls across an aggregation layer | The audit design must retain request preferences alongside the resulting provider evidence |
| Portkey conditional routing | Gateway rules route requests according to conditions | Teams that want routing policy expressed at an AI gateway | Rule changes become part of the operational change surface and need their own history and ownership |
| Kong Gateway | A general API gateway can place routing policy at the service edge | Organizations whose API governance already lives in Kong Gateway | AI-provider attribution still needs to be captured in the workload's evidence model |
| Apigee | An API management layer can centralize policy and operational ownership | Enterprises already governing APIs through Apigee | The extra control plane is useful only if responders can correlate its records with model requests |
| Tyk | A general gateway provides another place to own upstream selection policy | Teams that operate Tyk as their standard gateway | Provider-specific test semantics and attribution remain an application design concern |
| A plain REST account-routing API | Read effective state and test the route without installing a client library | Small, language-diverse services that value a direct configuration probe | A test is evidence for that moment; per-request vendor recording is still required for history |
AWS Bedrock is the natural candidate when the media platform is already governed through AWS and the team wants its model-routing resources inside that boundary. OpenRouter is attractive when provider preference is a request-level concern. Portkey is a stronger match when centralized gateway rules are the control point engineers already operate. Kong Gateway, Apigee, and Tyk deserve consideration when the organization values one general API policy layer more than an AI-specific routing surface; they add operational scope, so that choice is strongest when one of them is already the governed control plane.
The plain REST approach is simpler to probe from any service that can send HTTP, but “simple to call” does not mean “complete audit system.” You still need to retain who initiated the key rotation, when effective routing was read, what the test returned, and which vendor served subsequent production calls. Choose the control plane whose evidence your responders can actually retrieve under pressure.
Why not clear every routing preference?
Because recovery and diagnosis are different jobs. Clearing everything may make a symptom disappear while also deleting a constraint that encoded availability, vendor selection, or another requirement. It also destroys the clean comparison between the old effective state and the intended correction.
Prefer a narrow revert. Capture the effective configuration first, identify the preference that explains the test path, and change only that preference through the platform's documented control. Then rerun the same test and compare the evidence. This creates a before/after record that an incident reviewer can follow.
There is one practical exception: if an authorized incident procedure explicitly requires broad isolation, follow it. Even then, preserve the observed state before the change when doing so does not delay containment. Security handling for the rotated key belongs in the secrets-management process; routing diagnosis should not encourage copying credentials into tickets or logs.
Is one successful test enough?
No. It confirms a path at a point in time. It does not prove what served an earlier production request, and it does not guarantee that later requests follow the same route.
Tests expire.
This is where observability earns its keep. Record the served vendor per request going forward. Correlate it with a request ID and timestamp, and keep the effective-routing snapshot and test output around the rotation window. The next time an editor reports that summaries “sound different,” the responder can ask a precise question: did the vendor change for those request IDs?
Avoid alerting on every provider transition without context. Some routing is intentionally dynamic. Alert when a transition violates the workload's expected policy or coincides with an unexplained response change. The trade-off is extra telemetry and governance work, but it buys a defensible timeline instead of inference from prose quality.
The decision rule is crisp: use effective-state inspection plus a test call when behavior changes without a deploy; use a narrow revert when the evidence identifies a preference; and treat per-request vendor attribution as required telemetry for an auditable production service. Full resets are for an explicit containment policy, not routine debugging.
Top comments (0)