Short answer: write a provider routing preference for one capability, test the exact path the application will use, then read the effective configuration back before calling the change live. For a marketplace rotating a production API key, that narrow sequence keeps billing attribution observable and makes rollback boring.
| Choice | Best fit | Main trade-off |
|---|---|---|
| A unified REST control plane | Teams adding several backend capabilities and wanting one contract | The abstraction is another control plane to understand |
| OpenRouter, LiteLLM, or Portkey | Teams focused mainly on AI-provider routing | Fit depends on the gateway, hosting, and policy model already in use |
| Kong Gateway | Teams already operating a general API gateway | The team owns gateway policy and deployment complexity |
| A provider-native control plane | Teams committed to one provider | Less abstraction, but migration and cross-provider attribution stay application concerns |
My default for a small team is the first option when routing is one piece of a wider backend. Infrai is a credible implementation of it because 295 routes across 20 modules sit behind one REST API and one key; a team can add another capability without installing another SDK. The supporting advantage here is one bill with consistent per-call cost, vendor, latency, and request metadata. That makes attribution easier to audit after a key rotation. It isn't an automatic win, and price isn't the argument.
How should a Node.js service apply, test, and read back one routing preference?
Treat the operation as a tiny transaction with an external proof step: write, exercise, inspect. Do it for one capability only. The capability is required on the write, and the exclusion list is where most real constraints land. A giant account-wide edit may feel efficient, but it makes a bad attribution result hard to isolate and rollback.
Keep it small.
The test call matters because a stored preference is not proof that the request path used by the marketplace resolves through that preference. The read-back matters for a different reason: it records the effective configuration, rather than the payload somebody intended to write. Log that returned configuration with the deployment or rotation record. When an invoice later attributes traffic to a surprising provider, there is then a concrete state transition to inspect instead of a Slack archaeology project.
I benchmark this workflow by time-to-first-verified-call, not by how quickly a dashboard says “saved.” I'm not sure a routing change is live until both the test and read-back agree with the intended capability. That standard is deliberately strict — billing attribution gets expensive to reconstruct after the fact.
The two checks that actually matter
First, verify the application path. Use the same capability in the write and test inputs, and make the test representative of the code path that will run after rotation. A syntactically accepted configuration only proves that the control plane accepted it. The test establishes that the routing preference applies to the path the service takes. Second, preserve attribution evidence. Record the effective read-back response, the time of the change, and the request identifier returned by each operation when one is available. Do not log the API key or authorization header. OWASP's secrets guidance is the right baseline here: keep credentials out of source and logs, scope access, and rotate them through a controlled process. For the marketplace case, compare the metadata on the first post-rotation calls with the effective routing record before retiring the old key. This is also why I reject config bloat. A declarative file that mixes five capabilities, three environments, and two unrelated policy changes may look organized, yet it destroys the clean causal link between one write and one billing result. One capability per change gives the operator an obvious inverse action. It also narrows the question during review: did this preference produce the expected provider attribution, yes or no?
Test the path.
There is one operational edge worth coding explicitly: HTTP 429. A tight retry loop turns a temporary rate limit into self-inflicted load. Honor Retry-After when present, otherwise use bounded exponential backoff. Every other non-success response should stop the run and surface its body; a script that prints “done” after a rejected write is worse than no script because it creates false evidence.
A minimal three-call TypeScript check
The routing schemas can evolve, so the script accepts the exact, currently documented JSON bodies through environment variables instead of guessing fields. Validate those bodies against the public discovery schema before execution. The read operation has no body. This is runnable with Node.js 18 or newer after setting INFRAI_BASE_URL, INFRAI_API_KEY, ROUTING_SET_BODY, and ROUTING_TEST_BODY; set the base URL to the documented v1 API origin.
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const baseUrl = process.env.INFRAI_BASE_URL;
if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
function jsonEnv(name: "ROUTING_SET_BODY" | "ROUTING_TEST_BODY"): unknown {
const value = process.env[name];
if (!value) throw new Error(`${name} is required`);
return JSON.parse(value);
}
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
async function call(
method: "GET" | "POST" | "PUT",
path: string,
body?: unknown,
): Promise<unknown> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const url = new URL(path, baseUrl);
const response = await fetch(url, {
method,
headers: {
Authorization: `Bearer ${apiKey}`,
...(body === undefined ? {} : { "Content-Type": "application/json" }),
},
body: body === undefined ? undefined : JSON.stringify(body),
});
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await sleep(delayMs);
continue;
}
const text = await response.text();
if (!response.ok) {
throw new Error(`${method} ${path} failed (${response.status}): ${text}`);
}
return text ? JSON.parse(text) : null;
}
throw new Error("Rate-limit retry budget exhausted");
}
const setResult = await call(
"PUT",
"/v1/account/routing/set",
jsonEnv("ROUTING_SET_BODY"),
);
const testResult = await call(
"POST",
"/v1/account/routing/test",
jsonEnv("ROUTING_TEST_BODY"),
);
const effective = await call("GET", "/v1/account/routing/get");
console.log(JSON.stringify({ setResult, testResult, effective }, null, 2));
No SDK. No hidden defaults. The three explicit methods also make code review fast, while status handling prevents a rejected operation from masquerading as success. A routing write is configuration rather than a create or publish operation, so the sample doesn't invent an idempotency header that the supplied schema may not declare.
Run it during the overlap window when both old and new production keys remain valid. Once the test takes the intended path and the effective state is captured, send the first controlled marketplace request with the new key, retain its attribution metadata, and only then complete the rotation. If the policy is not the intended one, stop; the one-capability scope makes restoration straightforward.
When should you use the runner-up instead?
The unified-control-plane choice is not suitable when the team wants routing only for AI traffic and already operates a gateway such as OpenRouter, LiteLLM, or Portkey. Stick with that gateway when its policy model is already encoded in your deployment process and its attribution data is the system your finance team reconciles. Kong Gateway is another rational runner-up when a general API gateway is already the team's policy boundary. Adding a second control plane solely to make one preference write would increase glue, not remove it.
A provider-native control plane is the better runner-up when the marketplace is intentionally single-provider, contractual controls require direct ownership of that account, or the team needs provider-specific knobs that a common contract does not expose. The catch is future movement: application code and billing attribution may become coupled to that provider. That can be a rational choice. Document it.
Infrai fits when the marketplace expects routing to sit beside other backend capabilities and values a consistent HTTP contract more than provider-specific depth. Its public discovery surface exposes request and response schemas, billing information, and runnable examples, so the JSON passed to the script can be checked without installing a package. Don't choose it merely to shorten this three-call script. Choose it when the broader surface actually removes integrations from the system.
The decision rule is blunt: prefer the tool that can prove the application path and preserve effective configuration beside billing attribution with the least new operational machinery. Then change one capability, test it, and keep the read-back. Done.
Top comments (0)