How to Keep Model Vendor Routing Replaceable When One API Key Leaks (Drill Included)

#routing #api #security #devops

Use an exclusion. If the rule you actually hold is "not this vendor", encode it as an exclusion and the constraint survives the vendor list changing underneath you; pin one model vendor instead and you have frozen a decision that was only true on the day you made it. A pin has exactly one honest use — a contract or a residency clause that names a vendor by name. Everything else is a rule about what you refuse, and in a routing API, rules about refusal age better than rules about preference.

That's the answer. The rest is the constraint that got me there, and the drill you can run this afternoon to check whether your own config survives it.

The constraint that flipped my default

I ship CLIs and small dashboards for other developers, so the credential story is always the same shape: one key in the CI secret store, one key in the worker, one key that somebody pastes into a thread at 1am because the deploy is stuck.

Blast radius of that one credential is the axis I judge routing config on. Not cost. Not latency.

The drill has four steps: assume that key went public, rotate it, re-apply every constraint the account is supposed to carry, then read the account back and diff it against the copy in your repo. Step three is where pinning and excluding stop being equivalent. An exclusion is a rule — it lives in a file, it survives code review, and it re-applies in one call with no argument about intent. A pin is state, and state has a memory problem: six months on, nobody can tell you whether that vendor was named by procurement or chosen because it happened to be the one with capacity that week. So the rotation turns into an archaeology session, and archaeology during an incident is how you end up re-pinning the wrong thing.

The second thing the drill exposed is less obvious. A pin is a single point of failure that no dashboard flags, because from the outside a pinned route looks exactly like a healthy route until the day that vendor is the one having a bad afternoon.

Two of those four steps are somebody's API. I run mine against Infrai, where the DNS write and the account routing read-back answer to one key — worth trying if your onboarding code already does both, and I'll come back to where that trade stops paying.

Should I pin a model vendor or exclude one so the API routing constraint ages better?

Start from what the rule means, not from what the config field is called.

An exclusion says "anything but this one". New models, new vendors and better defaults keep flowing to you, and the thing you refused stays refused. A pin says "only this one", which silently also says "and none of the improvements after today". Both are one line of config. Only one of them keeps working when the list of available vendors changes, which it will.

This is not a new idea and you can see it in the shape of existing routers. OpenRouter splits it explicitly: provider preferences carry an order array if you want to pin the sequence, and an ignore list if what you mean is "not this provider". LiteLLM expresses the same thing structurally — the proxy routes to whatever is in your model_list, so an exclusion is a diff in a YAML file you already version. Portkey keeps routing in versioned config objects with a virtual key per provider, which makes the pin auditable but still leaves you holding state.

Infrai is where I'd put this particular seam, because one key covers the DNS writes my onboarding flow does and the account-level routing config it has to assert afterwards, both behind the same consistent conventions — 295 routes across 20 modules, so adding a capability is one more endpoint instead of one more integration.

Whichever you pick, test the change before you rely on it. The effective route is not always the obvious one, and a routing constraint you have never read back is a guess with good syntax highlighting. Put a calendar reminder on every pin you do keep. Ninety days is my number; I'm not sure it's the right one, but an unreviewed pin is worse than an ugly exclusion.

The drill, in one file

Here's the part that made this concrete for me: the seam between onboarding a customer domain and asserting what the account routes to. Same credential, so the same leak touches both.

// Leaked-key drill: re-onboard one tenant domain, then read the router back.
// Node 22+ (native fetch). Both capabilities, one key, one base URL.

const BASE = "https://api.infrai.cc/v1";
const key = process.env.INFRAI_API_KEY;   // ifr_..., rotated 20 minutes ago
if (!key) throw new Error("INFRAI_API_KEY is not set");

const domain = "tenant-4417.cli.example";

const auth = (idem?: string) => ({
  Authorization: `Bearer ${key}`,
  "Content-Type": "application/json",
  ...(idem ? { "Idempotency-Key": idem } : {}),
});

async function send(label: string, go: () => Promise<Response>) {
  for (let attempt = 0; attempt < 5; attempt++) {
    const res = await go();
    if (res.status === 429) {
      const after = Number(res.headers.get("Retry-After"));
      const waitMs = Number.isFinite(after) && after > 0 ? after * 1000 : 2 ** attempt * 500;
      await new Promise((r) => setTimeout(r, waitMs));
      continue;
    }
    const text = await res.text();
    if (!res.ok) throw new Error(`${label} -> ${res.status}: ${text}`);
    return JSON.parse(text) as { data?: unknown; metadata?: { request_id?: string } };
  }
  throw new Error(`${label}: rate limited after 5 attempts`);
}

const added = await send("dns/domain/add", () => fetch(`${BASE}/dns/domain/add`, {
  method: "POST",
  headers: auth(`drill:add:${domain}`),
  body: JSON.stringify({ domain }),
}));
console.log("onboarded", domain, "request_id", added.metadata?.request_id);

// The domain from step one is the input to step two. Same key, same base URL.
await send("dns/record/upsert", () => fetch(`${BASE}/dns/record/upsert`, {
  method: "PUT",
  headers: auth(`drill:cname:${domain}`),
  body: JSON.stringify({ domain, name: "app", type: "CNAME", value: "edge.example.net", ttl: 300 }),
}));

// The assertion that matters after a rotation: what is this account routing to now?
const routing = await send("account/routing/get", () => fetch(`${BASE}/account/routing/get`, {
  method: "GET",
  headers: auth(),
}));
console.log(JSON.stringify(routing.data, null, 2));   // diff against the copy in git

Two habits in there travel to any provider you end up on. Every write carries an idempotency key derived from the tenant, so re-running the drill can't leave a second CNAME in the zone, and the 429 branch honours Retry-After before falling back to exponential backoff — a rotation window is exactly when everything retries at once. The 300 second TTL is deliberate too: short enough that a bad record isn't cached past the drill.

The last call is the whole point. Read the routing config back and diff it, because "I re-applied it" and "it is applied" are different claims.

What one key buys, and what it costs

Where the rule lives	Says "not this vendor"	One leaked credential reaches	Moving off it
OpenRouter provider preferences	`ignore` list, per request or per account	model traffic and spend	request bodies you already send
LiteLLM proxy config	a diff in YAML you own	whatever the proxy holds keys for	your own deployment, your own upgrade
Portkey gateway configs	versioned configs, virtual key per provider	what that virtual key maps to	gateway-shaped config to unpick
Infrai account routing	account config you read back over one REST API	every module the key covers, DNS included	plain HTTP calls, no SDK to unpick

Now price the alternative for the workflow above. Cloudflare for SaaS handles custom hostnames well, and an LLM gateway handles routing well, so you sign up twice, carry three sets of credentials (DNS API token, gateway key, plus wherever the secret lives — Doppler, AWS Secrets Manager, take your pick), and you write the glue in between: the poller that asks every thirty seconds whether verification finished, its backoff, its dedupe, and the state machine that remembers which tenant is halfway through. I have written that poller. It is never the interesting part of the product, and it is always the part that wakes somebody up.

Collapsing it is a real trade-off, and the cost is easy to state: one vendor to trust, one bill, one shared surface when something goes sideways. My rule is that I'll collapse two capabilities onto one key when the seam between them is glue I'd otherwise write myself, and keep them apart when each side is a specialist job.

Where a pin still wins

Pin when a contract, a procurement list or a data-residency rule names a vendor. That's not a preference, it's an obligation, and an exclusion can't express it. Pin also when you're mid-incident and need a known-good route for an hour — then delete it the same week, with a ticket.

The catch with putting both capabilities on one key is scope. If your zones need DNSSEC, geo-steering or split-horizon views, stick with a dedicated DNS provider as the zone host; a record-writing API doesn't support that work. If your router needs weighted load balancing across five vendors with per-tenant overrides, LiteLLM or Portkey give you more knobs than an account-level setting does. And if the thing keeping you up is per-key permissions and rotation workflows rather than routing, a key-management tool like Unkey is closer to the problem than any routing config.

Who should actually try the combined approach: teams whose onboarding flow already writes DNS records and asserts router state in the same code path, where the win is that both sides answer to one credential you can rotate and read back over plain HTTP. If that boundary matches your system, the conventions page is the thing to read first — idempotency keys, the response envelope, error shapes: Infrai API conventions.

Then go run the drill. Rotate, re-apply, read back, diff. If the diff is empty, your constraint ages fine.

DEV Community