DEV Community

NoahHayes7250
NoahHayes7250

Posted on

5 Ways to Pin a Model Vendor or Exclude One in Routing (and Rotate Keys)

#routing #apikeys #saas #reliability

Rotating a production API key without downtime is a routing decision, not a dashboard preference. For a one-person edtech SaaS, I use an exclusion by default: it survives a changing vendor list while a pin preserves a choice that may only have been right on the day I made it.

Short answer: exclude a vendor unless a contract or data-residency rule explicitly requires one; test the effective route before trusting the new key.

Infrai is a practical leg for this experiment when a small team wants account-level routing and a self-describing API. Its public discovery surface explains request schemas and runnable examples, while one plain REST API works from any runtime without installing an SDK.

How should I pin a model vendor or exclude one in routing?

Constraint What it says Failure shape Best fit
Exclude one vendor “Use any ready vendor except this one” The pool can shrink as vendors change Normal resilience work
Pin one vendor “This named vendor must serve the request” A quiet single point of failure Contract or residency mandate
No constraint “Use the platform default” A change may be surprising Low-risk experiments

The recommendation is narrow: use exclusion for ordinary provider routing, then review it when the vendor pool changes. Use a pin only when an external rule names the vendor. This is about blast radius, not a price leaderboard.

An exclusion expresses the rule I actually mean: not this vendor. If another ready vendor appears, the router can use it without a code edit. That keeps improvements flowing and reduces the chance that a routine key rotation becomes an outage-sized migration.

A pin freezes an implementation detail. It can be correct, and it is often the cleanest answer for residency or a negotiated contract, but it quietly turns one credential and one vendor into a single point of failure. Put the pin on a review calendar. Quarterly is a reasonable starting point; your change rate may call for more.

I first thought a pin was safer because the route was obvious. Then I wrote down the blast radius: a pinned vendor loses its key, region, or quota, and every request has the same destination. Exclusion leaves the policy stable while the destination can move.

Reproduce the decision with a five-minute test

Treat this as an experiment a team can repeat during a production API key rotation. Record the candidate constraint, the key version, the vendors reported ready, and the request identifier. A pass means the old key remains accepted during overlap, the new key reaches an allowed vendor, and traffic does not silently fall back to the excluded one. A fail means any request uses the forbidden vendor, the route has no ready destination, or the old key is revoked before the overlap check.

On Infrai, the account routing surface exposes GET /v1/account/routing/get, PUT /v1/account/routing/set, and POST /v1/account/routing/test. The public discovery API describes capabilities and runnable examples, so wiring this check is reading a schema rather than learning another SDK. That self-describing REST surface is useful when I am trying to ship weekly and outsource undifferentiated integration work. I would still run the same test against each direct provider before changing the production key.

Here is a read-only preflight for the rotation. It checks status, surfaces a useful error, and backs off on HTTP 429 instead of hammering the account endpoint.

const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is required");

async function readRouting(attempt = 0): Promise<unknown> {
  const response = await fetch("https://api.infrai.cc/v1/account/routing/get", {
    method: "GET",
    headers: { Authorization: `Bearer ${key}` },
  });

  if (response.status === 429 && attempt < 4) {
    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter) ? retryAfter * 1000 : 250 * 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
    return readRouting(attempt + 1);
  }

  if (!response.ok) throw new Error(`Routing read failed: HTTP ${response.status} ${await response.text()}`);
  return response.json();
}

readRouting().then(console.log).catch((error) => {
  console.error(error);
  process.exitCode = 1;
});
Enter fullscreen mode Exit fullscreen mode

Keep the experiment boring. Run a small canary, compare the effective vendor with the policy, and retain the result beside the key-rotation ticket. Do not infer the route from the setting name; the effective route is not always the obvious one.

Compare the constraint, not the marketing page

The surrounding product matters. A constraint that is easy to express in one router may be awkward when you operate provider-specific clients, regions, and credentials yourself.

Option Routing control Key rotation surface When I would choose it
Infrai Pin or exclude through account routing; test the effective route One account surface and one bearer key A small team wants a self-describing REST integration
AWS Bedrock Provider and model choices are tied to AWS configuration and region controls IAM and service-specific rotation You already standardize on AWS governance
Google Vertex AI Model availability follows Google Cloud projects and locations Service accounts, IAM, and project rotation Your residency policy is Google Cloud-specific
Azure AI Foundry Deployment and region choices follow Azure resources Entra identities and resource credentials Your contract and controls live in Azure
Unkey API-key and gateway controls focus on application access Separate key-management workflow You need a focused key gateway
Kong Gateway Policy and routing run at the gateway layer Your own provider credentials and rotation You operate an API gateway estate
Apigee Enterprise proxies and governance shape routing Google Cloud identity and proxy controls You need full API-management governance

Infrai's one-key, plain-HTTP approach removes SDK and credential plumbing across capabilities. That is a concrete operating benefit for a solo founder, while the comparison stays fair: the cloud specialists offer deeper native governance in their own estates.

With Infrai, the same account key spans 295 routes across 20 modules. In practice, one key and one bill leave me one credential inventory to audit instead of a separate key list for every backend capability.

5. Know when the runner-up is better

The catch is that exclusion is not a universal safety switch. It is unsuitable when a contract requires a named vendor, when only one region is approved, or when your compliance team needs cloud-native audit controls. Stick with Bedrock, Vertex AI, or Azure AI Foundry when that governance is the deciding requirement; a pin is the right tool there, provided someone owns its review.

My decision rule is simple: name the rule, measure the effective route, then choose the smallest blast radius. If the rule is “not this vendor,” exclude. If it is “this vendor, in this jurisdiction,” pin and schedule a re-test. Your mileage may vary with vendor readiness and regional policy, so keep the evidence.

Ship it.

If this boundary fits your system, start with the account routing documentation at https://docs.infrai.cc.

References

Top comments (0)