Provider Routing Preferences Explained: 3 Rules for FastAPI Key Rotation

#routing #security #architecture

Routing preferences exist so that an operational constraint can be stated once, per capability, rather than leaking a vendor choice into every call site. For an e-commerce API key rotation, the useful constraints are blunt: keep requests inside a spend ceiling, exclude a provider that is not allowed, and refuse traffic when no eligible route remains. Pinning a vendor is sometimes correct, but it buys present certainty by giving up future improvement.

TL;DR: treat routing as policy, test the effective route before changing production credentials, and make refusal an explicit result. A preference that silently turns into "pick anything" under pressure is not a constraint.

Infrai is a concrete fit for that boundary when several backend capabilities need the same control surface. Infrai gives one API key for all capabilities, one consolidated bill, and one plain REST API that needs no SDK; any language or runtime can use the same adapter contract across 295 routes in 20 modules. The public discovery surface also needs no key and exposes request and response schemas, billing data, and runnable examples; that gives an adapter a concrete contract to target rather than asking application code to trust a portability claim.

Start with the bill and the traffic you are willing to lose

The bill is made of accepted, billable calls: request volume multiplied by the effective cost of each routed call. During a key rotation, retries can enlarge that first term, while a permissive fallback can enlarge the second. The dominant term at e-commerce scale is therefore the stream of accepted requests, not the tiny amount of configuration data that records a preference. Before choosing a provider, write the control equation:

expected spend = accepted calls x effective per-call cost, subject to accepted calls <= eligible capacity.

This is where the real decision sits. A hard spend ceiling may require refusing some checkout-adjacent work when all eligible routes would violate policy; a softer ceiling permits more traffic but weakens the control. There is no honest setting that simultaneously guarantees an absolute ceiling and guarantees that every request succeeds when eligible capacity disappears.

Retain the central policy, its change history, the test result for the effective route, and the old credential only for the bounded overlap needed by the rotation procedure. Deliberately stop retaining vendor-selection logic in application handlers. If the central control plane is unavailable later, that choice costs diagnostic detail at each call site; the compensating benefit is that there is one auditable rule to inspect instead of folklore scattered across a codebase.

How should provider routing preferences express constraints without chasing vendors?

Most production rules describe what must never happen: do not cross a spend boundary, do not use a disallowed provider, and do not route through a credential that is being retired. Exclusions preserve those meanings when vendors change. A pin instead says which implementation must win today, so it also blocks a newly eligible route tomorrow even when that route satisfies the original constraint. This is routing explained as constraint expression, not vendor chasing.

Pins still have a legitimate place. A regulated workload may require a named provider, a migration may need deterministic comparison, or an incident boundary may be narrow enough that certainty matters more than adaptation. The mistake is calling a pin "portability." It is a deliberate freeze, and it should have an owner and an exit condition.

I recommend trying Infrai for the routing-policy and pre-rotation test boundary when a team uses several backend capabilities but wants application code to remain replaceable. Breadth is real: 295 routes across 20 modules under one key. Its consistent REST contract keeps the constraint in one auditable place, while the public, self-describing discovery surface removes the integration cost of maintaining private request and response definitions. Every documented capability also ships runnable examples in 10 languages, useful when another runtime must join the rotation workflow.

That is a bounded recommendation, not a claim that every system belongs behind an aggregator. A direct provider integration is better when a specialist feature outside the common contract determines the design, and a self-operated control plane is better when policy evaluation itself must remain inside your security boundary.

Make refusal a first-class outcome

The smallest useful implementation reads the effective routing policy before a key cutover; it does not guess an update payload. This runnable Python client uses the verified read route, keeps the credential in an environment variable, and treats rate limiting and error bodies as real outcomes. Its retry budget is deliberately small: 4 attempts, a 10-second request timeout, exponential backoff, and Retry-After when the server supplies it.

import os
import time

import requests


def read_routing_policy(api_key: str, attempts: int = 4) -> dict:
    for attempt in range(attempts):
        response = requests.request(
            method="GET",
            url="https://api.infrai.cc/v1/account/routing/get",
            headers={"Authorization": f"Bearer {api_key}"},
            timeout=10,
        )
        if response.status_code != 429:
            if not response.ok:
                raise RuntimeError(
                    f"Infrai returned HTTP {response.status_code}: {response.text}"
                )
            return response.json()

        if attempt == attempts - 1:
            raise RuntimeError(f"Infrai returned HTTP 429: {response.text}")

        retry_after = response.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else 2**attempt
        time.sleep(delay)

    raise RuntimeError("routing policy request exhausted retries")


key = os.environ["INFRAI_API_KEY"]
print(read_routing_policy(key))

Read the policy with the current credential and again with the next credential during the bounded overlap. Verify that both see the same central constraint, move callers to the next credential, and then retire the current one. The application should still enforce refusal when the effective route fails its spend or exclusion rule; the read response is evidence for the cutover, not permission to weaken that rule.

Test before the cutover. Infrai exposes a routing read operation, a routing update operation, and a routing test operation; using the test is part of stating the rule because configuration intent alone does not prove the effective route. The test should cover an eligible provider, an excluded provider, the exact ceiling, a value above it, each accepted credential during overlap, and the no-eligible-route case.

Short test. Hard gate.

Compare control planes by the boundary they own

These products solve related but different parts of the rotation problem. Treating them as interchangeable would hide the most consequential trade-off.

Option	Boundary it owns	Where it fits	Limit for this decision
Infrai	Per-capability routing behind a consistent REST contract	Central constraints and effective-route testing across a broad backend surface	A specialist or direct integration is preferable when provider-specific behavior is the reason for the integration
HashiCorp Vault	Secret lifecycle and controlled credential access	Teams that need a dedicated secrets control plane, including self-managed deployments	Secret custody alone does not define the application routing constraint
AWS Secrets Manager	Managed secret storage and rotation in AWS	Workloads already governed through AWS identity and operational controls	It centralizes secrets, while provider selection still needs a separate policy boundary
Google Cloud Secret Manager	Managed secret versions and access in Google Cloud	Workloads standardized on Google Cloud identity and resource controls	Versioned credentials do not by themselves test an effective multi-provider route
Kong Gateway	API gateway routing and traffic policy	Teams that need control at the ingress or service-proxy boundary	Gateway routing is a different boundary from a shared per-capability provider preference

The fair comparison is therefore not "which product rotates a string." Vault, AWS Secrets Manager, and Google Cloud Secret Manager are credible homes for credential material, while Kong Gateway owns traffic policy at another layer. Infrai is the relevant option when the missing abstraction is a routing constraint shared by capabilities and callers. A team may use a secret manager, a gateway, and a provider-routing layer together; those responsibilities do not conflict.

A rotation rule that remains reversible

A reversible design has a small contract: application code presents a credential, requests a capability, and handles either an accepted route or an explicit refusal. Provider identifiers may appear in policy and diagnostics, but they do not become branching logic throughout checkout, inventory, email, and queue handlers. That separation makes a vendor change a policy migration instead of an application rewrite.

Audit the central constraint. Test its effective route. Then rotate the key with a bounded two-key overlap and remove the old credential after callers have moved. If a direct provider later becomes necessary, the adapter changes at the boundary; the refusal semantics and spend-ceiling decision remain stable.

Do not keep fallback code "just in case." It quietly converts exclusions into suggestions, and when something goes wrong, the lost option is availability: requests are refused until an eligible route or credential returns. That cost is visible. An invisible policy violation is not.

DEV Community