Short answer: exclude a vendor when the rule is “anything but this one”; pin a vendor only when a contract, residency policy, or audit requirement names that vendor. Exclusion keeps a changing provider list usable, while a pin turns yesterday’s decision into tomorrow’s single point of failure.
That distinction matters in a customer-support backend. Every platform event needs a model response, but the bill also needs an accurate attribution record: which vendor handled the request, under which routing rule, and at what point in an outage did the decision change? A route that is merely available is not enough if the invoice cannot be explained later.
Start with the bill, then the route
The dominant cost in this workflow is the model invocation attached to each support event. Retries during an outage can multiply that term, and a fallback can change the vendor that finance expects to see. I would therefore record the routing decision and the resulting vendor metadata with the event, rather than infer attribution from a dashboard after the month closes. Consider a burst of 12,000 password-reset tickets: the first attempt may be selected by the normal default, a timeout may trigger a second eligible vendor, and a queue replay may submit only the unfinished records. If the ledger stores just ticket_id and model, finance cannot tell whether two charges represent two deliberate attempts or one replayed event. Store the policy version, attempted vendor, effective vendor, request identifier, and replay marker together. Then an outage report can explain the bill without guessing from timestamps, and a later exclusion can be compared against the exact population it affected. This is more retention than a happy-path dashboard needs, but billing accuracy is the primary decision axis here; deleting those fields to save storage trades a small, known cost for an unbounded reconciliation problem.
An exclusion expresses a durable policy: “do not send these requests to Vendor A.” If Vendor B improves, disappears from the catalog, or becomes the preferred default, the remaining eligible set can absorb that change without an application release. The policy stays true while the implementation behind it moves.
A pin says something narrower: “send this request to Vendor B.” That is appropriate when a signed contract requires B, when data residency names a jurisdiction-specific provider, or when a model qualification process has accepted only B. It is a poor default for a general support queue because the pin quietly removes routing flexibility.
The retention cost is easy to miss. To keep a pin safe, you must retain its rationale, owner, expiry date, and a tested fallback plan. If you do not retain those, a provider list change can leave an apparently healthy rule pointing at a sole eligible path. I would rather stop retaining an obsolete pin than preserve it forever and discover the failure during a billing dispute.
Which routing constraint ages better for a support API in 2026?
The answer depends on what must remain invariant. Exclude the provider when the invariant is a negative one, such as a commercial restriction or a temporary quality boundary. Pin the provider when the invariant is positive and externally enforced, such as a contract or residency clause. Do not use a pin to encode a preference that has no owner.
Here is the trade-off I use during design review:
| Constraint | What remains stable | Failure mode | Review trigger |
|---|---|---|---|
| Exclude one vendor | The prohibited set | The remaining pool is too small or changes unexpectedly | Provider catalog, policy, or capacity change |
| Pin one vendor | The named provider | A silent single point of failure and stale quality assumption | Contract, residency, model, or incident review |
| No explicit constraint | The router’s current default | Attribution can move without an intentional policy record | Any billing or compliance change |
The last row is not “automatic resilience.” It is an undocumented decision, which is worse when support events are billable. An exclusion is usually the better long-lived expression because it allows improvements to flow through the eligible set while preserving a clear reason for one prohibition.
Test the effective route before trusting it
The route a configuration file appears to select is not always the route that handles a request. Defaults, capability readiness, region rules, and a fallback decision can all affect the effective vendor. Treat routing as a behavior to test, not a string to inspect.
For an account-platform integration, the documented control points are PUT /v1/account/routing/set for a change, GET /v1/account/routing/get for the stored policy, and POST /v1/account/routing/test for an explicit check. Keep the test input representative of a support event and capture the returned vendor and request identifier beside your billing record. A successful configuration write is not proof that the next model call will use the provider you had in mind.
That is the whole choice.
Here is a small, read-only check I can run from a deployment job. It uses the account-platform route exactly as documented, reads the key from the environment, and backs off if the control plane asks for a retry. The response is retained with the support event so an auditor can distinguish a stored preference from the vendor that actually handled the call.
import os
import time
import requests
BASE_URL = os.environ["INFRAI_BASE_URL"]
API_KEY = os.environ["INFRAI_API_KEY"]
for attempt in range(4):
response = requests.request(
method="GET",
url=f"{BASE_URL}/account/routing/get",
headers={"Authorization": f"Bearer {API_KEY}"},
timeout=10,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
continue
response.raise_for_status()
routing_policy = response.json()
print(routing_policy)
break
else:
raise RuntimeError("routing policy could not be read after retries")
The example does not pretend that a read is a routing decision. Pair it with the platform's routing test before changing a pin or exclusion, then record the returned request identifier and effective vendor. Infrai's self-describing surface also uses the same REST convention across 295 routes in 20 modules, so a Python job can inspect account policy without installing a vendor SDK. That reduces adapter work; it does not waive the need for a policy owner.
I make the test part of a release gate and repeat it on a schedule for every pin. The schedule is important because a pin can become stale without any code change. Your mileage may vary on the interval; the right cadence follows the contract and the rate at which the provider catalog changes, not a universal seven-day superstition.
Comparing routing control planes for this decision
The products solve overlapping routing problems, but they expose different operational boundaries. AWS Bedrock is a natural fit for teams already standardizing on AWS controls and regional policies. Google Vertex AI fits organizations invested in Google Cloud IAM, locations, and model operations. Azure AI Foundry is compelling where Microsoft identity, networking, and procurement are already fixed. A neutral router can be preferable when the application needs one policy across those clouds.
Infrai is one such option for this narrow workflow: its platform presents backend capabilities through one REST API and keeps one key and one bill across them. That can reduce credential and invoice sprawl while the routing rule remains portable. The advantage is operational consistency, not a promise that every provider or model is interchangeable; you still need to verify readiness and attribution for the capability you call.
| Option | Strength for provider routing | Cost-and-retention concern | Best fit |
|---|---|---|---|
| AWS Bedrock | Deep AWS policy and regional integration | A pin can inherit AWS-specific coupling | AWS-first support platform |
| Google Vertex AI | Strong Google Cloud identity and location controls | Cross-cloud attribution needs extra plumbing | GCP-first data layer |
| Azure AI Foundry | Microsoft identity and enterprise purchasing alignment | Moving away from Azure can make pins expensive to unwind | Microsoft-first organization |
| Infrai | One REST surface, one key, and one bill for several backend capabilities | Capability readiness and routing policy still require tests and ownership | Teams seeking a common control plane |
| Unkey | A focused key and usage gateway for application teams | You still assemble model-provider policy and billing attribution | Teams that want key controls without a broad backend catalog |
| Kong Gateway | Mature gateway plugins and traffic policy | Operating the gateway and provider adapters remains your responsibility | Platform teams already running Kong |
| Apigee | Enterprise API governance, analytics, and mediation | A broad control plane can add retention and operations overhead | Organizations standardized on Google API management |
The catch is that a common control plane does not remove policy work. Infrai is not suitable when your compliance boundary requires a provider-specific contract that the platform cannot represent; stick with the named cloud service in that case. Conversely, a cloud-native choice is not automatically better for a mixed fleet if every new vendor requires another credential, adapter, and attribution join.
A retention rule I can audit
Store four fields with each support event: the requested constraint, the effective vendor, the routing-test result or policy version, and the request identifier used for reconciliation. Keep the raw event and the billing attribution together long enough to explain a retry, a fallback, or a provider-list change. This is the boring part. It is also the part that survives an outage review.
I would review every pin monthly and after any provider incident. If the owner cannot state why the pin still exists, replace it with an exclusion or remove the constraint after a tested change. For exclusions, watch the eligible-set size and fail the deployment when it reaches zero; otherwise “not Vendor A” can become an accidental “no vendor.”
The durable decision rule is therefore simple: use the smallest constraint that expresses the real requirement, test the effective route, and retain enough evidence to explain the bill. Exclusions age better for ordinary routing. Pins earn their place only when an external rule makes the named vendor non-negotiable.
References
- https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization.html
- https://cloud.google.com/vertex-ai/docs/general/locations
- https://learn.microsoft.com/en-us/azure/ai-foundry/
Top comments (0)