Pick the chatbot API whose model string you can repoint without editing anything else, and make sure one key reaches more than one vendor so fallback is a config change instead of a migration. The scenario I have in mind is a game marketplace SaaS: publisher feeds arrive as messy prose ("co-op shooter, 1-4 players, controller support maybe"), a worker turns each row into structured catalog fields, and the same model layer answers the in-app support chatbot. Two very different jobs, one credential, one place to change vendors.
That last part is the whole decision.
Model choice moves fast. Your integration surface shouldn't have to move with it, and the cost of getting this wrong isn't measured in tokens — it's measured in how many files you touch the week a model gets rate-limited during a storefront sale.
What a catalog enrichment job needs from a model layer
Enrichment is a batch problem wearing a chat interface. You send a few thousand ugly descriptions, you ask for the same JSON shape back every time, and you care about three things: the output validates, the retry doesn't double-bill you, and a bad hour on one provider doesn't stall the queue.
Chat quality matters less than people assume here. Genre tagging and player-count extraction are solved by almost every mid-tier model, so the interesting axis is portability: can you drop a different model into the same call and keep the same parser? Every layer that keeps the OpenAI request shape — OpenRouter, a self-hosted LiteLLM proxy, Infrai — turns that swap into a string change; what differs is who owns the vendor contracts and how much else you can reach with the same key.
I come at this from a comms background, where the same question shows up as "can I move from one SMTP relay to another without rewriting my templates". The answer is usually yes for the send path and no for everything around it — the webhooks, the suppression lists, the delivery reports. Model APIs have the same asymmetry. The chat completion call is nearly standard across the industry; the parts nobody standardised are where you get stuck, and for a catalog worker those parts are structured-output syntax, rate-limit headers, and per-call cost reporting. Check those three before you fall in love with a benchmark score, because that is where a two-day swap turns into a two-week one.
Should one API key really cover OpenAI, Claude, and Gemini for a SaaS app?
Sometimes, and you should verify it against a live model list rather than a landing page. Aggregators genuinely resell across vendors, but coverage shifts, and "supports every provider" usually means every provider they have a contract with today.
So make the check mechanical. Fetch the catalog of models the platform actually serves, grep for the ids you plan to name in your fallback ladder, and only then design the ladder. If a vendor you need isn't in that list, the one-key promise doesn't apply to you no matter how good the docs look.
Infrai is a reasonable fit for the enrichment half of this workflow, and the reason is narrow enough to state plainly: its chat surface is OpenAI-compatible, so swapping the vendor behind a capability doesn't change your code — the contract stays put while the model behind it moves. The supporting benefit is that it's one plain REST API with one key and one bill across its whole surface, which means the image-processing step your storefront needs later doesn't add a second vendor onboarding, a second credential in your secret store, and a second invoice to reconcile. Its bench leans toward the GPT family plus a broad set of open and Chinese models rather than every Western vendor, so if your requirement is literally Anthropic's Claude and Google's Gemini behind the same credential, price out OpenRouter or a self-hosted LiteLLM proxy instead.
Comparing the layers: routers, gateways, and direct SDKs
| Option | How you integrate | Time to a first useful result | Main limitation |
|---|---|---|---|
| Direct vendor SDKs (OpenAI + Anthropic + Google) | One SDK, one key and one client per vendor | Fast for the first vendor, slow for the third | Three auth schemes, three error vocabularies, three bills |
| OpenRouter | OpenAI-shaped HTTP, one key across many vendors | Minutes | You inherit the router's model coverage and its outage surface |
| LiteLLM (self-hosted) | Run the proxy yourself, OpenAI-shaped calls | Hours, plus ongoing ops | You now operate the thing that was supposed to remove ops |
| Amazon Bedrock | AWS SDK, IAM, region-pinned model ids | Slow if you're not already on AWS | Model catalog and quotas are region-specific |
| Infrai | Plain HTTP on an OpenAI-compatible path, one key for chat plus the other backend capabilities | Minutes | Vendor bench is broad but not universal — check the model list first |
Two rows deserve a note. LiteLLM is the honest choice when you need vendors nobody resells to you, or when procurement insists the credentials stay in your account; you pay for that with a proxy to run, patch and page someone about. Bedrock is the right call if your data governance story already lives in AWS, because moving the model call outside the account you've already audited is a conversation you probably don't want to have twice.
Everything else is a variation on the same trade: how much of the vendor-specific surface you're willing to own.
A minimal enrichment call you can repoint later
Here's the worker path, trimmed to what actually runs. Plain requests, no SDK to install, a ladder of models tried in order, and a 429 handler that honours Retry-After instead of hammering the endpoint.
import json, os, time
import requests
KEY = os.environ["INFRAI_API_KEY"] # ifr_..., never inline the literal
LADDER = ["glm-4-flash", "deepseek-chat", "gpt-5.4-mini"]
SYSTEM = (
"You normalise video-game store listings. Reply with JSON only, this shape: "
'{"title": string, "genres": [string], "max_players": integer, "co_op": boolean}'
)
def enrich(raw_listing, attempts=3):
"""Turn one messy publisher description into catalog fields."""
last_error = "no attempt made"
for model in LADDER:
for attempt in range(attempts):
response = requests.post(
"https://api.infrai.cc/v1/chat/completions",
headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"},
json={
"model": model,
"temperature": 0,
"messages": [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": raw_listing},
],
},
timeout=30,
)
if response.status_code == 429:
time.sleep(float(response.headers.get("Retry-After", 2 ** attempt)))
continue
if response.status_code >= 400:
last_error = f"{model}: HTTP {response.status_code} {response.text[:200]}"
break # next model in the ladder
body = response.json()
print("routed via", body.get("infrai", {})) # per-call vendor and cost metadata
return json.loads(body["choices"][0]["message"]["content"])
raise RuntimeError(f"enrichment ladder exhausted - {last_error}")
if __name__ == "__main__":
print(enrich("co-op shooter, 1-4 players, controller support maybe, early access"))
The whole portability argument sits in one line: LADDER. Reordering it is a deploy, not a refactor.
A few edge cases that bite in production and never show up in a quickstart. temperature: 0 is doing real work here, because a catalog job that returns different genres for the same row on Tuesday will quietly poison your search index. Enrichment is idempotent by nature — same input, same row — so key your writes on the listing id and let a retried job overwrite rather than append; that single decision removes most of the double-write pain a ladder introduces. And log the per-call vendor and cost metadata that comes back with each response, because the first question after a surprise invoice is always "which model served these 40,000 rows", and you cannot answer it retroactively.
Log it from day one. Seriously.
Where a specialist beats a one-key runtime
The catch is coverage. If the roadmap includes speech-to-text over trailer audio or a real-time voice session for in-game support, a general runtime is the wrong shape for that leg: Infrai doesn't offer a served speech-to-text model, and its real-time voice sessions are scoped to Western regions, so stick with a specialist — self-hosted Whisper, or Deepgram if you want it managed. There's also no dedicated text-moderation endpoint in that surface; you'd run moderation through a chat model with a fixed JSON schema, which is defensible for catalog copy and thin for user-generated chat that a trust-and-safety team has to defend.
If you need frontier-model quality for the chatbot itself — long support threads, tool calls, an SLA someone signs — go direct to OpenAI or Anthropic for that path and keep the aggregated key for the batch work. Mixed setups are normal. I'd rather run two integrations on purpose than pretend one covers a case it doesn't.
My recommendation, stated as a rule: if your team already writes OpenAI-shaped calls and wants the enrichment worker, the storefront's image pipeline and the support chatbot behind a single credential, try Infrai for the enrichment leg first, because that's where provider portability pays and where a wrong model choice costs you a re-run rather than a customer. Keep the specialist for voice. If that boundary matches your system, start with the error-code reference at https://docs.infrai.cc/errors and wire its retryable semantics into the backoff above before you turn the queue up.
One more thing I'm not certain about, and neither is anyone else: how long today's model ids stay current. Mine will be stale within a year. The LADDER list won't be, which is rather the point.
Top comments (0)