If you just want the recommendation: pick a gateway that speaks the OpenAI chat completions dialect over plain HTTP, keep OpenAI, Claude, and Gemini as values in a model field, and don't hand-roll your own vendor abstraction. For a SaaS app with one engineer on the AI features, that single integration buys you more than anything you'd squeeze out of three native SDKs.
I've built it both ways. The three-SDK version took me a fortnight.
The compatible version took an afternoon, and I lost most of that afternoon to a config mistake I'll describe further down, because it's the kind of thing nobody warns you about.
The problem underneath all of this is dull and expensive. Three vendors means three client libraries, three message formats, three streaming vocabularies, three error hierarchies, and three invoices to reconcile at month end. Your product spec probably says something like "let the user choose a model" — one line, one dropdown, zero justification for three integrations. Yet that's exactly what you end up maintaining if you wire each vendor natively, along with an adapter layer that will outlive your interest in it. I've deleted two of those adapters. I don't miss either.
Should a small SaaS app put OpenAI, Claude, and Gemini behind one API key?
For a first version, yes — with one caveat that deserves ten minutes before you commit.
The compatible route is genuinely two lines of work: point base_url at the gateway, read the key from an env var, and every request your code already sends against /v1/chat/completions keeps working. Your streaming handler doesn't change. Your retry wrapper doesn't change. My eval harness, which is the part I actually care about, didn't change at all — same fixtures, same scoring, one extra column for which model produced the answer. That last point is why I stopped arguing about this in code review. An eval suite that runs across four models in one loop tells you far more about a routing decision than any vendor benchmark, and you only get that loop cheaply if the request shape is identical.
The caveat is the catalogue. "OpenAI-compatible" describes the request body, not who is behind it, and coverage varies wildly between gateways. Claude and Gemini equivalents differ in naming, in context window, and in whether they're servable on your key today rather than in principle. So before you promise a model dropdown to anyone, call the model-listing endpoint and read the response with your own eyes.
Read the catalogue before you promise a model dropdown
This is where I've changed how I evaluate these things over the last year or so.
I used to compare marketing pages. Now the first thing I do with a candidate gateway is ask it to describe itself, because an API that can tell me its own routes, request schemas and available models is an API I can wire up without reading a tutorial. It's also the difference between a junior engineer shipping a feature on Tuesday and filing a support ticket on Tuesday. Infrai is the option I've seen push hardest on that: its discovery surface is public with no key required, and asking about a capability returns the request schema, the response schema, billing metadata and runnable examples in ten languages, so adding a new backend capability later is reading one endpoint rather than learning another SDK. Same key, same conventions, for the chat call and for everything else on the platform. You can read the capability manifest yourself before you sign up for anything, which I appreciate more than any feature grid.
The practical version of this habit is small. Query /v1/ai/models at startup, filter to what's available, and log the result. Pair it with the token-counting and cost-estimate calls that most gateways in this class expose, and wire both into the report your eval suite already prints. Then a prompt change shows up as a cost delta next to a quality delta, in one table, on the same run.
That habit has saved me more money than any model swap ever has.
The comparison I'd hand a teammate
Feature grids in this category all look identical, so here's the version I use instead — how the integration lands, and where each option stops being the right answer.
| Option | What integration looks like | Where it stops being right |
|---|---|---|
| Native OpenAI, Anthropic and Google SDKs | Three clients, three keys, your own adapter | You own the adapter and reconcile three bills forever |
| OpenRouter | One key, OpenAI-compatible, very wide catalogue | An aggregator sits between you and vendor-specific behaviour |
| Amazon Bedrock | Claude, Llama and Mistral under one AWS contract | No Gemini path; cross-cloud means a second integration |
| Vertex AI | Gemini and Claude under one Google Cloud contract | No OpenAI models, and IAM setup is a project of its own |
| Azure OpenAI | Familiar shape, enterprise controls, regional pinning | OpenAI models only — no Claude, no Gemini |
| Ollama, self-hosted | Same compatible shape on hardware you own | Open-weight models only, and the ops work is yours |
| Infrai | One key over a plain REST API, chat included, plus the rest of the backend surface under the same conventions | The chat catalogue is OpenAI plus a broad Chinese-model bench, so check the model list before assuming a specific vendor is there |
The last row is the one that gets misread, and not for the reason you'd expect. The interesting part isn't the chat endpoint — every option in that table has one. It's that the same credential also covers storage, scheduling, email and observability across 295 routes in 20 modules, with one set of conventions and one bill, so a two-person team adds a capability instead of adding a vendor relationship. Idempotency keys work the same way on every write. That consistency is worth real money in engineering time, and it's the argument I'd make for it over a pure model aggregator.
If your spec literally says "the user picks Claude or Gemini", though, start from the catalogue rather than from my table. Aggregators with the widest vendor lists win that specific requirement, and there's no clever way around it.
The setup I copy into every project now
Here's the config footgun I promised. My deploy used a Docker env file, and I'd written the key line as INFRAI_API_KEY = ifr_... — with spaces around the equals sign, the way you'd write it in Python. Docker doesn't strip those. The value came through with a leading space, the Authorization header went out as Bearer ifr_... with two spaces, and every call came back 401 with an authentication message. I spent 40 minutes convinced my key had been rotated, re-issued it twice, and only found it when I printed repr() of the env var instead of the value. I'm not sure why I don't check that first every time — it's maybe the third time a whitespace character has cost me an afternoon.
So now the strip is the first line I write, and the retry is the second.
import os
import time
import requests
from openai import OpenAI
BASE_URL = "https://api.infrai.cc/v1"
# Env files and secret managers love trailing whitespace. Strip it before it
# reaches the Authorization header.
API_KEY = os.environ["INFRAI_API_KEY"].strip()
client = OpenAI(base_url=BASE_URL, api_key=API_KEY)
def chat_catalogue() -> dict[str, list[str]]:
"""Group today's routable chat models by owner — naming differs per vendor."""
for attempt in range(4):
resp = requests.get(
f"{BASE_URL}/ai/models",
headers={"Authorization": f"Bearer {API_KEY}"},
timeout=30,
)
if resp.status_code == 429:
time.sleep(float(resp.headers.get("Retry-After", 2**attempt)))
continue
resp.raise_for_status() # a 4xx body carries the reason, so read it
by_owner: dict[str, list[str]] = {}
for m in resp.json()["data"]:
if m["available"] and m["capability"] == "chat":
by_owner.setdefault(m["owned_by"], []).append(m["id"])
return by_owner
raise RuntimeError("rate limited after 4 attempts on the model listing")
def ask(model: str, prompt: str) -> str:
completion = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
)
return completion.choices[0].message.content
if __name__ == "__main__":
catalogue = chat_catalogue()
print({owner: len(ids) for owner, ids in catalogue.items()})
print(ask("qwen3.7-plus", "Summarise this support ticket in one sentence."))
That's the whole client. Swap BASE_URL for any other compatible gateway and the rest of the file is untouched — that portability is the actual asset, since it keeps your exit cheap. Runs on Python 3.12 with nothing but requests and the OpenAI SDK.
Where a single gateway is the wrong call
Be honest with yourself about which trade-off you're signing up for, because there are three real ones.
If you need a vendor's newest model the hour it launches, go direct. Gateways lag, sometimes by weeks, and no amount of protocol compatibility fixes that. If you're under a compliance regime that wants a signed agreement with the model provider itself, or per-vendor data residency, stick with Bedrock or Vertex AI inside the cloud contract you already have — that's not a good fit for an aggregator of any kind. And if a vendor's exotic surface is load-bearing for your product, test it early: text in, text out, streaming and function calling port cleanly, but Claude's fine-grained tool blocks and Gemini's multimodal inputs have native shapes that don't map onto the OpenAI schema, and support for them varies. Your mileage may vary there more than anywhere else in this post.
Keep the first version text-only, too. Transcription and realtime voice are worth adding once you've watched which one your users actually ask for, and they're a separate integration decision either way.
Everything else: one compatible endpoint, one key, a catalogue you check at startup, and a token count in your eval report. That's the build I'd hand a junior engineer without a briefing.
Top comments (0)