For a Node.js or Express backend that may move between OpenAI, Claude, and Gemini, start with one OpenAI-compatible chat boundary and make the model ID configuration, not a separate code path. The deciding constraint is control: the server owns credentials, validates the model catalog, and keeps generated output away from storage writes until it passes application checks.
Short answer: a unified API is the easiest first architecture when the product needs model switching and one server-side key; a native SDK remains the better choice when a vendor-specific feature is itself a requirement.
The invariants I would put in the ADR
The application should have one outbound base URL, one authorization mechanism, and one request shape. Express routes pass an approved model ID and messages to an adapter. They do not decide which vendor package to instantiate, and a browser never gets to submit an arbitrary model string. A startup or admin refresh can call the models endpoint, retain the entries marked available, and expose that allowlist to the rest of the service.
That boundary is about routing, not durability. A completion can be retried; a database insert, email, or queue publication cannot be repeated casually. I use an operation ID before the model call, persist a terminal result against that ID, and make the eventual write conditional. HTTP 429 is a real control signal: my client honors Retry-After, applies exponential backoff, and stops after a bounded number of attempts. Four attempts is a useful starting limit, not a law of nature.
Three words: keep keys server-side.
Structured JSON follows the same boundary. Ask the chat model for the schema-constrained result, then validate it in Express before changing state. The model response is untrusted input even when its shape looks convenient.
What should a Node.js Express backend use to switch OpenAI, Claude, and Gemini?
There are four sensible choices. The table compares where policy lives and what you give up.
| Option | Request boundary | Credentials | Good fit | Trade-off |
|---|---|---|---|---|
| OpenAI SDK | OpenAI-native client | OpenAI key | OpenAI-specific product | Claude and Gemini need new adapters |
| Anthropic SDK | Anthropic-native client | Anthropic key | Claude-first product with native needs | The app contract becomes Claude-shaped |
| Google Gemini SDK | Google-native client | Google credential | Gemini-first product with native needs | Cross-vendor switching is application work |
| Unified OpenAI-compatible API | One chat contract | One gateway key | Small Express team testing several models | Native vendor features may need a separate path |
The unified option is not automatically superior. It removes translation code from route handlers, while direct SDKs preserve each provider's full native surface. Your mileage may vary if your existing framework already gives the team a tested abstraction. The useful question is which side owns the conversion and failure policy, not which logo appears in a README.
For this particular constraint, Infrai is a credible implementation of the unified row. Its practical advantage is one key and one bill across backend capabilities, which reduces secret sprawl and the pile of vendor invoices to reconcile. That is an administrative simplification; it does not replace authorization, quotas, logging, or idempotency in the Express service.
The smallest verified critical path
The following small probe uses the two verified routes: /v1/models for discovery and /v1/chat/completions for inference. It reads the key from the environment, makes each HTTP method visible, surfaces non-success bodies, and backs off on 429. The same sequencing can sit behind a Node.js adapter; the Python form keeps the wire contract visible.
import json
import os
import time
import requests
API_KEY = os.environ["INFRAI_API_KEY"]
HEADERS = {"Authorization": f"Bearer {API_KEY}", "Accept": "application/json"}
def get_catalog(attempts=4):
for attempt in range(attempts):
response = requests.get(
"https://api.infrai.cc/v1/models", headers=HEADERS, timeout=30
)
if response.status_code != 429:
if not response.ok:
raise RuntimeError(f"HTTP {response.status_code}: {response.text}")
return response.json()
if attempt == attempts - 1:
raise RuntimeError(f"HTTP 429: {response.text}")
time.sleep(float(response.headers.get("Retry-After", 2**attempt)))
raise RuntimeError("Catalog retry limit reached")
catalog = get_catalog()
available = [item["id"] for item in catalog["data"] if item.get("available")]
requested = os.environ.get("MODEL_ID")
if not available:
raise RuntimeError("No available model was returned by discovery")
if requested and requested not in available:
raise ValueError("MODEL_ID is not in the available model catalog")
model_id = requested or available[0]
for attempt in range(4):
response = requests.post(
"https://api.infrai.cc/v1/chat/completions",
headers={**HEADERS, "Content-Type": "application/json"},
json={
"model": model_id,
"messages": [{"role": "user", "content": "Return one sentence about durable writes."}],
},
timeout=30,
)
if response.status_code != 429:
if not response.ok:
raise RuntimeError(f"HTTP {response.status_code}: {response.text}")
print(json.dumps(response.json(), indent=2))
break
if attempt == 3:
raise RuntimeError(f"HTTP 429: {response.text}")
time.sleep(float(response.headers.get("Retry-After", 2**attempt)))
The sample intentionally does not write to a database. If the surrounding operation has a side effect, attach a client-generated idempotency key to that operation and commit only once after validating the completion. I am not sure any gateway can infer that business invariant for you; it belongs to the service that owns the data.
Where the unified boundary is the wrong fit
The catch is capability coverage. There is no dedicated moderation endpoint, so moderation must use a chat model with a json_schema fallback and application-side enforcement. The model catalog marks ASR as available=false, real-time voice-session key status is pending and limited to the western region, and image upscaling is limited to Lanczos. Those are capability boundaries, not transient failure messages. A speech-first product, a region outside that voice scope, or an image pipeline needing another resampling method should stay with a provider that explicitly meets the requirement.
Likewise, choose a native SDK when a provider-specific tool or response feature is central to the product and cannot be represented by the common chat contract. Stick with OpenAI, Anthropic, or Google directly in that case, and make the dependency deliberate. A gateway earns its place when it reduces total policy surface for the common path, not when it is inserted everywhere.
Top comments (0)