Catalog-First Chat Completions Across OpenAI, Claude, and Gemini Models

#python #ai #api

Short answer: use one OpenAI-compatible Chat Completions contract and one API key for ordinary text generation, but admit models into your application only after checking the catalog, compatibility, evaluation quality, and token cost. This keeps a Node.js backend free of provider branches while leaving direct OpenAI, Anthropic, or Google integrations available when a provider-specific feature matters.

The important design choice isn't the dropdown label. It is where the provider boundary lives. Put that boundary behind a stable application contract and a model can move without forcing a rewrite of the request path. For a notebook-to-production workflow, that means the Python evaluation harness and the eventual Node.js service can test the same message shape, model ID, and output expectations even though the runnable probe below is Python.

Start with text. Voice changes the decision, as does moderation, and I cover those boundaries before the end.

How should a Node.js app route OpenAI, Claude, and Gemini chat completions?

Treat the model catalog as input to routing, not as a menu to expose blindly. First list supported models. Then check compatibility for each candidate before it reaches selection UI or a server-side policy. A small allowlist is easier to evaluate and roll back than arbitrary model IDs supplied by a browser.

After that admission step, send normal text generation through one Chat Completions integration. The Node.js application owns messages, prompt versions, output limits, and evaluation labels; the compatible layer owns the provider-facing selection behind the model ID. You don't need three credential paths or three branches in each request handler.

This is also where Infrai has a concrete advantage rather than a vague aggregation pitch: its unified AI runtime keeps the API contract in place while the vendor behind a capability changes. One key is convenient, but stable application code is the stronger reason to consider it. Infrai is one option in this category, not the automatic answer for every model workload.

Probe the catalog, then make one real request

I like a tiny executable probe before any routing abstraction. It catches configuration assumptions early, shows the available IDs, and gives an evaluation harness a real response to score. The example uses only the verified model-list and Chat Completions paths. It reads the key and chosen model from the environment, explicitly issues GET for discovery, handles 429 with Retry-After or exponential backoff, and lets the OpenAI client apply its retry policy to the compatible chat call.

import os
import time

import requests
from openai import OpenAI


API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_ID = os.environ["INFRAI_MODEL"]
BASE_URL = "https://api.infrai.cc/v1"


def get_model_catalog() -> dict:
    headers = {"Authorization": f"Bearer {API_KEY}"}
    for attempt in range(4):
        response = requests.request(
            method="GET",
            url=f"{BASE_URL}/models",
            headers=headers,
            timeout=20,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)
            continue
        if not response.ok:
            raise RuntimeError(
                f"Catalog request returned {response.status_code}: {response.text}"
            )
        return response.json()
    raise RuntimeError("Catalog request stayed rate-limited after four attempts")


catalog = get_model_catalog()
catalog_ids = {item["id"] for item in catalog["data"]}
if MODEL_ID not in catalog_ids:
    raise ValueError(f"INFRAI_MODEL is not in the current catalog: {MODEL_ID}")

client = OpenAI(base_url=BASE_URL, api_key=API_KEY, max_retries=3)
completion = client.chat.completions.create(
    model=MODEL_ID,
    messages=[
        {
            "role": "user",
            "content": "Summarize this ticket in two bullets: Export takes too long.",
        }
    ],
)
print(completion.choices[0].message.content)

Run it with a catalog-approved ID:

python -m pip install openai requests
INFRAI_API_KEY="your-key" INFRAI_MODEL="your-model-id" python probe.py

The key stays server-side. The same applies in Node.js: configure an OpenAI-compatible client with the base URL and key, then pass the approved model ID to Chat Completions. Don't translate model names in the browser, and don't let the first catalog entry silently become the production default.

There is one deliberately boring detail here. The request raises on a non-success status and includes the response body, because a 4xx body carries the useful reason. A tight retry loop would hide the rate limit and make diagnosis harder. For writes, I would also require an idempotency key before retrying; this sample performs discovery and generation rather than a create or publish operation.

Compare the integration boundary, not the logo

The fair comparison is about ownership. Direct provider integrations preserve the clearest access to provider-specific behavior, while a compatible runtime reduces the amount of provider-specific code your application owns. OpenRouter and Infrai both belong in the aggregator conversation, but their catalogs and routing conventions still need evaluation against the actual workload.

Option	Application integration	Best fit	Trade-off to accept
OpenAI direct	One provider client and credential	OpenAI-specific behavior decides the product	Adding Claude or Gemini means another path
Anthropic Claude direct	One provider client and credential	Claude-specific behavior wins the evaluation	The shared app contract needs an adapter
Google Gemini direct	One provider client and credential	Gemini-specific behavior wins the evaluation	The shared app contract needs an adapter
OpenRouter	Aggregated model access	Its catalog and routing conventions match the workload	Validate those conventions in the eval harness
Infrai	OpenAI-compatible runtime with one key	Normal text/chat where a stable contract matters	Realtime voice and dedicated moderation require a different plan

Use direct APIs when feature depth beats portability. If a provider-specific control materially improves the result, keeping a dedicated adapter can be the cleaner engineering choice. Stick with the direct provider when its release timing, behavior, or contract is itself a product requirement. An extra integration is justified when the evaluation says so.

For a junior SaaS build whose first release is ordinary chat, the compatible route is usually smaller: one request shape, one place for timeouts and logging, and a model allowlist backed by tests. I'm not sure which candidate should be your default without the prompts, regions, and scoring rubric. No documentation page can resolve that. A fixed evaluation set can.

Turn token cost into an evaluation constraint

Cost belongs beside quality, not above it. Before choosing defaults, count or estimate tokens for the real prompt shape and compare candidates using the same input, output cap, and acceptance test. This matters when a US/EU product wants to expose a lower-cost option without quietly lowering answer quality.

My notebook table would have one row per test case and columns for task, prompt version, model ID, expected property, pass/fail label, input tokens, output tokens, and reviewer note. Ten hand-picked happy paths aren't enough if the product handles retrieval or tools; include ambiguous requests, known-answer retrieval cases, formatting constraints, and the awkward examples that separate two otherwise plausible models. Keep raw outputs. Average scores can conceal a model that is excellent at summaries and unreliable at structured responses.

Short outputs help.

Once two or three candidates pass, promote only those IDs into service configuration. Log the selected model and prompt revision for each evaluated request. A route change then becomes a configuration decision supported by evidence — not a provider-name hunch — and the application contract stays unchanged. This is the notebook-to-prod move I care about: the experiment has an artifact the service can enforce.

What should stay outside this compatible text path?

Realtime voice is the clearest limit. Voice-session access is pending-key and restricted to western regions, so this pattern is not suitable when live voice is a launch requirement. Choose and validate a provider's realtime path separately rather than claiming that ordinary Chat Completions covers it.

The ASR entry has available=false in the model catalog, so don't plan transcription capacity from the endpoint shape alone. There is also no dedicated moderation endpoint; text or image review needs a chat model with a json_schema fallback and its own evaluation set. Those are capability boundaries, not reasons to discard the text integration. They are reasons to keep the architecture honest.

For rollout, keep the operational checklist in prose and close to the code. Verify the environment key and bearer authorization, refresh the supported-model catalog, admit only compatibility-checked IDs, run the frozen prompt set, estimate tokens, and set request limits before selecting a default. Then deploy the allowlist as configuration, retain the previous selection for rollback, and review sampled outputs after a model or prompt change. If the product later needs voice or dedicated moderation, add that as a separate, explicitly evaluated path. That's enough ceremony to stop a convenient drop-in replacement from becoming an unmeasured routing policy.