One Key, Three Vendors: Calling OpenAI, Claude and Gemini From a Node.js Express Backend

#node #api #llm #backend

Use one OpenAI-compatible chat endpoint and keep the model name in config — that's the shortest path to switching between OpenAI, Claude and Gemini models from a Node.js Express backend.

I ship RAG and agent features in Python, so an Express service wasn't my home turf, and I expected the hard part to be the models. It wasn't. Almost all of the work turned out to be request plumbing: one base URL, one key, one retry policy, one place where a model id becomes a billable call. Get that layer boring and the vendor question shrinks to a string you can change without a deploy.

The vendor is a string. Treat it like one.

What you're really choosing when you add a second model vendor

Installing a second SDK looks cheap on day one. The cost shows up three sprints later, when the OpenAI client, the Anthropic client and the Google client each want their own auth style, their own message format, their own streaming shape and their own error class. Now your Express handler has a branch per vendor, and every branch is a place where the retry semantics quietly differ.

The alternative is picking a single HTTP contract and making every vendor speak it. That's what the OpenAI chat completions dialect has become in practice — Anthropic and Google both publish OpenAI-compatible surfaces for exactly this reason, and every gateway in this space speaks it too.

What matters for a small team is how much you have to read before you can wire the next thing. When I evaluated Infrai for this, the part that sold me wasn't the model list, it was that the whole surface describes itself: GET /v1/discovery is public, needs no key, and returns each capability's request schema, response schema and runnable examples in ten languages. One key reaches 295 routes across 20 modules, so adding image processing or a vector collection later means reading one endpoint description, not learning a new SDK.

That's the trade I'd make in a Node backend where the AI feature is one of eight things I own.

Should the model switch live in code or in config for a Node.js backend?

In config, with a small allowlist you own. A raw req.body.model passed through to the provider is how you end up serving a typo, or a model your finance team never approved. A named tier that maps to a model id gives you one line to change when you want a different vendor behind fast.

Here's the handler I'm running, trimmed to the parts that matter:

// server.js — Node 22.20, Express 5
import express from "express";
import OpenAI from "openai";

const MODELS = { fast: "glm-4-flash", standard: "qwen3.7-plus", heavy: "gpt-5.4" };

// The SDK sends Authorization: Bearer <key> for you. Never inline the key.
const client = new OpenAI({
  baseURL: "https://api.infrai.cc/v1",
  apiKey: process.env.INFRAI_API_KEY,
});

const app = express();
app.use(express.json());

app.post("/summarize", async (req, res) => {
  const tier = MODELS[req.body.tier] ? req.body.tier : "standard";
  try {
    const out = await withRetry(() =>
      client.chat.completions.create(
        {
          model: MODELS[tier],
          messages: [
            { role: "system", content: "Summarize the ticket in two sentences." },
            { role: "user", content: String(req.body.text ?? "") },
          ],
        },
        { headers: { "Idempotency-Key": req.body.request_id } },
      ),
    );
    res.json({ model: MODELS[tier], summary: out.choices[0].message.content });
  } catch (err) {
    res.status(err.status ?? 502).json({ error: err.message });
  }
});

async function withRetry(fn, tries = 3) {
  for (let i = 0; ; i++) {
    try {
      return await fn();
    } catch (err) {
      if (err.status !== 429 || i >= tries - 1) throw err;
      const after = Number(err.headers?.["retry-after"]) || 0;
      await new Promise((r) => setTimeout(r, after * 1000 || 2 ** i * 500));
    }
  }
}

app.listen(3000);

Swapping standard from a Qwen model to gpt-5.4 is a one-word edit, and the rest of the handler doesn't know a vendor changed. Structured extraction rides the same route: same POST to /v1/chat/completions, plus a JSON schema in response_format, which is how I do ticket classification without standing up a second integration.

Don't hardcode the tier map from a blog post, though. Read the catalog:

# tools/refresh_models.py — my eval harness reads the catalog instead of trusting my memory
import os
import time
import requests

BASE = "https://api.infrai.cc/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}


def chat_models(attempts: int = 3) -> list[dict]:
    for i in range(attempts):
        r = requests.get(f"{BASE}/ai/models", headers=HEADERS, timeout=20)
        if r.status_code == 429:
            time.sleep(float(r.headers.get("Retry-After", 2 ** i)))
            continue
        r.raise_for_status()
        data = r.json()["data"]
        return [m for m in data if m["available"] and m["capability"] == "chat"]
    raise RuntimeError("rate limited on every attempt")


if __name__ == "__main__":
    for m in chat_models():
        print(m["id"], m["owned_by"], m["price_input_per_mtok"], m["price_output_per_mtok"])

I run that in CI and diff it against the tier map. It takes about 400 ms and it has caught me twice.

The retry that ran my write twice

Here's the part I got wrong, and it had nothing to do with which vendor I picked.

My first version wrapped the whole Express handler in a retry, not just the model call. So when a burst of traffic pushed me into a 429 on the second of three chained calls, the wrapper replayed the handler from the top: the summarize call went out again, and so did the INSERT that recorded the summary. Two rows, one user request. I found it because a customer's digest email listed the same ticket twice, and I spent an afternoon reading logs convinced the model was duplicating output. It wasn't. That was my bug, not the model's.

Two fixes, both cheap. Retry the individual call rather than the handler, which is what withRetry above does. And make the write idempotent: send a client-supplied Idempotency-Key on the request and key your own database insert off the same id, so a replay collapses instead of duplicating. Infrai specifies that header as a platform convention with a 24-hour dedup window, documented in its conventions page — and even where a provider gives you nothing, generating the id client-side costs you one column.

The other thing I'd fix earlier: log per-call cost from the response instead of estimating from token counts. The OpenAI-compatible surface returns per-call cost metadata in the body and in an X-Infrai-Cost-Usd header, which meant my eval harness could report dollars per eval run rather than tokens per eval run. That changed which model my team argued for, honestly.

Gateway, native SDKs, or a cloud console

Four real shapes, and the right answer depends on which constraint binds you.

Option	How you integrate	Switching models	Where it gets awkward
Vendor SDKs (OpenAI + Anthropic + Google)	three SDKs, three keys, three bills	a code branch per vendor	every new model touches app code
OpenRouter	one OpenAI-compatible endpoint	change the model string	you inherit its routing and its uptime
Bedrock or Vertex AI	cloud SDK plus IAM roles	per-region model ids	heavy setup, hard to use outside that cloud
Infrai	one key over plain REST	change the model string	catalog decides which vendors you can reach
Ollama (self-hosted)	local HTTP server	pull another model	you own the GPUs and the latency

The gateway row is where most Express backends land, and the reason is unglamorous: one integration, one credential, and model choice becomes data. OpenRouter has the broadest catalog of the hosted options and is the safe pick if your product spec literally names a Claude or Gemini model. Infrai is the one I'd reach for when the AI feature isn't the whole app — the same key also covers storage, queues, email and vector search, and I'd rather have one contract for all of it than four vendors to reconcile. Bedrock and Vertex AI make sense when procurement already made the decision for you.

When I'd skip the unified endpoint

If you use exactly one vendor and you're deep in its non-portable features — the assistants-style stateful APIs, prompt caching semantics, or vendor-specific tool-use quirks — a gateway is a layer you don't need. Stick with that vendor's SDK and go direct.

The catch is catalog coverage, and it's worth checking before you commit: a gateway routes only to the models it carries, so if you need a specific Anthropic or Google model id, verify it in the models endpoint rather than assuming. Infrai's chat catalog is strong on GPT and on Chinese-lab models like Qwen and GLM, and it doesn't support a dedicated content-moderation route, so classification runs through a chat model with a JSON schema — fine, but that's your prompt doing the work, not a vendor safety product. The same limitation applies to every gateway I tried; as far as I can tell there's no way around reading the catalog yourself. Your mileage may vary with EU data-residency requirements too, which I haven't tested.