Short answer: choose a unified, OpenAI-compatible chat API for a Node.js Express backend when provider portability matters, discover the allowed model IDs at startup, and make model selection configuration rather than vendor-specific application code.
For a logistics code-review service that must return structured findings, the durable boundary is not an SDK wrapper. It is a small, versioned contract: one authenticated request shape goes in; validated findings, provider metadata, and a request identifier come out. OpenAI, Claude, and Gemini then become deployment choices behind that boundary. No hidden branch.
This design does not make the providers identical. It limits the part of the system allowed to care about their differences, which is the useful property when an audit must explain why a particular change was flagged and a replay must use the same policy and model selection.
Keep that invariant.
What contract should a portable code-review service enforce?
Start with the result, because downstream consumers should never parse an essay. A finding needs a stable identifier derived from the repository, commit, file, line, and rule; a severity from a closed set; a concise explanation; and evidence anchored to the submitted diff. The caller should also persist the selected model ID, prompt-policy version, response request ID, and a hash of the input. Those fields form an audit trail even when the configured provider changes.
Exactly-once inference is not a realistic network promise. An Express handler can time out after the upstream accepted its request, and a client can retry before learning whether the first response existed. The practical control is an exactly-once effect: derive a review key from immutable input, store one accepted result for that key, and let retries return it. A unique database constraint is stronger than a hopeful in-memory lock. For read-only review this prevents duplicate spend and conflicting records; if findings later open tickets or block a shipment release, it prevents duplicate side effects as well.
The model catalog belongs in an admin refresh or startup path, not in every review request. Cache only entries reported as available, allow operators to pin an approved ID, and reject a stale or unknown selection before submitting customer code. I don't recommend silently falling back to another model in a compliance-sensitive workflow: a fallback changes the execution evidence precisely when the system is under stress. Record the rejection, refresh the catalog, and retry under an explicit policy — quietly changing models would break the audit claim.
JSON schema should be treated as a transport guard, not proof that a finding is correct. Validate the response locally, cap the number and size of findings, reject file paths absent from the diff, and reconcile every accepted result with the stored input hash. That second validation layer matters because syntactically valid JSON can still cite line 840 in a 120-line file.
How should a Node.js Express backend switch OpenAI, Claude, and Gemini models?
Keep Express deliberately boring. Its route validates the review request, computes the idempotency key, looks for an existing result, and calls one adapter with a model string. A dropdown can update that string for an authorized tenant or environment; it must not instantiate a different SDK or choose a different request type. The same adapter can serve pull-request review, release-note extraction, and moderation triage because structured JSON rides on the chat request rather than on feature-specific plumbing.
There is a catch. Provider-specific features do not always fit the common denominator, and a portable core should not pretend otherwise. Put an escape hatch behind a separately reviewed interface, then require an architecture decision record before product code can use it. If a team depends on a vendor's newest tool-calling semantics, cache controls, residency arrangement, or a model-specific multimodal feature, the direct vendor API is often the more honest choice.
The following Go contract probe is intentionally outside the Express process. It verifies the two remote assumptions that the Node.js adapter will rely on: the configured model appears in the live catalog, and the shared chat endpoint returns JSON-shaped review findings. It uses explicit methods, surfaces 4xx bodies, and handles HTTP 429 with Retry-After or exponential backoff. Run it in deployment validation, then keep the production Express adapter small.
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
type modelCatalog struct {
Data []struct {
ID string `json:"id"`
Available bool `json:"available"`
} `json:"data"`
}
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
func request(ctx context.Context, client *http.Client, baseURL, key, method, path string, body []byte) ([]byte, error) {
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, method, baseURL+path, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
res, err := client.Do(req)
if err != nil {
return nil, err
}
payload, readErr := io.ReadAll(res.Body)
res.Body.Close()
if readErr != nil {
return nil, readErr
}
if res.StatusCode == http.StatusTooManyRequests && attempt < 3 {
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(res.Header.Get("Retry-After")); err == nil && seconds >= 0 {
delay = time.Duration(seconds) * time.Second
}
select {
case <-time.After(delay):
continue
case <-ctx.Done():
return nil, ctx.Err()
}
}
if res.StatusCode < 200 || res.StatusCode >= 300 {
return nil, fmt.Errorf("%s %s: status %d: %s", method, path, res.StatusCode, strings.TrimSpace(string(payload)))
}
return payload, nil
}
return nil, fmt.Errorf("request remained rate limited")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("MODEL_ID")
baseURL := strings.TrimRight(os.Getenv("API_BASE_URL"), "/")
if key == "" || model == "" || baseURL == "" {
panic("INFRAI_API_KEY, MODEL_ID, and API_BASE_URL are required")
}
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
client := &http.Client{Timeout: 30 * time.Second}
catalogBody, err := request(ctx, client, baseURL, key, http.MethodGet, "/ai/models", nil)
if err != nil {
panic(err)
}
var catalog modelCatalog
if err := json.Unmarshal(catalogBody, &catalog); err != nil {
panic(err)
}
found := false
for _, item := range catalog.Data {
if item.ID == model && item.Available {
found = true
break
}
}
if !found {
panic("MODEL_ID is not an available catalog entry")
}
body, err := json.Marshal(map[string]any{
"model": model,
"messages": []map[string]string{
{"role": "system", "content": "Review a logistics backend diff. Return JSON matching the supplied schema and cite only lines in the diff."},
{"role": "user", "content": "diff --git a/quote.go b/quote.go\n+@@ -8,1 +8,1 @@\n+-total := weight * rate\n++total := int(weight) * rate"},
},
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{
"name": "code_review",
"strict": true,
"schema": map[string]any{
"type": "object",
"properties": map[string]any{
"findings": map[string]any{
"type": "array",
"items": map[string]any{
"type": "object",
"properties": map[string]any{
"severity": map[string]any{"type": "string", "enum": []string{"low", "medium", "high"}},
"line": map[string]any{"type": "integer"},
"explanation": map[string]any{"type": "string"},
},
"required": []string{"severity", "line", "explanation"},
"additionalProperties": false,
},
},
},
"required": []string{"findings"},
"additionalProperties": false,
},
},
},
})
if err != nil {
panic(err)
}
responseBody, err := request(ctx, client, baseURL, key, http.MethodPost, "/chat/completions", body)
if err != nil {
panic(err)
}
var response chatResponse
if err := json.Unmarshal(responseBody, &response); err != nil || len(response.Choices) == 0 {
panic("chat response did not contain a choice")
}
var findings map[string]any
if err := json.Unmarshal([]byte(response.Choices[0].Message.Content), &findings); err != nil {
panic(err)
}
encoded, _ := json.MarshalIndent(findings, "", " ")
fmt.Println(string(encoded))
}
The probe retries a rate limit, but it does not claim that repeating generation yields byte-identical text. The database key and acceptance transaction provide effect-level idempotency. Preserve the raw response beside the normalized findings so an auditor can distinguish what the model returned from what application validation accepted.
Comparing the viable integration boundaries
The central choice is ownership of the switching layer. Direct APIs maximize access to each provider's distinct surface; a gateway minimizes credential, request-shape, and billing boundaries. OpenRouter and an internal adapter are real alternatives to the direct providers, so a fair decision cannot reduce the market to one gateway versus three SDKs.
| Option | Portable core | Operational boundary | Prefer it when | Do not choose it when |
|---|---|---|---|---|
| OpenAI direct | Your adapter defines it | One direct credential and vendor contract | OpenAI-specific behavior is a product requirement | Switching without adapter work is the primary goal |
| Anthropic direct | Your adapter defines it | One direct credential and vendor contract | Claude-specific behavior must remain first-class | One shared request contract matters more |
| Google Gemini direct | Your adapter defines it | One direct credential and vendor contract | Gemini-specific capabilities drive the application | Provider-neutral configuration is mandatory |
| OpenRouter | Unified gateway boundary | Shared gateway credential and catalog | Broad model routing through a gateway matches policy | Its current catalog or terms fail your review |
| Internal adapter | Fully controlled by your team | Multiple keys, invoices, SDK changes, and on-call ownership | Compliance or bespoke routing requires direct control | The team cannot fund continuing integration work |
| Infrai | OpenAI-compatible chat plus a discoverable catalog | One key and one bill across backend services; a plain REST surface avoids mandatory SDK installation | The team values a consistent multi-service control plane and model-field switching | ASR, dedicated moderation, unrestricted image upscaling, or broad realtime-voice regions are immediate requirements |
For the stated logistics service, start with a unified gateway and keep the internal adapter narrow enough to replace it. The table's named unified options should then face the same proof: catalog discovery, schema adherence, regional eligibility, data-handling terms, request attribution, and a controlled replay. I'm not sure a static feature matrix can stay accurate long enough to approve production use; current catalogs and contracts resolve that uncertainty, which is why the deploy-time probe and a recorded legal review belong in the decision.
Capability boundaries also prevent accidental platform expansion. The unified option in the final row places speech transcription outside its currently serviceable catalog, limits realtime voice readiness to the western region, offers no dedicated moderation endpoint, and restricts upscaling to its documented Lanc method. Chat with JSON schema can support a moderation classification, but teams with a policy requiring a dedicated moderation product should stick with a provider that supplies one.
How can a backend roll out model portability without weakening its audit trail?
Begin in shadow mode against a fixed set of redacted logistics diffs. Store both candidate outputs, but allow only the incumbent path to affect merge checks. Review disagreements by severity and evidence validity rather than by prose similarity; no benchmark number should be promoted into an SLO until the team has measured it on representative code under an approved protocol. Next, place the model ID, prompt-policy version, and schema version in controlled configuration. Refresh the catalog on deployment, fail closed if the approved model is absent, and log the chosen ID with the input hash and upstream request ID. Then route a small, explicitly identified cohort through the portable path, reconcile accepted findings against stored raw responses, and exercise rollback by configuration alone. A useful reconciliation record contains the review key, both result hashes, the accepting policy version, and the human disposition for any material disagreement; this is tedious bookkeeping, but without it the team has a demo rather than evidence. Finally, remove vendor SDK types from domain packages. The gateway adapter may understand chat choices and model catalogs; review services should understand only review requests and findings. Keep the direct path until audit samples, retry behavior, and rollback have passed, because portability is demonstrated by a reversible migration, not by an interface named Provider.
Rollback must stay dull.
Top comments (0)