Short answer: In a Node.js backend, use one proxy with one API key, keep logical model names in application code, resolve those names against the unified runtime catalog at deployment, and make request identity, retries, and audit evidence explicit.
For a B2B SaaS product extracting fields from supplier invoices, that design is less about making three vendors look alike than about preventing a vendor decision from leaking into every customer workflow. OpenAI, Anthropic Claude, and Google Gemini can sit behind one backend policy boundary; the browser sees invoice-fast or invoice-careful, while the proxy owns the API key, provider selection, validation, and evidence needed for reconciliation.
This is the least complex option that preserves a credible exit path.
Define the supplier invoice acceptance test first
The durable interface is the extraction result: supplier name, invoice number, invoice date, currency, total, and an explicit validation status. Provider request objects are implementation details. If a frontend sends a vendor model ID, that ID becomes stored product state in saved workflows, analytics, support screenshots, and sometimes customer integrations; replacing it later is a data migration rather than a configuration change.
Define two or three logical service levels instead. invoice-fast might serve interactive review, while invoice-careful handles a low-confidence retry. The mapping belongs on the server and should be versioned alongside the extraction schema. A mapping record needs the logical name, resolved model ID, catalog observation time, prompt version, output schema version, and rollout state. I don't treat the map as a loose environment-variable alias, because an unexplained mapping change can alter financial data even when the HTTP contract stays identical.
There are four controls worth keeping separate: catalog validation establishes that the configured model is available; deterministic request identity prevents an application retry from creating two ledger events; a bounded retry budget handles throttling without hiding prolonged contention; and an append-only audit record connects the invoice hash, logical model, resolved model, prompt version, and upstream request metadata. Exactly-once execution isn't a realistic property of an arbitrary network call. Exactly-once business effect is achievable when the durable ledger rejects a second commit for the same tenant and request ID.
One detail matters more than it first appears. Never write raw invoice text into that audit record. Hash the canonical input, encrypt the source document in the system of record, and retain only the fields required by the applicable accounting, privacy, and audit policies. I'm not sure which retention period applies to your customers; legal jurisdiction, contract terms, and whether an invoice contains personal data determine that answer, so the proxy should accept a policy-derived retention class rather than bake in a universal number.
How can a Node.js backend proxy implement OpenAI, Claude, and Gemini model mapping?
Even if the surrounding service is Node.js, the boundary is language-neutral: one HTTPS endpoint, Bearer authentication, and JSON. The implementation below is Go because a transport example is easier to audit when every network decision is visible, but the same state machine applies in Node.js. The env setup supplies AI_BASE_URL, INFRAI_API_KEY, MODEL_INVOICE_FAST, MODEL_INVOICE_CAREFUL, MODEL_PROFILE, and INVOICE_TEXT; set the base to the approved runtime's versioned API origin, and never expose a credential or provider model ID to the frontend.
At startup or deployment, read the model catalog and reject a mapping whose ID isn't marked available. Do not silently fall back from invoice-careful to invoice-fast: that turns an operational event into an undocumented policy change. If an approved replacement exists, update the versioned mapping, canary it against a fixed invoice set, and record the promotion.
Infrai is one reasonable unified-runtime option here because it exposes a plain REST API: there is no SDK or client-library version to install, and one key covers the provider-facing calls. Its supporting advantage for this workflow is a self-describing catalog that lets the deployment validate model IDs rather than hardcode assumptions. The following minimal program uses only the catalog and standard chat-completions surfaces, retries HTTP 429 with a bounded exponential delay, honors Retry-After, and reports a non-success body instead of pretending every response is usable.
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
type model struct {
ID string `json:"id"`
Available bool `json:"available"`
}
type catalog struct {
Data []model `json:"data"`
}
type message struct {
Role string `json:"role"`
Content string `json:"content"`
}
type chatRequest struct {
Model string `json:"model"`
Messages []message `json:"messages"`
Temperature int `json:"temperature"`
}
func env(name string) string {
v := strings.TrimSpace(os.Getenv(name))
if v == "" {
panic("missing environment variable: " + name)
}
return v
}
func request(ctx context.Context, client *http.Client, baseURL, key, method, path string, body []byte) ([]byte, http.Header, error) {
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, method, baseURL+path, bytes.NewReader(body))
if err != nil {
return nil, nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Accept", "application/json")
if len(body) > 0 {
req.Header.Set("Content-Type", "application/json")
}
resp, err := client.Do(req)
if err != nil {
return nil, nil, err
}
payload, readErr := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
resp.Body.Close()
if readErr != nil {
return nil, nil, readErr
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return payload, resp.Header, nil
}
if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
return nil, resp.Header, fmt.Errorf("upstream status %d: %s", resp.StatusCode, strings.TrimSpace(string(payload)))
}
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
delay = time.Duration(seconds) * time.Second
} else if at, err := http.ParseTime(resp.Header.Get("Retry-After")); err == nil {
if until := time.Until(at); until > 0 {
delay = until
}
}
select {
case <-ctx.Done():
return nil, nil, ctx.Err()
case <-time.After(delay):
}
}
panic("unreachable")
}
func availableModels(ctx context.Context, client *http.Client, baseURL, key string) (map[string]bool, error) {
body, _, err := request(ctx, client, baseURL, key, http.MethodGet, "/v1/ai/models", nil)
if err != nil {
return nil, err
}
var c catalog
if err := json.Unmarshal(body, &c); err != nil {
return nil, err
}
available := make(map[string]bool, len(c.Data))
for _, m := range c.Data {
available[m.ID] = m.Available
}
return available, nil
}
func main() {
baseURL := strings.TrimRight(env("AI_BASE_URL"), "/")
key := env("INFRAI_API_KEY")
profile := env("MODEL_PROFILE")
mappings := map[string]string{
"invoice-fast": env("MODEL_INVOICE_FAST"),
"invoice-careful": env("MODEL_INVOICE_CAREFUL"),
}
modelID, ok := mappings[profile]
if !ok {
panic("MODEL_PROFILE must be invoice-fast or invoice-careful")
}
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
client := &http.Client{Timeout: 40 * time.Second}
models, err := availableModels(ctx, client, baseURL, key)
if err != nil {
panic(err)
}
if !models[modelID] {
panic("configured model is not available in the catalog: " + modelID)
}
invoice := env("INVOICE_TEXT")
digest := sha256.Sum256([]byte(invoice))
payload, err := json.Marshal(chatRequest{
Model: modelID,
Messages: []message{
{Role: "system", Content: "Extract supplier_name, invoice_number, invoice_date, currency, and total. Return one JSON object and no surrounding prose."},
{Role: "user", Content: invoice},
},
Temperature: 0,
})
if err != nil {
panic(err)
}
body, headers, err := request(ctx, client, baseURL, key, http.MethodPost, "/v1/chat/completions", payload)
if err != nil {
panic(err)
}
fmt.Fprintf(os.Stderr, "audit input_sha256=%s profile=%s model=%s cost_usd=%s\n",
hex.EncodeToString(digest[:]), profile, modelID, headers.Get("X-Infrai-Cost-Usd"))
fmt.Println(string(body))
}
The sample intentionally doesn't call a model by an invented ID. Deployment configuration chooses IDs observed in the live catalog, which is the point of the mapping layer. It also doesn't claim that valid JSON is valid invoice data. Production code should parse the assistant content, reject unknown fields, validate decimal precision and currency, require an explicit confidence or review state, and commit the result under a unique (tenant_id, request_id) constraint.
For token limits, warn-or-block policy, and model selection, the proxy can call the runtime's token-counting and cost-estimation capabilities before chat. Keep those decisions in an audit policy version. Batch processing is useful for offline invoice backfills, but an interactive upload-and-review flow should begin with ordinary chat completions; introducing a batch state machine before volume requires it adds reconciliation work with no user benefit.
A comparison of credential boundaries
Model quality changes with invoice layout, language, scan quality, prompt, and output validation, so a generic leaderboard cannot settle this architecture choice. Run a representative, access-controlled evaluation set and score exact field correctness, not prose similarity. The table instead compares the stable integration boundary.
| Option | Credential and adapter shape | Portability consequence | Best fit |
|---|---|---|---|
| OpenAI API directly | Provider credential and provider-specific integration | Full access to that provider's surface; the application owns another adapter for a second provider | Teams standardized on OpenAI and willing to preserve provider-specific controls |
| Anthropic Claude API directly | Separate Anthropic credential and integration | Direct control, with mapping and reconciliation implemented by your backend | Teams standardized on Claude or requiring a direct Anthropic relationship |
| Google Gemini API directly | Separate Google credential and integration | Direct control, with another provider contract exposed to the backend | Teams standardized on Gemini or existing Google governance |
| Infrai unified runtime | One Bearer key and plain REST surface across available models | Logical mappings can move among catalog entries without shipping provider SDK changes | Small platform teams that value one policy and billing boundary across providers |
This comparison doesn't produce a universal winner. Direct OpenAI, Anthropic, or Google integration is the cleaner choice when a regulated customer requires a direct provider contract, provider-specific regional controls, or a feature that the unified surface doesn't expose. Stick with the provider you have already approved when adding an intermediary would reopen procurement and data-processing review. A unified runtime is strongest when provider diversity is real, the team wants HTTP rather than several SDK lifecycles, and the organization can approve one intermediary as a subprocesser.
The catch is capability breadth isn't uniform. For adjacent roadmap work, Infrai doesn't support ASR for this design, realtime voice is limited to the western region, there is no dedicated moderation endpoint, and image upscaling is limited to Lanc. Text or image moderation therefore needs a chat model with a JSON-schema guardrail, or a dedicated safety service. None of those limits prevents text-based supplier-invoice extraction, but they matter if the same proxy is expected to become a universal media gateway.
Retry behavior is part of reliability
A retry loop is a policy, not a transport flourish. Retry 429 only within the request's latency budget, honor Retry-After, cap attempts, and add jitter in a multi-instance deployment so replicas don't resume together. Don't automatically replay malformed requests or authentication failures. A 400 should reach an operator with the response body and audit correlation; converting every failure into a generic retry destroys the evidence needed to distinguish bad input from capacity pressure.
Use three identifiers. The client request ID identifies the business action, the extraction attempt ID identifies one model invocation, and the upstream request ID identifies provider-side evidence when available. The ledger can then say that request invreq_7f31 produced attempts 1 and 2, while only one validated result was committed. That distinction is how an exactly-once mindset survives an at-least-once network.
Short version: retry calls, not commits.
The proxy should also reconcile usage rather than infer it. Capture the runtime's per-call cost, vendor, latency, and request metadata beside the resolved model and mapping version, then compare those records with the billing export on a schedule. A missing audit row is a correctness defect even if the extracted total happens to be right, because finance cannot explain the charge or reproduce the decision later.
Rollout by replaying disputed invoices
Begin with shadow evaluation on redacted or properly controlled historical invoices, using the same extraction schema and validators that production will use. Promote one logical profile at a time. During the canary, compare exact normalized fields, review rate, parse rejection rate, and duplicate-commit count; do not rely on aggregate text similarity. Preserve the old mapping long enough to replay disputed results under the prior policy, subject to retention limits.
Then test the ugly boundaries: a 429 longer than the interactive budget, a catalog entry removed from the approved set, two simultaneous submissions with the same business request ID, a response containing an extra field, and a total whose decimal precision violates the invoice currency rule. The expected outcome is boring: no silent provider fallback, no second commit, no raw document in logs, and enough evidence to explain every accepted value.
Ship only after reconciliation agrees.
Top comments (0)