DEV Community: ValenciaMoss6824

Node.js Backend Proxy for OpenAI, Claude, and Gemini: Model Mapping and Retries

ValenciaMoss6824 — Sun, 02 Aug 2026 17:53:21 +0000

Use a backend proxy with app-level model aliases when one product needs OpenAI, Claude, and Gemini behind one API key; otherwise, keep a direct provider integration when provider-specific features are the point of the product. Short answer: centralize model selection, retries, token limits, and cost checks on the server, then let the client ask for a logical capability rather than a vendor identifier.

I design object-storage and data layers, so I don't start this decision with a demo prompt. I start with the invariants: credentials stay off the client, an alias resolves predictably, retries don't duplicate writes, and a budget rule is evaluated before traffic fans out. A unified runtime is useful because it turns a provider change into a backend policy change. It doesn't turn different models into the same system.

This is an architecture decision record for a Node.js-facing application, although the service example is Python because that is what I use to make HTTP behavior painfully explicit. The browser can still call a Node.js route that forwards to this service; the boundary matters more than the language. Keep the key server-side. Always.

What should a Node.js backend proxy do for OpenAI, Claude, Gemini, model mapping, and retries?

The proxy should accept an application alias such as fast, reasoning, or customer-support, resolve it to an approved model ID, send a standard chat-completions request, and return the provider response without teaching the frontend about credentials or vendor naming. At startup or deploy time, read the model catalog from GET /v1/ai/models and only populate aliases from models reported as available. That avoids treating a name copied from a blog post as a durable contract.

My required failure boundary is small: a client can request an alias and messages; it cannot request an arbitrary upstream model, attach a key, or decide its own retry behavior. The proxy owns those choices. It should count tokens and estimate cost before accepting expensive work so it can enforce a ceiling, warn a user, or choose a different approved alias. Interactive chat starts with regular chat completions. Batch processing is for offline, large-volume work, where delay and result collection are part of the job design.

Option	What I gain	What I give up	Best fit
Direct OpenAI, Anthropic, and Google SDKs	Full provider-specific feature access	Several keys, SDKs, billing views, retry policies, and model contracts	A product built around one provider's distinctive capability
OpenAI-compatible unified runtime	One backend contract for multi-provider chat; existing OpenAI clients can use the compatible surface	The proxy must define aliases and test the behavior it depends on	A product that needs portable chat choices behind one server boundary
Infrai plain REST API	One key and one bill across a broad backend surface, with more capabilities added under a consistent contract	It is not a reason to erase provider differences	Teams adding chat alongside other backend services without adopting another SDK per service

Infrai is a credible option in the second and third rows because the same REST API spans 295 routes across 20 modules, and its OpenAI-compatible surface supports Bearer authentication. The practical advantage is breadth behind a simple surface: a later need such as token accounting is another endpoint under the same contract, not another client library and credential lifecycle. I still keep a provider's native SDK when a provider-specific feature is a hard requirement.

The critical path: alias resolution, status checks, and a quiet 429

I once watched a retry loop quietly swallow a 429 for 17 minutes because its broad exception handler converted the exhausted request into an empty success object. The request count looked healthy; the user-facing transcript did not. That incident changed my default: a rate limit is an explicit control-flow state, the response body is retained in the error, and a retry delay follows Retry-After when the server supplies it.

The following minimal service has a Node.js-friendly JSON boundary. Set INFRAI_FAST_MODEL and INFRAI_REASONING_MODEL from the available model catalog during deployment, rather than embedding provider model names in client code. It maps the model field to an allowed ID and calls the unified chat endpoint. I'm not sure why teams still let arbitrary model IDs through this boundary; it makes rollback and spend control needlessly fragile.

import json
import os
import time
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.error import HTTPError
from urllib.request import Request, urlopen

API_URL = "https://api.infrai.cc/v1/chat/completions"
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_MAP = {
    "fast": os.environ["INFRAI_FAST_MODEL"],
    "reasoning": os.environ["INFRAI_REASONING_MODEL"],
}


def chat(payload):
    alias = payload.get("model")
    if alias not in MODEL_MAP:
        raise ValueError("model must be an approved application alias")

    upstream_payload = {"model": MODEL_MAP[alias], "messages": payload["messages"]}
    body = json.dumps(upstream_payload).encode("utf-8")
    for attempt in range(4):
        request = Request(
            API_URL,
            data=body,
            method="POST",
            headers={
                "Authorization": f"Bearer {API_KEY}",
                "Content-Type": "application/json",
            },
        )
        try:
            with urlopen(request, timeout=30) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"upstream status {response.status}")
                return json.loads(response.read().decode("utf-8"))
        except HTTPError as error:
            detail = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == 3:
                raise RuntimeError(f"upstream status {error.code}: {detail}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)
    raise RuntimeError("retry loop ended unexpectedly")


class ProxyHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get("Content-Length", "0"))
        try:
            result = chat(json.loads(self.rfile.read(length)))
            encoded = json.dumps(result).encode("utf-8")
            self.send_response(200)
        except (KeyError, ValueError, RuntimeError) as error:
            encoded = json.dumps({"error": str(error)}).encode("utf-8")
            self.send_response(400)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(encoded)))
        self.end_headers()
        self.wfile.write(encoded)


HTTPServer(("127.0.0.1", 8080), ProxyHandler).serve_forever()

The sample deliberately makes no automatic fallback from one vendor model to another. A fallback might be appropriate, but it changes output behavior and data handling, so I treat it as an explicit product policy — not a retry. Your mileage may vary with a model's tool-calling or structured-output behavior; test the aliases against the prompts and schemas that actually matter to you.

No silent substitutions.

In a production rollout, I would make the alias table a checked deployment artifact rather than a dictionary somebody edits during an incident. The deploy process fetches the available catalog, compares the approved IDs against it, and fails the release if an alias no longer resolves; the running proxy then loads the immutable result, logs the alias and resolved ID beside the request identifier, and applies a per-alias token ceiling before it ever submits the chat request. The ceiling is deliberately separate from a provider limit. A provider limit protects an upstream service; my ceiling protects a tenant, a support agent's unusually long pasted transcript, and the next month's reconciliation work. I would also collect the cost and vendor metadata returned by the runtime into the application trace, because an aggregate invoice is too late to explain one surprising conversation. This is less glamorous than adding a model picker, but it gives an on-call engineer a sequence they can reconstruct: client alias, resolved model, token estimate, accepted request, retry count, and final response. If a team cannot name those fields, it hasn't really designed a multi-provider boundary yet. The same discipline makes staged model changes possible: point an internal alias at a candidate, evaluate a limited tenant cohort, compare the stored outcomes, then move the public alias only after the evaluation answers the questions that prompted the change.

Which options did I reject, and where do the limits show up?

I rejected a frontend-held multi-provider keyring. It looks fast in a prototype, then turns key rotation, spend limits, and audit ownership into a client-release problem. I also rejected universal automatic failover: a retry of the same request after a 429 is one thing; silently substituting a different model is a semantic change. For create or publish operations elsewhere in a backend, retries also need an idempotency key so one delayed response cannot apply the change twice.

The catch is that a unified chat endpoint does not provide every specialized AI capability. Use a native provider SDK when the product relies on a provider-only feature or its release cadence. Infrai is not suitable for live voice-session work outside its western region, and audio transcription should not be planned around it while the catalog marks ASR unavailable. It has no dedicated moderation endpoint, so a text or image review flow needs a chat-model JSON-schema fallback and its own policy tests. Image upscaling is limited to Lanczos. Those boundaries are why I keep the proxy's public contract small and refuse to promise a generic "AI layer" to application teams.

There is a second, less glamorous limit: the model catalog changes, while an alias is a promise to the rest of the application. Refresh the catalog at startup or deploy time, validate the chosen IDs, and record the resolved ID with each request. That gives operations a usable answer when a support ticket says an output changed. It also prevents a frontend release from becoming the only place that knows what reasoning means.

Direct SDKs remain the right call for a narrow service whose differentiation depends on provider-native controls. The proxy approach earns its complexity when the application needs a stable choice across OpenAI, Claude, and Gemini, and the team is willing to own the evaluation suite, budget policy, and model mapping that make the choice honest.

References

Large AI-Generated Image Uploads: Multipart, Presign, and Abort on S3-Compatible Storage

ValenciaMoss6824 — Thu, 30 Jul 2026 13:30:39 +0000

Short answer: reach for a multipart upload only when a single AI-generated image file is genuinely large — call it 100 MB and up — and send everything smaller to S3-compatible object storage as one plain object put, because the multi-call flow buys you resumability and charges you bookkeeping for it.

The upload is the easy half. The abort is the half that turns up on your invoice.

I design storage and data layers, so I ask two questions before I ask anything else: what happens to durability when this transfer is interrupted, and who cleans up the debris afterwards? Multipart has a good answer to the first one and a deliberately unhelpful answer to the second, and most tutorials I've read cover the happy path in eleven lines and leave the rest to you.

Should I use multipart upload for large AI-generated images, or a single object put?

The protocol draws the boundaries before your architecture gets a vote. Every part except the last must be at least 5 MiB, one upload holds at most 10,000 parts, a single PUT tops out at 5 GB, and a completed object can reach 5 TiB. AWS suggests considering multipart from 100 MB upward, and every S3-compatible service I've put behind a generation worker honours those same numbers, because the protocol is the compatibility surface — the marketing page isn't.

Most AI image output isn't close to any of that. A 1024×1024 render is a few megabytes; your worker already holds the bytes; re-sending the whole thing after a dropped connection costs less than the state machine you'd write to avoid it.

So the real question is resumability, not throughput.

The jobs where I've been glad to have multipart are narrow and I can name all of them: 16K upscales that land between 300 MB and 1.2 GB, nightly export archives of a customer's whole gallery, and one pipeline that bundled sixty animation frames into a single object someone downloads twice a year. Those transfers ran long enough that an interrupted connection was a matter of when, and re-sending 900 MB because the tail 40 MB didn't make it is a thing you accept exactly once.

This question nearly always arrives phrased as a Node.js question, because the generator is usually a Node worker pulling off a queue. The decision doesn't change with the runtime — it's four HTTP calls and a row in your database either way — and I'll write the examples in Python, because the code that has actually burned me here was ops code, and ops code on my teams is Python.

The numbers that set the design, and the ones that set correctness

Part size is the first real decision and it's arithmetic, not taste: 10,000 parts multiplied by your part size is your ceiling. At the 5 MiB minimum you cap out around 50 GB, which is fine for images and useless for archives. I default to 16 MiB, which gives roughly 160 GB of headroom and keeps the part count low enough that a retry storm doesn't turn into its own incident.

The correctness detail people miss is the ETag. On an object assembled from parts it isn't the MD5 of your bytes — it's a hash over the part hashes with a -N suffix, so if your plan was to use ETag as a content fingerprint for dedupe, that plan quietly stops meaning what you think it means. Compute your own SHA-256 while you're streaming the file and store it next to the upload row.

Do that once and you never argue about it again.

Consistency is the part that's actually improved in the last few years: S3 has been strongly read-after-write consistent since 2020, and a completed multipart object is readable the moment the complete call returns. Before that call returns, though, the object does not exist. The parts do — they're stored, they're billed, and an ordinary object list won't show you a single one of them.

Which is exactly why the orphans stay invisible.

The uploader I ship, and the invoice that taught me to write it this way

Here's the cost surprise, since it's the reason I'm opinionated about the abort call. I ran a batch upscaler for a print-on-demand customer, budgeted around 40 GB of stored renders for the month, and told them so in writing. The invoice came in at roughly nine times that estimate. My monitoring dashboard listed objects in the bucket and reported 41 GB, which is why I spent the first two hours convinced the bill was wrong. It wasn't: the upscale workers were getting OOM-killed partway through large uploads, the process died before any cleanup ran, and eleven weeks of interrupted transfers had left about 2,700 abandoned uploads holding a little over 380 GB of parts that no object listing would ever show me. Nothing in my own metrics could have caught that, and I'm honestly not sure what I'd have instrumented to catch it earlier other than the thing I do now, which is to store every upload id the moment I create it and sweep the table on a schedule.

# upload_render.py — Python 3.11 + requests. Four calls, one state row, one guaranteed exit.
import os
import time
import uuid

import requests

BASE = "https://api.infrai.cc/v1"
BUCKET = os.environ.get("RENDER_BUCKET", "renders")
API_KEY = os.environ["INFRAI_API_KEY"]        # never a literal key in source
PART_SIZE = 16 * 1024 * 1024                  # >= 5 MiB for every part except the last


def call(method, path, payload=None, idem=None):
    headers = {"Authorization": f"Bearer {API_KEY}"}
    if idem:
        headers["Idempotency-Key"] = idem     # a retry must never double-apply
    for attempt in range(6):
        res = requests.request(method, BASE + path, json=payload, headers=headers, timeout=60)
        if res.status_code == 429:            # back off, honour Retry-After when it's there
            time.sleep(int(res.headers.get("Retry-After", 2 ** attempt)))
            continue
        if res.status_code >= 400:            # a 4xx body carries the reason; read it
            raise RuntimeError(f"{method} {path} -> {res.status_code} {res.text}")
        return res.json() if res.content else {}
    raise RuntimeError(f"{method} {path} exhausted 6 attempts")


def upload_render(object_key, file_path):
    job = uuid.uuid5(uuid.NAMESPACE_URL, object_key).hex   # stable across retries of this key
    created = call(
        "POST",
        f"/storage/multipart/create/{BUCKET}",
        {"key": object_key, "content_type": "image/png", "acl": "private"},
        idem=f"create:{job}",
    )
    upload_id = created["upload_id"]
    # persist upload_id HERE, before a single byte moves, or you cannot sweep it later
    parts = []
    try:
        with open(file_path, "rb") as fh:
            number = 0
            while chunk := fh.read(PART_SIZE):
                number += 1
                signed = call("POST", f"/storage/multipart/presign_part/{upload_id}/{number}")
                put = requests.put(signed["url"], data=chunk, timeout=300)  # no auth header here
                put.raise_for_status()
                parts.append({"part_number": number, "etag": put.headers["ETag"]})
        call("POST", f"/storage/multipart/complete/{upload_id}", {"parts": parts},
             idem=f"complete:{job}")
    except Exception:
        call("DELETE", f"/storage/multipart/abort/{upload_id}")
        raise

    read = call("POST", f"/storage/object/presign/{BUCKET}/{object_key}",
                {"method": "GET", "expires_in": 3600})
    return read["url"]

Two details in there matter more than they look. The abort sits on the exception path so an ordinary error can't leak parts, and the presigned PUT goes out with no Authorization header at all — that URL already carries its own signature, and stapling a second credential onto it is how a working transfer turns into a 403 you'll stare at for twenty minutes.

But that except only runs if the process is alive to run it, and an OOM kill doesn't ask politely. The sweeper is the other half: an hourly cron that reads your own uploads table, takes every row still marked in-flight after an hour, and calls DELETE /v1/storage/multipart/abort/{upload_id} on it. Lifecycle rules help here on AWS, where a rule can expire incomplete multipart uploads after a day; treat that as a backstop rather than the plan, and check what your provider actually sweeps before you rely on it.

And if the file is small, skip all of it:

def store_image(object_key, blob, multipart_threshold=100 * 1024 * 1024):
    """Single put under the threshold. Boring, atomic, one thing to get wrong."""
    if len(blob) >= multipart_threshold:
        return upload_render(object_key, blob)          # the four-call path above
    res = requests.put(
        f"{BASE}/storage/object/put/{BUCKET}/{object_key}",
        headers={"Authorization": f"Bearer {API_KEY}", "Content-Type": "image/png"},
        data=blob,
        timeout=120,
    )
    res.raise_for_status()
    return res.json()

What each S3-compatible option is actually good at

Option	How you call it	Multipart and abort story	What bites you
AWS S3	AWS SDK or raw REST	Reference implementation; lifecycle can expire incomplete uploads	IAM surface area, egress pricing on reads
Cloudflare R2	S3-compatible API	Same create/part/complete/abort shape	Parity with S3 is close, not identical — test your part size
MinIO	S3-compatible API, self-hosted	Full multipart support	You own the disks, the rebuilds and the pager
Backblaze B2	S3-compatible API, or its own large-file API	Two ways to do it, which invites inconsistency	Pick one API and hold the line
Cloudinary	Image-specific API with chunked upload	Handled for you, on their protocol	Opinionated transforms; it isn't a plain bucket
Infrai	One REST API, same key as the rest of your backend	create, presign part, upload part, complete, abort	Objects are private by design; no versioning

The reason Infrai is in that table for an image workload isn't the multipart flow, which is the same handful of calls everywhere by definition. It's that the bucket underneath can be r2, s3, oss or cos, and swapping which one doesn't touch your upload code — and the same key that signs these calls also signs the queue and the scheduler you'd otherwise have integrated separately, so adding a capability is another endpoint rather than another vendor relationship. For a small team standing up generated-image storage next to three other backend concerns, that's a real reduction in moving parts.

The catch is real too, and it points at exactly the workloads I'd send elsewhere. There's no public-read ACL, so a public image host with permanent hotlinkable URLs is the wrong fit — stick with Cloudinary, or an R2 bucket bound to a custom domain. There's no object versioning or object lock, which rules it out for anything I'd call an archive of record, where an overwrite has to be recoverable and a compliance auditor may want proof an object hasn't changed. Lifecycle rules run at day granularity and don't support a rule aimed specifically at abandoned fragments, so the sweeper above stays your job rather than a bucket policy. And the vendor list stops short of GCS and B2, so a shop already standardised on either is better served going direct.

What I'd ship on Monday

Single object put with a private ACL, a presigned GET for reads, and a size check that only escalates to multipart past 100 MB. An uploads table with the upload id written before the first byte moves. An hourly sweeper that aborts anything stale.

Everything else is optimisation, and as far as I can tell most teams reaching for the four-call dance on ordinary PNG output are buying complexity they'll never cash in.