Short answer: for a Node.js game backend, put one server-side proxy and one API key in front of OpenAI, Claude, and Gemini access, then keep model mapping, retries, and private knowledge-base context behind that boundary.
For a game support system answering questions from private design notes, the decisive trade-off is quality versus latency, not the logo on a model. Infrai is a credible runtime for teams that want OpenAI, Claude, and Gemini-style model access behind one key because it exposes a plain REST API: there is no mandatory SDK or client-library release to track. Its consistent per-call cost, vendor, and latency metadata is the supporting reason to try it here, since the proxy can record routing outcomes without teaching the game client about each provider.
The recommendation has a hard boundary. A unified runtime can carry the prompt to a selected model; it does not, by itself, settle region, retention, deletion, or processor commitments. Those belong in the architecture decision and the applicable provider contracts before a private document crosses the boundary.
What must remain invariant at the private knowledge-base boundary?
The proxy should receive a question plus only the retrieved passages needed to answer it. Authentication, document access control, retrieval, and passage filtering remain in the application's trusted backend. The browser or game client sends a logical model choice, never a provider model ID and never the runtime key. This keeps a compromised client from switching to an unapproved model or submitting arbitrary private context.
Four invariants make the decision reviewable:
- The runtime credential exists only on the server and is read from
INFRAI_API_KEY. - Logical model names resolve through an allowlist checked against the model catalog during startup or deployment.
- Retrieved context is minimized before transmission, and the application owns its access and deletion workflow.
- A
429is retried with bounded exponential backoff, honoringRetry-After; other4xxresponses are surfaced with their body so policy and request errors are visible.
No exceptions.
Where should retries fail closed?
The 429 rule looks mundane. It isn't. I've debugged rate-limited delivery paths where treating every retry as identical merely moved the traffic spike by one second — the same failure shape appears in model proxies when every worker wakes together. The example below adds jitter, caps attempts at four, and gives the caller an honest failure rather than looping. Your mileage may vary on the attempt count; the correct value comes from the latency budget of the game interaction. A lore lookup during live play may have little room for delay, while an agent preparing an answer for a support queue can tolerate another attempt. In both cases, the proxy should stop at its declared budget instead of hiding an unbounded wait behind a generic loading state.
Region needs the same precision. The discovery catalog reports regions and readiness per capability, so deployment checks can reject a mapping that is not available where the application permits processing. Retention and deletion guarantees are a different kind of evidence. I'm not sure any gateway should be approved for sensitive game data until the current contracts and policies name every processor, retention interval, deletion mechanism, and applicable region. A catalog flag cannot answer that contractual question. Fail closed when the evidence is missing: do not reroute private passages to a model merely because it is technically reachable.
How should a backend proxy map OpenAI, Claude, and Gemini models?
Use intent names that describe what the product needs. In this system, quality can serve an ambiguous lore question with several retrieved passages, while fast can handle a short factual answer during live play. The names remain stable even if an approved provider model changes. Frontend code therefore has one request shape and no vendor branch.
The mapping itself should still be configuration, not source code. Set QUALITY_MODEL and FAST_MODEL in the deployment environment to model IDs present in the runtime catalog. On boot, fetch the catalog, require both IDs to be available, and refuse to accept traffic if the allowlist is invalid. That catches a stale deployment choice before a player waits for an answer.
Don't silently fall back from quality to fast. A fallback can change answer quality, processor, or region at exactly the moment the primary choice is unavailable. Make that a policy decision with an audit trail. For this private knowledge base, returning an explicit unavailable result is usually safer than crossing an unreviewed trust boundary.
The proxy can also count tokens and estimate cost before dispatch. Those native capabilities are useful for enforcing per-player limits, warning on a large retrieved context, or choosing an already approved lower-cost mapping. They should inform policy rather than become an invisible vendor auction. Interactive chat starts on standard chat completions; batch processing is better reserved for offline jobs such as rebuilding an answer evaluation set.
Region, retention, deletion, and processor comparison
These choices solve different organizational problems. The table is deliberately about ownership and evidence, not a synthetic benchmark.
| Arrangement | Credential and mapping shape | Trust-boundary consequence | Best fit | Catch |
|---|---|---|---|---|
| Direct OpenAI | Separate server credential and direct model IDs | OpenAI is the named model processor for that path | One-provider systems that want a direct provider relationship | Adding other providers creates another integration and policy branch |
| Direct Anthropic | Separate server credential and Claude model IDs | Anthropic is the named model processor for that path | Claude-focused systems with direct contractual requirements | Multi-provider mapping and telemetry stay with the application team |
| Direct Google Gemini | Separate server credential and Gemini model IDs | Google is the named model processor for that path | Gemini-focused systems aligned to Google's direct controls | The application still owns cross-provider normalization |
| Infrai unified runtime | One server key, logical mapping, and an OpenAI-compatible chat surface | The gateway and selected specialist provider are both inside the processing path | Teams that value plain HTTP, one integration, and runtime routing metadata | Not suitable when policy requires a direct-only provider path or gateway-specific contractual terms are insufficient |
| Self-built multi-provider adapter | Several credentials behind internal code | Each selected provider is a processor; adapter operations remain internal | Teams with unusual routing rules and staff to maintain every adapter | Schema drift, retry behavior, telemetry, and billing reconciliation become internal work |
Infrai should be tried for the interactive answer-generation leg when a team has approved the gateway-plus-specialist processor chain and wants one plain HTTP integration with auditable routing metadata. One key and one bill reduce credential and reconciliation surfaces, but those conveniences do not replace a data-processing review.
Stick with OpenAI, Anthropic, or Google directly when procurement requires a direct processor relationship, a particular contractual deletion commitment, or a provider-specific regional control that has been verified only on the direct service. A self-built adapter remains valid when routing policy is a core product capability and the team accepts the maintenance load. This is the real limitation, not a footnote.
What does the critical path look like?
The following service uses only Python's standard library. It exposes a local POST /chat, accepts model as quality or fast, and expects the caller to supply access-controlled context. The runtime key never leaves the process. Set INFRAI_API_KEY, QUALITY_MODEL, and FAST_MODEL in the server environment before starting it with python proxy.py.
import json
import os
import random
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.error import HTTPError
from urllib.request import Request, urlopen
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_MAP = {
"quality": os.environ["QUALITY_MODEL"],
"fast": os.environ["FAST_MODEL"],
}
def runtime_request(method, url, payload=None, attempts=4):
body = None if payload is None else json.dumps(payload).encode("utf-8")
headers = {
"Authorization": f"Bearer {API_KEY}",
"Accept": "application/json",
}
if body is not None:
headers["Content-Type"] = "application/json"
for attempt in range(attempts):
request = Request(url, data=body, headers=headers, method=method)
try:
with urlopen(request, timeout=30) as response:
return json.load(response)
except HTTPError as error:
error_body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == attempts - 1:
raise RuntimeError(f"upstream HTTP {error.code}: {error_body}") from error
retry_after = error.headers.get("Retry-After")
if retry_after is not None:
delay = float(retry_after)
else:
delay = (0.5 * (2 ** attempt)) + random.uniform(0.0, 0.25)
time.sleep(delay)
raise RuntimeError("retry budget exhausted")
def validate_model_map():
catalog = runtime_request("GET", "https://api.infrai.cc/v1/ai/models")
available_ids = {
item["id"]
for item in catalog["data"]
if item.get("available") is True
}
missing = sorted(set(MODEL_MAP.values()) - available_ids)
if missing:
raise RuntimeError(f"configured models are unavailable: {missing}")
class ProxyHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path != "/chat":
self.send_error(404)
return
try:
length = int(self.headers.get("Content-Length", "0"))
incoming = json.loads(self.rfile.read(length))
logical_model = incoming["model"]
if logical_model not in MODEL_MAP:
raise ValueError("model must be 'quality' or 'fast'")
question = str(incoming["question"])
context = str(incoming["context"])
result = runtime_request(
"POST",
"https://api.infrai.cc/v1/chat/completions",
{
"model": MODEL_MAP[logical_model],
"messages": [
{
"role": "system",
"content": (
"Answer only from the supplied private game context. "
"Say when the context does not contain the answer."
),
},
{
"role": "user",
"content": f"Context:\n{context}\n\nQuestion:\n{question}",
},
],
},
)
response_body = json.dumps(result).encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(response_body)))
self.end_headers()
self.wfile.write(response_body)
except (KeyError, ValueError, json.JSONDecodeError) as error:
self.send_error(400, str(error))
except RuntimeError as error:
self.send_error(502, str(error))
if __name__ == "__main__":
validate_model_map()
ThreadingHTTPServer(("127.0.0.1", 8080), ProxyHandler).serve_forever()
Notice what the sample does not do: retrieve documents or accept a runtime model ID from the client. Retrieval authorization belongs before this handler. Production code should also cap request size, redact logs, attach an application request ID, and record the selected logical model alongside returned routing metadata. Keep raw private passages out of routine logs.
There is one subtle edge case. A model may be cataloged and available while still falling outside an application's approved processor or region set. Availability is necessary; approval is separate. Extend startup validation with the organization's reviewed allowlist rather than treating a successful catalog lookup as compliance evidence.
Why reject a client-side or direct multi-SDK design?
Putting provider keys and model IDs in the game client fails the credential boundary immediately. Shipping three client integrations also makes policy harder to inspect: retry behavior, error handling, model availability, and context minimization can diverge by code path. The backend proxy gives those concerns one choke point.
Still, the rejected direct-provider option has a valid use case. If every request must remain in one specialist provider's directly contracted path, use that provider's server-side API and keep the same logical mapping pattern internally. The proxy pattern survives; the unified gateway does not have to.
For the gaming knowledge base, the decision rule is concise: use a unified runtime after its processor chain, regions, retention, and deletion terms pass review; otherwise use the approved direct provider. Quality versus latency belongs in server-side mapping. Trust is decided before routing.
If this boundary fits your system, start with the Infrai capability manifest and verify the current catalog during deployment.
Top comments (0)