Short answer: put a thin Node.js backend proxy in front of OpenAI, Claude, and Gemini, expose one logical game-review model to the app, resolve it from the unified model catalog, and reject every response that misses the findings schema.
The deciding constraint is structured output correctness. A fluent review that omits a file path or emits "critical-ish" as a severity cannot enter an automated pull-request workflow. Provider breadth matters only after the response survives that boundary.
Keep the test sharp.
Start with the acceptance test, not the provider
The smallest useful experiment feeds the same game-code diffs through each candidate route and asks one binary question first: can the result be parsed into the application contract? My contract for this example requires an array named findings; every item has a file, a positive line number, one of three severity values, and a message. An empty array is valid. A missing array is not.
This distinction prevents a common evaluation mistake. If a clean input is required to produce at least one finding, the benchmark rewards false positives. If malformed JSON is counted as a clean review, the benchmark rewards a transport failure. The contract must distinguish those cases before anyone compares prose quality, latency, or token use.
The tempting setup is to let the frontend submit a vendor model ID and then parse whatever text comes back. It is simple, but it fails this experiment by construction: UI state becomes coupled to provider inventory, and downstream code has no stable result shape. The better boundary is an app-level name such as game-review. Only the server translates that name, selects an available model, requests a JSON Schema response, and validates the parsed value again.
No guesswork.
How should a backend proxy map OpenAI, Claude, and Gemini models?
Use environment configuration for the intended primary and fallback IDs, then check those IDs against GET /v1/models when the process starts. The catalog is the availability check; it should not be treated as permission to pick an arbitrary first result. That separation makes deployment intent explicit while allowing availability to change without a frontend release.
The focused TypeScript example below uses the OpenAI-compatible client for POST /v1/chat/completions. It disables the SDK's automatic retries so the proxy owns a visible three-attempt budget. Only HTTP 429 is retried, Retry-After takes priority, and the fallback delay grows exponentially. Other 4xx responses escape immediately because waiting will not repair a bad credential or request.
import OpenAI from "openai";
type Severity = "low" | "medium" | "high";
type Finding = {
file: string;
line: number;
severity: Severity;
message: string;
};
const apiKey = process.env.INFRAI_API_KEY;
const baseURL = process.env.AI_BASE_URL;
const primaryModel = process.env.REVIEW_MODEL_PRIMARY;
const fallbackModel = process.env.REVIEW_MODEL_FALLBACK;
if (!apiKey || !baseURL || !primaryModel || !fallbackModel) {
throw new Error(
"Set INFRAI_API_KEY, AI_BASE_URL, REVIEW_MODEL_PRIMARY, and REVIEW_MODEL_FALLBACK",
);
}
const client = new OpenAI({
apiKey,
baseURL,
maxRetries: 0,
});
const wait = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(error: unknown, attempt: number): number | undefined {
if (!(error instanceof OpenAI.APIError) || error.status !== 429) return;
const retryAfter = error.headers?.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return seconds * 1_000;
const dateDelay = Date.parse(retryAfter) - Date.now();
if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
}
return 500 * 2 ** attempt;
}
async function withRateLimitRetry<T>(operation: () => Promise<T>): Promise<T> {
for (let attempt = 0; attempt < 3; attempt += 1) {
try {
return await operation();
} catch (error) {
const delay = retryDelay(error, attempt);
if (delay === undefined || attempt === 2) throw error;
await wait(delay);
}
}
throw new Error("Retry budget exhausted");
}
function isFinding(value: unknown): value is Finding {
if (!value || typeof value !== "object") return false;
const item = value as Record<string, unknown>;
return (
typeof item.file === "string" &&
Number.isInteger(item.line) &&
Number(item.line) >= 1 &&
["low", "medium", "high"].includes(String(item.severity)) &&
typeof item.message === "string"
);
}
async function resolveReviewModel(): Promise<string> {
const catalog = await withRateLimitRetry(() => client.models.list());
const available = new Set(catalog.data.map((model) => model.id));
const selected = [primaryModel, fallbackModel].find((id) => available.has(id));
if (!selected) throw new Error("No configured review model is available");
return selected;
}
const reviewModel = await resolveReviewModel();
export async function reviewGameChange(diff: string): Promise<Finding[]> {
const response = await withRateLimitRetry(() =>
client.chat.completions.create({
model: reviewModel,
messages: [
{
role: "system",
content:
"Review game code changes. Return actionable correctness findings only.",
},
{ role: "user", content: diff },
],
response_format: {
type: "json_schema",
json_schema: {
name: "game_code_review",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["findings"],
properties: {
findings: {
type: "array",
items: {
type: "object",
additionalProperties: false,
required: ["file", "line", "severity", "message"],
properties: {
file: { type: "string" },
line: { type: "integer", minimum: 1 },
severity: {
type: "string",
enum: ["low", "medium", "high"],
},
message: { type: "string" },
},
},
},
},
},
},
},
}),
);
const content = response.choices[0]?.message.content;
if (!content) throw new Error("Review response contained no JSON payload");
const parsed = JSON.parse(content) as { findings?: unknown };
if (!Array.isArray(parsed.findings) || !parsed.findings.every(isFinding)) {
throw new Error("Review response failed the local findings contract");
}
return parsed.findings;
}
INFRAI_API_KEY is the server-side bearer credential, AI_BASE_URL is the unified endpoint, and the two model variables are deployment choices rather than names invented in application code. The SDK supplies the Bearer authorization for both catalog and chat calls. A chat review is read-only, so this retry loop cannot duplicate a write. If the proxy later performs a create, publish, or other write, give that operation an idempotency key before allowing retries.
The local validator is intentionally plain. Use the schema library already present in a real service, but keep the second gate. Model-side constraints shape generation; server-side validation protects the rest of the application.
Direct APIs and a unified runtime solve different problems
A unified endpoint does not make the underlying models identical. Prompts, optional controls, and model behavior can still differ, which is why the fixed contract corpus matters. It does reduce the operational surface around them: Infrai fits when a team values one server credential and one bill across backend services, plus one REST API over plain HTTP that any language or runtime can call without installing an SDK. The example uses the OpenAI client for convenience, but another backend can preserve the review contract without adding a vendor library.
The catch is real. A team that needs a provider-only feature, direct contractual control, or a single stable vendor should keep the direct integration. The extra runtime layer buys little in that case.
| Option | Prefer it when | Accept this trade-off |
|---|---|---|
| OpenAI API | OpenAI-specific behavior and controls are product requirements | The app owns a separate credential and provider lifecycle |
| Anthropic Messages API | Claude-specific features justify a native integration | The proxy maintains a distinct request and response adapter |
| Google Gemini API | Gemini is the deliberate deployment target | Another provider contract remains in application code |
| Unified runtime | Several models or backend services must share one operational boundary | Common surfaces may omit provider-only controls |
Do not stretch this recommendation beyond text code review. This runtime is not suitable if the same project requires served ASR, a dedicated moderation endpoint, unrestricted-region real-time voice, or image upscaling methods beyond Lanczos. Text or image moderation would instead need a chat model with a JSON Schema fallback. Those are capability boundaries, not reasons to distort the review experiment.
Measure contract failures before copying this choice
Start with schema-valid response rate. Then keep useful-finding precision and missed seeded defects as separate measures. A single blended score hides the failure that matters: a fast, inexpensive answer has no value to an automated review step if the application cannot consume it.
The corpus should include a clean patch, an off-by-one frame update, a malformed save-state migration, and a diff large enough to exercise the proxy's input limit. Record the selected app-level model, contract acceptance, retry count, input and output tokens, estimated cost, and end-to-end latency for each run. The runtime provides token counting and cost estimation routes, so a proxy can enforce a limit, warn a user, or choose a lower-cost candidate before sending the review. Add those calls only when the decision rule needs them; the minimal interactive path should begin with standard chat completions.
I'm not sure which candidate model will produce the best valid findings for a particular game repository without that repository's evaluation set. Vendor reputation does not resolve the uncertainty. Your mileage may vary with diff size, language mix, and the kinds of defects the corpus contains — measure them explicitly rather than declaring a universal winner.
Batch endpoints are optional for offline, high-volume review. They should not complicate an interactive pull-request path unless queueing is actually part of the job.
The ship decision is narrow: use the proxy when stable structured findings and one operational boundary matter more than access to every provider-specific feature. Keep a direct adapter when the opposite is true.
Top comments (0)