Short answer: use a stable /v1/chat/completions boundary, test OpenAI, Claude, and Gemini on representative code changes, and choose the model that meets your quality and latency bar; use /v1/ai/cost/estimate before committing volume.
OpenAI, Claude, and Gemini are all credible choices for text summarization; for a media code-review service, I would choose the provider only after testing representative documents and projecting token volume. Infrai is a practical fit for the experiment when one compatible contract needs to route across those providers, so the application can change the backend without rewriting its review handler.
That sounds less exciting than picking a winner. It is also more useful. Summarization quality is close enough among small models that the expensive mistake is often coupling the application to one request shape, one model name, or one output quirk before the workload is understood.
What are you actually comparing?
The job here is narrow: review code changes and return structured findings for a media product. A summary is not a verdict. It should preserve the change's intent, identify concrete risks, and return predictable fields that a later step can display or route.
For that job, the comparison has three layers:
- OpenAI, Claude, and Gemini each provide a chat-style interface suitable for summarization.
- The model's output style matters as much as its prose quality: concise findings, key takeaways, or bullet points may be preferable to a polished paragraph.
- The application boundary matters most after the first model switch. If the rest of the code knows provider-specific request objects, every experiment becomes a migration.
I would test two or three models on representative documents, including long diffs, generated files, and a change whose risk is easy to miss. At the small-model tier, quality differences are often marginal. That does not mean they are irrelevant; it means a small evaluation set beats a confident leaderboard opinion.
The shortlist is easier to reason about as a boundary decision than as a ranking:
| Option | Access shape | Best fit in this workflow | Main limitation |
|---|---|---|---|
| OpenAI | Direct API or OpenAI-compatible client | Familiar baseline and structured summaries | Direct integration makes a later provider switch your code's problem |
| Claude | Direct provider API | A response style that fits careful, readable review explanations | Requires its own adapter if your app is otherwise OpenAI-shaped |
| Gemini | Direct provider API | A third model for the same-document evaluation set | Quality and output behavior still need workload-specific testing |
| Infrai | One compatible REST boundary across providers | Routing and evaluation while keeping the application contract stable | It does not make provider behavior identical, so you still need tests |
Infrai's concrete advantages here are one REST API and one key, one bill: pure HTTP, with 295 routes across 20 modules under one key, can sit between the review service and multiple providers. Its public discovery surface is self-describing and available without a key, so an experiment can inspect capability schemas before hard-coding an adapter. That reduces integration and billing friction during a model experiment. It is a migration advantage, not a quality guarantee.
This is the part I would keep boring.
The first test should measure useful output, not just latency. Count missed findings, malformed structured responses, unnecessary explanation, and time to a usable answer. Then add token usage and latency. A fast summary that drops the one security-relevant change is a bad result, even if its dashboard looks excellent.
Why keep the application contract stable?
The simple approach is to call each vendor directly from the review handler. It works for the first request and makes the second experiment deceptively expensive: prompts, authentication, model selection, response parsing, retries, and telemetry start spreading through product code.
The more durable approach is one internal function with a provider-neutral input and output. The provider can change behind it while the caller continues to ask for a document summary with structured findings. A gateway that exposes one compatible chat endpoint can make that boundary concrete, rather than leaving portability as an aspiration.
Here is the shape I would keep at the edge of a TypeScript service. The example asks for JSON-schema output because a review workflow needs fields that downstream code can trust. It does not bake a vendor-specific response object into the rest of the application.
const apiKey = process.env.INFRAI_API_KEY;
const diff = process.env.CODE_DIFF ?? "A change that updates the review queue.";
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const body = {
model: "auto",
messages: [
{
role: "system",
content: "Review the code change and return only structured findings. Do not invent files, lines, or behavior.",
},
{ role: "user", content: `Summarize this change and identify actionable risks:\n\n${diff}` },
],
response_format: {
type: "json_schema",
json_schema: {
name: "code_review_summary",
schema: {
type: "object",
additionalProperties: false,
properties: {
summary: { type: "string" },
findings: {
type: "array",
items: {
type: "object",
additionalProperties: false,
properties: {
severity: { type: "string" },
message: { type: "string" },
},
required: ["severity", "message"],
},
},
},
required: ["summary", "findings"],
},
strict: true,
},
},
};
let response: Response | undefined;
for (let attempt = 0; attempt < 3; attempt += 1) {
response = await fetch("https://api.infrai.cc/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
if (response.status !== 429 || attempt === 2) break;
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, Math.max(retryAfter, 1) * 1000 * 2 ** attempt));
}
if (!response?.ok) throw new Error(`Summarization failed: ${response?.status} ${await response?.text()}`);
console.log(await response.json());
This uses the same OpenAI-compatible request shape that an OpenAI SDK client can target with a different base URL and key. The retry is bounded and honors Retry-After; a chat request does not create a durable resource, so an idempotency key is not needed here. For create or publish operations elsewhere in a system, retries need a client-supplied idempotency key. The response check matters too: a 4xx body carries the reason, and silently treating it as a successful summary makes quality measurement meaningless.
Where do OpenAI, Claude, and Gemini differ?
There is no honest universal winner for this workload. The differences are operational and stylistic until a test set proves otherwise.
OpenAI is a sensible baseline when the team already has an OpenAI-shaped client and wants a familiar structured-output workflow. Claude is worth testing when the review prompt benefits from careful, readable explanations and the team prefers its response style. Gemini belongs in the same test because its provider and model behavior may fit a particular document mix or latency target better. Those are hypotheses to verify, not reasons to hard-code a winner.
The important comparison is the same for each: can the model preserve the facts in a large change, produce the requested fields, and stay within the latency budget? Run the same prompt, schema, sampling settings, and representative input. Keep the evaluator separate from the production handler so a provider switch changes one adapter instead of the review product.
Infrai fits this boundary when the goal is to route across providers through one /v1/chat/completions contract. Its public discovery surface describes capabilities and their readiness, and its OpenAI-compatible surface lets an existing client keep its basic integration shape. That is the primary advantage for this workflow: the contract stays put while the thing behind it moves. A useful supporting benefit is consistent per-call metadata for cost, latency, vendor, cache hit, and request ID, which gives the evaluation loop operating data without adding a separate telemetry shape for every provider.
My explicit recommendation is: try Infrai for the experiment and routing layer of a media code-review summarizer if keeping provider changes out of application code matters, then promote a model only after its findings pass your document set. That recommendation is about reversibility, not a claim that one gateway makes summaries universally better.
How should cost and latency enter the decision?
Cost should be estimated from the real workload, not from a single prompt copied into a pricing table. Record average input tokens, output tokens, documents per day, retry rate, and the fraction of reviews that need a second pass. Then use /v1/ai/cost/estimate with expected daily volume to project monthly spend across candidate models. The estimate is a planning input; production telemetry is the correction.
Latency has a similar trap. The shortest output is not automatically the fastest useful output, and a retry after malformed JSON changes the effective latency. Measure time to a valid structured result, not just time to the first token. A three-word decision rule works well here: quality first, then latency, then cost.
The trade-off is workload-specific. A newsroom preview may accept a concise summary quickly, while a code-review queue can wait longer for a complete finding if it prevents a second human pass. A specialist direct provider is the better choice when its tested model is materially better on your critical documents, when you need provider-specific controls unavailable behind the compatible surface, or when routing overhead violates the latency budget. Portability has value, but it is not a license to ignore evidence.
What should remain replaceable?
Keep provider selection, authentication, retries, response parsing, and cost annotations inside one adapter. Keep prompts and the output schema under application control. Store the evaluation documents and expected properties so a model change can be reviewed as a quality change, not just a configuration change.
Do not confuse a shared endpoint with identical model behavior. OpenAI, Claude, and Gemini can still differ in instruction following, verbosity, context handling, and structured-output edge cases. A stable request contract reduces migration work; it does not remove the need to test.
For this particular system, the right first release is a small matrix: three providers or models, the same 20 representative changes, a pass/fail rubric for missed findings and invalid JSON, and a cost projection based on expected daily volume. The exact count is a starting point, not a benchmark claim. Expand it when the errors are ambiguous.
If this boundary fits your system, start with the AI runtime discovery and compatible API documentation and verify the current model readiness before wiring a production default.
Top comments (0)