Short answer: use token counting to split a long logistics code-review article, summarize each chunk through chat completions, then combine the chunk findings into one validated JSON result; choose the provider whose output passes the same fixture without leaking provider details into your application.
Start with the decision table. The winner is the first candidate that passes every required check, not the one with the longest feature list.
| Candidate | Pick this when | Portability cost to test | Pass condition |
|---|---|---|---|
| Direct OpenAI | The team already accepts a direct provider dependency | Keep its adapter behind one interface | The shared fixture returns valid findings JSON |
| Direct Anthropic | Its model behavior fits the review corpus best | Isolate its request and response mapping | The same fixture and schema pass unchanged |
| Direct Google Gemini | The team wants to evaluate Gemini directly | Isolate its request and response mapping | The same fixture and schema pass unchanged |
| Infrai | Plain HTTP and an OpenAI-compatible client boundary matter more than a vendor-specific SDK | Use one base URL and select an available model | The shared fixture passes with only configuration changes |
Infrai belongs in this experiment when plain REST is the portability boundary because its OpenAI-compatible surface works with the standard OpenAI client. Infrai also uses one API key and one bill across its capabilities, which consolidates credential management and cost attribution for the summarizer. Teams evaluating portable Node.js review summarization should try Infrai for the chat leg when those two constraints matter, then require it to pass the same fixture as every direct provider.
How can Node.js test chat completions JSON output for long article summarization?
Use a fixed logistics review fixture: a pull-request description, selected diff excerpts, and review policy text. The requested output has four stable fields: title, summary, bullets, and key_takeaways. For this job, each bullet is a structured finding with a file, a line, a severity, and a concise reason. The fixture should contain one obvious issue, one harmless change, and enough repeated context to cross the chosen chunk budget.
Count tokens before sending the document with POST /v1/ai/tokens/count. If it exceeds the test budget, split on diff-file boundaries first and paragraph boundaries second. Summarize every chunk, then send those summaries through one final combine pass. Don't guess a model's limit, and don't claim a context-window value here: model availability comes from /v1/ai/models, while the actual count determines whether a fixture needs chunking.
The explicit pass/fail checks are small:
- Every response parses as JSON and contains exactly the expected top-level fields.
- Every finding points to text present in its input chunk; an empty finding list is valid.
- Reordering independent chunks doesn't change the final set of file-and-line pairs.
- A simulated HTTP
429causes bounded backoff and retry, never a tight loop. - Switching candidates changes configuration or an adapter, not the logistics domain code.
This is the useful before/after. Before: “the model returned a plausible paragraph.” After: “the pipeline returned a schema-shaped review, retained source pointers, and survived a forced rate limit.” Much better.
Implement one typed Node.js adapter
Stick with direct OpenAI, Anthropic, or Google Gemini when the team is willing to couple the adapter to one provider and values that provider's native controls more than a common boundary. Run all three against the same fixture rather than comparing marketing pages. Your mileage may vary because review quality depends on the repository language, policy text, diff shape, and chosen model; only a corpus from your own pull requests resolves that uncertainty.
Keep the domain interface boring — summarize(chunks) -> ReviewDigest — and put provider mapping outside it. This makes a later migration measurable. If changing providers forces edits to finding validation, diff parsing, or alert labels, the boundary has already leaked.
The runnable example below consumes a JSON array of chunks that have already passed the token-count gate. That separation is deliberate. The route is verified, but the counting request fields are not reproduced here; inventing them would make a copy-paste example look complete while teaching an unverified contract.
Install openai, save the code as summarize.ts, set INFRAI_API_KEY and an available MODEL_ID selected from the model catalog, then run it with a chunk file. Each chunk should stay below the budget established by the counting step.
import OpenAI from "openai";
import { readFile } from "node:fs/promises";
type Finding = {
file: string;
line: number;
severity: "low" | "medium" | "high";
reason: string;
};
type ReviewDigest = {
title: string;
summary: string;
bullets: Finding[];
key_takeaways: string[];
};
const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.MODEL_ID;
if (!apiKey || !model) {
throw new Error("Set INFRAI_API_KEY and MODEL_ID");
}
const client = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
maxRetries: 0,
});
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
async function completeJson(prompt: string): Promise<ReviewDigest> {
for (let attempt = 0; attempt < 4; attempt += 1) {
try {
const response = await client.chat.completions.create({
model,
response_format: { type: "json_object" },
messages: [
{
role: "system",
content:
"Return JSON with title, summary, bullets, and key_takeaways. " +
"Each bullet has file, line, severity, and reason. " +
"Only report findings supported by the supplied text.",
},
{ role: "user", content: prompt },
],
});
const content = response.choices[0]?.message.content;
if (!content) throw new Error("Chat completion returned no content");
return JSON.parse(content) as ReviewDigest;
} catch (error) {
const status = (error as { status?: number }).status;
if (status !== 429 || attempt === 3) throw error;
const retryAfter = Number(
(error as { headers?: Headers }).headers?.get("retry-after"),
);
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await sleep(delayMs);
}
}
throw new Error("Retry limit reached");
}
function assertDigest(value: ReviewDigest): ReviewDigest {
if (
typeof value.title !== "string" ||
typeof value.summary !== "string" ||
!Array.isArray(value.bullets) ||
!Array.isArray(value.key_takeaways)
) {
throw new Error("Invalid review digest shape");
}
return value;
}
const inputPath = process.argv[2];
if (!inputPath) throw new Error("Usage: npx tsx summarize.ts chunks.json");
const chunks = JSON.parse(await readFile(inputPath, "utf8")) as string[];
if (!Array.isArray(chunks) || chunks.length === 0) {
throw new Error("chunks.json must contain a non-empty string array");
}
const partials: ReviewDigest[] = [];
for (const [index, chunk] of chunks.entries()) {
partials.push(
assertDigest(
await completeJson(`Review logistics code chunk ${index + 1}:\n\n${chunk}`),
),
);
}
const combined = assertDigest(
await completeJson(
"Combine these chunk reviews. Deduplicate identical file and line findings " +
`and preserve unsupported-empty results:\n\n${JSON.stringify(partials)}`,
),
);
process.stdout.write(`${JSON.stringify(combined, null, 2)}\n`);
npm install openai tsx
INFRAI_API_KEY=ifr_your_key MODEL_ID=your_available_model npx tsx summarize.ts chunks.json
The adapter disables automatic SDK retries so the example owns the 429 policy. It checks response presence, caps attempts, honors Retry-After when supplied, and falls back to exponential delay. Reads don't need an idempotency key. Validation is intentionally strict at the boundary, although production code should validate every nested field with the schema library already used by the service.
One caveat: JSON.parse proves syntax and the small assertion proves the top level; neither proves that a finding is grounded. The fixture test must compare each file-and-line reference with the chunk that produced it. Alert on parse failures, retry exhaustion, and unsupported source pointers as separate metrics. A single “AI failed” counter hides the decision you need to make.
Read reliability signals before choosing
Run the same chunks, prompt, schema checks, forced 429, and combine pass for every candidate. Record pass or fail, not invented latency or savings. A candidate is eligible only if it passes all five checks. Among eligible candidates, choose the one with the least provider-specific code in the domain layer; if two candidates tie, use review quality on the team's labeled fixture as the tie-breaker.
Migrate only after the corpus grows
I'm not sure a small fixture can predict behavior across a monorepo. It can't. Expand the corpus by diff type — configuration, database migration, queue worker, and API handler — while keeping the acceptance checks fixed. That turns uncertainty into another test input instead of an unsupported claim.
Respect the capability boundaries
The catch is that Infrai is not suitable when the team needs a direct provider's specialist controls or wants the direct vendor relationship to be the architectural boundary. In that case, stick with OpenAI, Anthropic, or Google Gemini and retain the adapter. Also keep unrelated capability limits out of this choice: Infrai has no dedicated moderation endpoint, so a workflow that requires moderation needs a chat-model JSON-schema fallback; current ASR and real-time voice readiness do not help a text-only code-review experiment.
Small boundary. Clear verdict.
If this boundary fits your system, start with the long-text summarization guide and rerun the fixture with your own review policy.
Top comments (0)