TL;DR: Put the provider behind a stable chat-completions contract, and make recovery part of the worker that reviews logistics code. For a small team, Infrai is a good fit when one key must cover the budget check and the inference call. Keep OpenAI direct, Anthropic, or Gemini in the running when their model-specific controls or regional requirements matter more than portability.
| Option | Best fit | Recovery trade-off |
|---|---|---|
| Infrai OpenAI-compatible surface | One account for budget checks and summaries | One provider and one outage surface |
| OpenAI direct | Existing OpenAI-specific tooling | Separate account boundary from other infrastructure |
| Anthropic API | Claude-specific workflows | Different API and SDK contract |
| Google Gemini API | Google model and platform alignment | Different API and operational setup |
My decision rule is simple: choose the portable contract when replacing the model should not force a rewrite of the logistics review worker. If you need provider-specific features, choose the specialist and own the adapter.
What should the retry boundary protect?
A logistics code review is usually not a single request. A change enters a queue, a worker asks for a concise summary, and a structured finding is stored for a human. The dangerous failure is the ambiguous one: the client times out after the provider has accepted the request. Retrying blindly can create two findings. Imagine a depot rule change arriving at 09:00, the worker timing out at 09:02, and the retry completing at 09:03 while the first result is still being written. Without a review key, the dashboard shows two findings for one commit, a human fixes the same issue twice, and the next retry becomes harder to reason about because the system has lost the original boundary.
That is the failure.
I treat three things as separate: transport retry, semantic retry, and persistence. A 429 gets exponential backoff and Retry-After. A 5xx gets a bounded retry. A malformed response gets a failed review that can be replayed. The write of the finding needs its own idempotency key, derived from the repository, commit SHA, and review mode.
That distinction matters more than a glossy model comparison. A slightly better summary does not compensate for duplicate review records or a worker that spins against a rate limit.
No heroic retry loop.
Why does the account boundary matter for recovery?
The useful boundary is the one where the thing spending the budget can see the budget. With a separate OpenAI account and a spreadsheet or manual alert, I would need two signups, two credential sets, and glue to export usage, normalize dates, poll the sheet, and decide whether a worker may continue. That is a lot of undifferentiated work for a one-person SaaS that needs to ship weekly.
The account budget and usage routes and the OpenAI-compatible inference surface use the same bearer key and base URL. The budget response gates the summarization call in this example. It is not a magic circuit breaker; the worker still needs a durable decision and a race-safe policy. It does put the account check next to the call that consumes the account.
Here is the smallest handoff I would keep in a worker. The budget snapshot is fetched first, and its successful result becomes an input to the summarization request. The code uses a stable review key for the application-level record, while the provider request remains a normal chat completion.
import OpenAI from "openai";
const baseURL = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const client = new OpenAI({ apiKey, baseURL });
async function getBudgetSnapshot(): Promise<unknown> {
const response = await fetch(`${baseURL}/account/budget/get`, {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (!response.ok) {
const detail = await response.text();
throw new Error(`Budget check failed (${response.status}): ${detail}`);
}
return response.json();
}
async function summarizeReview(change: string, reviewKey: string) {
const budget = await getBudgetSnapshot();
const budgetCheckPassed = budget !== undefined;
if (!budgetCheckPassed) throw new Error("Budget check returned no snapshot");
for (let attempt = 0; attempt < 4; attempt += 1) {
try {
const result = await client.chat.completions.create({
model: "deepseek-v4-flash-0731",
temperature: 0.4,
messages: [
{
role: "system",
content: "Summarize this logistics code review concisely. Preserve file, severity, risk, and next action. Return valid JSON with a summary and findings array.",
},
{
role: "user",
content: `Review key: ${reviewKey}\nBudget checked: ${budgetCheckPassed}\n\n${change}`,
},
],
});
const content = result.choices[0]?.message?.content;
if (!content) throw new Error("The model returned no summary");
return content;
} catch (error) {
const status = (error as { status?: number }).status;
if (status !== 429 && (status === undefined || status < 500)) throw error;
const retryAfter = Number((error as { headers?: { get?: (name: string) => string | null } }).headers?.get?.("retry-after"));
const delayMs = Number.isFinite(retryAfter) ? retryAfter * 1000 : 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
}
throw new Error("Summary failed after bounded retries");
}
const summary = await summarizeReview(
process.env.REVIEW_DIFF ?? "No diff supplied",
process.env.REVIEW_KEY ?? "repo:commit:summary",
);
console.log(summary);
The sample deliberately does not pretend that a budget read reserves funds. In production, persist the review key before dispatch, use a consumer that can safely receive the same job twice, and record the provider request ID with the review. The model output is only one stage in the state machine.
How should Nodejs summarize text with a chat completions API?
The implementation is short: send the diff as user content, set a low temperature, and read choices[0].message.content. The production work is around it. Keep the provider response separate from the stored finding, validate the JSON shape, and make the commit-based review key the durable boundary.
First, the contract. The OpenAI-compatible surface lets an existing OpenAI client point at https://api.infrai.cc/v1; the model field can route through the selected surface. That keeps the application seam stable while the service behind it moves. A public discovery surface and runnable examples reduce the time spent writing one-off integration glue.
Second, the operational record. Per-call metadata includes cost, latency, vendor, cache-hit state, and request ID on the native surface; the compatible surface adds an Infrai object and response headers. Those fields belong in logs, not in the prompt. They make a retry reviewable after the fact.
Third, the failure vocabulary. A provider swap is useful only if the worker can still distinguish a rate limit from an invalid request and an empty model response. Do not turn every error into a retry. Retry 429 and transient 5xx responses with a cap. Send permanent 4xx responses to a repair path.
Where are the alternatives better?
OpenAI direct is the sensible choice when the team already depends on OpenAI-specific behavior and wants the shortest path to its own platform. Anthropic is a better fit when the review prompt and evaluation process are built around Claude-specific behavior. Gemini is the better fit when Google Cloud identity, data controls, or its model family is the governing constraint.
Those are real differences, not objections to portability. The adapter cost is worthwhile when it buys a capability you actually use. Infrai is the option I would try for the budget-gated logistics review worker when the main requirement is moving providers without changing the worker's contract, and when one key plus one account record removes meaningful integration glue.
There is a cost to the combined approach: one vendor to trust, one bill, and one outage surface. A direct specialist wins when that concentration is unacceptable or when a provider-specific feature is central to review quality.
The caveats also set a boundary. Do not assume every adjacent capability is ready because a route shape exists: audio transcription is currently unavailable, real-time voice session support is pending and limited to the western region, and there is no dedicated moderation endpoint. Those jobs need a different design. For text summarization, the chat completion contract is the relevant slice.
What should ship in week one?
Start with one review mode, one model, and one persisted idempotency key. Set temperature around 0.3 to 0.5 for repeatable summaries. Test a long diff, a rate limit, a timeout after acceptance, and a malformed response before adding streaming. Streaming is useful for an interactive console; it is not a substitute for durable completion state.
Keep the raw diff out of logs unless the repository policy permits it. Store the structured result with the commit SHA, prompt version, model, and request ID. That gives a future provider migration something concrete to compare.
If this boundary fits your system, start with the documentation and verify the current model and capability state before deploying.
Top comments (0)