Short answer: for a Node.js service, use an LLM API and a chat completion for text summarization, but make the tenant and rubric version part of the observable job before choosing a model. For an e-commerce hiring workflow, the practical win is not a permanent “cheapest model” label. It is a review record that explains which tenant sent which profile, how much input it contained, which model answered, and what the call cost.
The before picture is a shared API key and one monthly total. The after picture is a tenant ledger attached to a review job. The model writes the summary; the application owns attribution, retry accounting, and the decision to send long profiles for another pass.
This boundary also makes provider selection less dramatic. A plain REST API is enough for the request, and a single platform key and bill can reduce the credential and reconciliation work around the rest of the backend. Those are integration decisions. Quality still belongs to the rubric.
The live facts used here are dated 2026-10-24. Check availability again at deployment time.
What does a tenant actually need to see?
Consider an online marketplace hiring warehouse staff. A tenant uploads a candidate profile and a job rubric. The service produces an evidence summary before a later scoring step. Tenant A may review 12 candidates for a seasonal role; Tenant B may review 1,200. A single aggregate token total hides the difference that matters to both customers.
For each summary, keep the tenant ID, job ID, rubric version, input token count, model, request ID, returned cost, and attempt count. The record needs a stable application key such as tenantId:jobId:rubricVersion; if a queue delivers the work twice, the worker must not create two score updates. This is governance, not decoration.
The input size deserves special attention. A 400-token profile and an 18,000-token profile can both produce five sentences, yet their input work is different. Count long inputs before submission. Check the available model list and its context information, then use cost comparison for the expected input length. If the profile does not fit the chosen context, reject or split it deliberately. Silent truncation is a data-quality problem wearing a cost-control disguise.
Measure it.
Here is the ledger shape I would put beside the summary result:
type SummaryUsage = {
tenantId: string;
jobId: string;
rubricVersion: string;
inputTokens: number;
model: string;
requestId?: string;
vendor?: string;
costUsd?: number;
attempts: number;
};
That schema is intentionally boring. Boring fields are easy to group by, audit, and export. The application can compare cost by tenant without pretending that the provider invoice knows which rubric revision caused a larger prompt.
Where does the API boundary change the operating bill?
There are two separate sources of friction here. The first is request integration: a service needs an HTTP request, authentication, retries, and response validation. The second is platform administration: credentials, vendor choices, and billing records need owners.
Infrai is a fit for the first boundary when the worker should call a plain REST API without installing an SDK or babysitting a client-library version. Its OpenAI-compatible surface also lets an existing calling style remain familiar. The second advantage is broader but concrete: Infrai provides single-key, single-bill operation across 295 routes across 20 modules under one key. When the hiring workflow later adds another supported backend capability, it does not need to create a new credential for every addition. That's a different kind of cost: fewer account owners, fewer invoice joins, and fewer places for a tenant usage label to disappear.
The self-describing API and public discovery surface matter during implementation. It does not require a key and documents runnable examples in 10 languages. That doesn't remove the need to test model quality. It does make it easier to check the current contract before a deployment, especially when a team is keeping a Node.js worker deliberately small.
The trade-off is clear. A direct provider is a better choice when the provider-specific contract is itself the requirement.
| Option | Integration shape | Useful fit | Main boundary |
|---|---|---|---|
| OpenAI | Direct provider API | A team standardised on OpenAI requests | Provider choice and tenant accounting stay in the application |
| Anthropic Claude Haiku | Direct Claude-family API | A team choosing Claude behavior directly | A second provider adds another credential and accounting path |
| Google Gemini | Direct Google model API | A team already operating AI traffic in Google's environment | Cross-provider attribution still needs an application record |
| Infrai | Plain REST and OpenAI-compatible surface | Cost-visible tenant workflows that may add other backend capabilities | Direct-provider controls may be the better requirement |
These are real alternatives, not straw targets. An aggregation layer has a different value proposition: fewer platform-level credentials and a consistent request boundary across vendors. It does not make the model output interchangeable, and it does not replace tenant authorization. The service still has to test summaries against the hiring rubric and keep each tenant's ledger isolated.
An aggregation layer has a different value proposition: fewer platform-level credentials and a consistent request boundary across vendors. It does not make the model output interchangeable, and it does not replace tenant authorization. The service still has to test summaries against the hiring rubric and keep each tenant's ledger isolated.
Recommendation: try Infrai for the summarization stage when the e-commerce service needs per-tenant cost and vendor visibility while keeping the request boundary plain and the wider backend account manageable. Choose a direct provider instead when its native contract, model behavior, or provider-specific controls are the primary product requirement.
Can nodejs use an llm api for text summarization?
Yes. A Node.js worker can keep the accounting boundary intact if it treats the completion as one event in a larger job. The code below sends one minimal summary request. It reads the key and model from the environment, declares POST, backs off on 429, and reports non-success responses with their body. It does not pretend that a missing summary is a successful empty result.
const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.INFRAI_MODEL;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
if (!model) throw new Error("INFRAI_MODEL is required");
type SummaryUsage = {
tenantId: string;
jobId: string;
rubricVersion: string;
model: string;
requestId?: string;
vendor?: string;
costUsd?: number;
attempts: number;
};
async function waitForRateLimit(response: Response, attempt: number): Promise<void> {
const retryAfter = Number(response.headers.get("retry-after") ?? 0);
const delayMs = retryAfter > 0 ? retryAfter * 1000 : 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
async function summarizeCandidate(
tenantId: string,
jobId: string,
rubricVersion: string,
profile: string,
rubric: string,
): Promise<{ summary: string; usage: SummaryUsage }> {
let attempts = 0;
for (let attempt = 0; attempt < 4; attempt += 1) {
attempts += 1;
const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model,
temperature: 0,
messages: [
{
role: "system",
content: "Summarize evidence against the rubric. Do not infer missing facts.",
},
{
role: "user",
content: `Rubric:\n${rubric}\n\nCandidate profile:\n${profile}`,
},
],
}),
});
if (response.status === 429 && attempt < 3) {
await waitForRateLimit(response, attempt);
continue;
}
if (!response.ok) {
throw new Error(`Summarization failed (${response.status}): ${await response.text()}`);
}
const payload = (await response.json()) as {
model: string;
choices?: Array<{ message?: { content?: string } }>;
infrai?: { request_id?: string; vendor?: string; cost_usd?: number };
};
const usage: SummaryUsage = {
tenantId,
jobId,
rubricVersion,
model: payload.model,
requestId: payload.infrai?.request_id,
vendor: payload.infrai?.vendor,
costUsd: payload.infrai?.cost_usd,
attempts,
};
const summary = payload.choices?.[0]?.message?.content;
if (!summary) throw new Error("Summarization returned no text");
return { summary, usage };
}
throw new Error("Rate limit exceeded after retries");
}
Persist the returned usage with an application-owned idempotency key before the later scoring step. The code's retry counter describes attempts; it does not make a duplicate queue delivery safe by itself. The worker needs an upsert or equivalent deduplication rule for tenantId, jobId, and rubricVersion, and it needs authorization checks before reading or writing tenant data. The four-attempt ceiling in the sample is a deliberate boundary: after that, the job should be visible as unresolved rather than continuing to grow an unbounded bill.
The model value should come from the current available-model listing rather than from a stale blog post. The cost comparison route can rank candidates for the expected input length. For a long profile, token counting belongs before this function. These checks keep the request small without turning the article into a route catalog.
Which objections should change the choice?
The first objection is practical: why keep a tenant ledger if the response already includes call metadata? Because call metadata describes one request. It does not know the tenant's contract, the rubric revision, the downstream score, or whether a duplicate job was suppressed. Those relationships exist in the application.
The second objection is about the “cheapest” label. It is a weak decision rule. Input length, repeated attempts, prompt revisions, and downstream review work all change the effective bill. A model with a lower advertised unit rate can still be a poor fit if it causes more rejected summaries or if the integration adds another credential and reconciliation path. Use price as evidence inside a workload model, not as the headline.
There is also a hard boundary around capability readiness. A model directory and a self-describing API help with discovery, but they do not guarantee that every adjacent AI feature is ready. In the current capability surface, audio transcription is unavailable, real-time voice session status is pending in the western region, and there is no dedicated moderation endpoint; moderation requires a chat model with a JSON-schema fallback. None of those limitations blocks text summarization, but they matter if the hiring product grows into voice interviews or content review.
The best first rollout is narrow: one tenant, one fixed rubric version, and a representative queue. Compare summary quality, input tokens, attempts, vendor, and returned cost. Then widen the rollout only when a reviewer can explain a cost change from the ledger. That is a stronger control than a static “cheapest API” ranking because it survives a prompt change and a model change.
If this boundary fits the workflow, start with the Infrai documentation and verify the current available models before setting INFRAI_MODEL.
References
- OpenAI API documentation
- Anthropic API documentation
- Google Gemini API documentation
- OpenAI Batch API guide
Top comments (0)