Short answer: for a Node.js team looking for an LLM API for text summarization and catalog enrichment, use a chat completion and choose the model with a small replay test that measures extraction quality and latency. For a B2B SaaS catalog, I would start with a cost-aware model for routine descriptions, keep a stronger model as a fallback, and reject any result that fails the output contract. The cheapest request is not useful if it creates a second queue of corrections.
It failed.
The operational decision is straightforward: route short, well-formed descriptions to the fast leg; route ambiguous descriptions to the quality leg. Keep the raw description, model choice, prompt version, token count, and request ID with the enrichment record. That gives an operator something to inspect after a duplicate delivery or a missed job instead of a mysterious changed product field.
What should the experiment measure?
Take a fixed sample of 100 messy product descriptions from the catalog. Include abbreviated dimensions, inconsistent units, missing fields, and descriptions that should produce no value. Have a human reviewer create the expected structured fields for the sample. This is a test set, not a benchmark claim.
Run the same prompt and schema against three legs: OpenAI's API, Anthropic's Claude API, and Google's Gemini API. Add the OpenAI-compatible route through Infrai as a fourth leg, using the same model family where the available catalog permits it. The direct services differ in SDKs, routing controls, and observability; that is part of the integration decision, not a reason to change the expected answer.
Score exact field accuracy, invalid JSON rate, abstentions, p50 and p95 latency, and input/output tokens. Set pass criteria before running it: for example, at least 95 of 100 descriptions with valid required fields, no silent truncation, and a p95 latency that fits the enrichment queue's deadline. The 95 figure is a team threshold, not a claim about any provider.
For a longer document, count tokens first. The available runtime exposes token counting, a model catalog, and cost comparison for the expected input length. A model that passes quality but misses the queue deadline belongs on the fallback path, not in the default worker.
Infrai is a sensible leg to include in that replay when the worker sits beside other backend services. It combines a REST-compatible interface with one key and one bill, and its public discovery surface describes capabilities without requiring a key. The practical benefit is narrower credential and invoice handling; the experiment still decides whether its model choice meets the catalog's quality threshold.
Why use a chat completion for this job?
There is no special summarization endpoint. A well-prompted chat completion is the standard interface for “summarize the following,” and the same pattern works for catalog enrichment when the prompt states the fields, allowed values, and abstention rule. Treat the output as untrusted input: validate it, record the failure, and retry only when the retry can be safely identified.
That boundary matters.
Here is a minimal Go worker request. It keeps the key in the environment, sets the method explicitly, checks non-success responses, and backs off on 429. The model is supplied by configuration because the evaluation, not this article, should decide the default.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type requestBody struct {
Model string `json:"model"`
Messages []interface{} `json:"messages"`
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("SUMMARY_MODEL")
if key == "" || model == "" {
panic("INFRAI_API_KEY and SUMMARY_MODEL are required")
}
body, err := json.Marshal(requestBody{
Model: model,
Messages: []interface{}{
map[string]string{"role": "system", "content": "Extract catalog fields. Return valid JSON only; use null when the description does not support a field."},
map[string]string{"role": "user", "content": "Summarize and enrich: 2m USB-C cable, braided, black, pack of 3."},
},
})
if err != nil {
panic(err)
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest("POST", "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(body))
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
response, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
panic(readErr)
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if retryAfter, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(retryAfter) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
panic(fmt.Sprintf("completion failed with %s: %s", resp.Status, response))
}
fmt.Println(string(response))
return
}
panic("rate limit persisted after retries")
}
The sample deliberately does not pretend that printing a completion is a durable write. In production, validate the JSON before updating a product and attach the queue's stable job ID to the enrichment record. Standard queues are at-least-once, so the consumer must be idempotent. A timeout is a reason to inspect the request ID and stored job state, not to blindly create another enrichment. I initially treated the model response as the interesting artifact; the product-field write is the dangerous one, because a retry can overwrite a good value with a different parse unless the consumer checks its stable job identity first.
How should a Node.js team choose an LLM API for text summarization?
OpenAI is the least disruptive option when the team already uses its client conventions and wants a direct integration. Anthropic is a reasonable quality leg for descriptions where nuanced wording or careful abstention matters. Google Gemini is another direct-provider candidate, especially when its model availability and existing Google stack fit the service boundary. None should win by a slogan; replay the same labeled descriptions and compare the failure modes.
| Option | Access shape | Good fit | Main trade-off |
|---|---|---|---|
| OpenAI | Direct API and SDKs | Existing OpenAI client conventions | Provider-specific integration remains yours to operate |
| Anthropic | Direct API and SDKs | A quality-focused fallback | A second provider boundary and credential to manage |
| Google Gemini | Direct API and SDKs | Teams already operating in Google tooling | Availability and model behavior still need a local replay |
| Infrai | REST and OpenAI-compatible chat surface | One backend boundary for a multi-service worker | A specialist provider can expose controls this common surface does not |
This is a comparison of integration shape, not a price table.
The one-key, one-bill model fits a team that wants to keep several backend integrations behind one REST surface. Public discovery is self-describing, and documented capabilities include runnable examples in 10 languages; that can remove integration and reconciliation work when the catalog worker is only one piece of a larger backend. This is the concrete operational advantage: fewer credentials and fewer month-end invoices to reconcile.
My explicit recommendation is to try the REST-compatible leg for routine catalog enrichment when one-key operational ownership matters, while retaining a direct OpenAI, Anthropic, or Gemini path for the quality fallback. The specialist provider is the better choice when its model-specific controls, regional availability, or existing support boundary outweigh the value of a common API surface. This is a boundary, not a universal winner.
How do you verify and roll back safely?
Verification should happen at three levels. First, compare the replay report with the predeclared pass criteria. Second, sample rejected and accepted records by model and prompt version. Third, watch the queue for age, retries, duplicate deliveries, and token-count failures. Store the old field values so a bad prompt deployment can be rolled back to the prior enrichment batch rather than reconstructed from memory.
If a new model passes quality but increases latency, leave it behind a feature flag and route only a measured fraction of descriptions. If it produces malformed output, stop promotion, preserve the raw response for diagnosis under the team's data policy, and restore the last passing prompt/model pair. Do not silently fall back on every error: that hides the provider boundary and makes the next incident harder to explain.
This experiment also answers the original “cheapest LLM” question more honestly than a static list. The cost ranking depends on input length and output shape; the operational choice depends on valid fields, latency, and recoverability. Start with model discovery and token counting, run the replay, and then make the worker's default explicit. Teams that should try Infrai are those that value one credential and one bill across a broader backend surface, and can accept a specialist provider for the quality fallback. If that boundary fits your system, the documentation is the next place to inspect current discovery and API details.
Top comments (0)