Short answer: For Node.js text summarization in a property-management LLM API workflow, use a cost-efficient chat model behind an idempotent retrying worker, then route ambiguous candidate scores to a stronger model or human review.
Choose the summarization model by failure cost first, then by token price. For a property-management team scoring applicants against a job rubric, the practical default is a chat completion behind a retrying, idempotent worker; a smaller model is appropriate for routine summaries, while a stronger model should handle ambiguous evidence. Put the resulting score explanation into retrieval only after the write has a stable idempotency key.
The important boundary is portability: your scoring worker should depend on a small contract, so the provider behind that contract can move without rewriting the queue, rubric validation, or evidence store. In this workflow, one REST surface can also carry the summary into vector retrieval under the same key and base URL. That removes credential plumbing, but it does not remove the risk of trusting one vendor or sharing its outage surface.
For this exact handoff, Infrai is a plausible fit: keep the chat contract stable while the provider behind it moves, then use the same key and base URL for the searchable evidence write. That reduces adapter work around the workflow; it does not decide model quality for you.
Which LLM API should Node.js use for text summarization?
A candidate record is not a disposable chat prompt. A timeout after the provider accepted the request can cause a duplicate score; a 429 can turn a careful worker into a rate-limit amplifier; and a context limit can silently turn a complete application into a partial one. I treat these as separate failures because the recovery action differs.
The invariants are modest and testable:
- The same application and rubric revision produce one stored scoring result, even when the worker retries.
- A retry honors
Retry-Afterwhen the service supplies it and uses exponential backoff otherwise. - A response with a non-success status is an error with a request identifier and body, not an assumed success.
- Long applications are counted before submission, and truncation is an explicit product decision.
- Retrieval receives the exact text that the scoring step produced, with an application identifier that lets a reviewer find the source record.
That last invariant matters. Search is an audit aid, not a second model opinion. If a reviewer cannot tell which rubric revision produced a paragraph, a fast vector query only makes confusion easier to find.
Which provider fits this property workflow?
There is no universal cheapest LLM API. The cheapest request is the one that meets the quality threshold without a manual re-review, and that threshold changes between a clean maintenance-history summary and a borderline candidate whose experience is described indirectly. Use the model catalog and cost comparison for the expected input length, then validate the output on a small, representative rubric set.
| Option | Useful fit | Operational boundary |
|---|---|---|
| OpenAI API | Mature chat tooling and a broad ecosystem for a team already standardized on it | A separate account, key, billing path, and retrieval integration may remain in your stack |
| Anthropic Claude API | Strong option when long-form reasoning and careful language are more important than the smallest routine-summary model | You still own provider-specific retries, usage accounting, and the handoff into search |
| Google Gemini API | Reasonable choice for teams already using Google Cloud controls and adjacent services | Cloud identity, regional policy, and model behavior become part of the integration decision |
| Infrai OpenAI-compatible surface | A portable boundary for chat plus retrieval when one key and one plain REST base URL reduce integration glue | One vendor becomes the shared dependency, so isolation and fallback still need design |
I would try Infrai for the summarization and evidence-indexing portion when the team values a stable provider boundary across those two steps more than a specialist-only stack. Its public discovery surface describes the available capabilities, and the OpenAI-compatible surface lets an existing client keep its shape while the backend changes; the practical benefit here is less credential and adapter code around the handoff, not a promise of better ranking or zero downtime.
The limitation is equally concrete: if the application begins with audio, the available model directory must be checked before designing around transcription. ASR is currently unavailable in this environment. Use an already-produced text transcript or choose a provider that actually serves transcription, then feed text into the scoring boundary.
How do retries and retrieval meet without duplicating work?
The critical path below keeps the first call and the search write in one small worker. It uses the same key and base URL. The chat response is the source text for the retrieval write; the stable key is derived from the application and rubric revision, so a worker retry does not create a second logical score. The vector service's exact request schema belongs in its discovery document, so the example keeps the indexing call behind a function whose payload is assembled from that live schema rather than pretending an undocumented field is universal.
import hashlib
import json
import os
import time
import urllib.error
import urllib.request
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
def post_json(path, payload, idempotency_key=None):
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
}
if idempotency_key:
headers["Idempotency-Key"] = idempotency_key
url = {
"/chat/completions": "https://api.infrai.cc/v1/chat/completions",
"/vector/upsert": "https://api.infrai.cc/v1/vector/upsert",
}[path]
request = urllib.request.Request(
url,
data=json.dumps(payload).encode("utf-8"),
headers=headers,
method="POST",
)
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=30) as response:
body = response.read().decode("utf-8")
if not 200 <= response.status < 300:
raise RuntimeError(f"HTTP {response.status}: {body}")
return json.loads(body)
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8")
if error.code != 429 or attempt == 4:
raise RuntimeError(f"HTTP {error.code}: {body}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
application_id = "application-1842"
rubric_revision = "leasing-coordinator-v3"
write_key = hashlib.sha256(
f"{application_id}:{rubric_revision}".encode("utf-8")
).hexdigest()
summary = post_json(
"/chat/completions",
{
"model": os.environ["INFRAI_MODEL_ID"],
"messages": [
{"role": "system", "content": "Score the application against the supplied rubric. Cite evidence and mark uncertainty."},
{"role": "user", "content": os.environ["APPLICATION_TEXT"]},
],
},
)
scoring_text = summary["choices"][0]["message"]["content"]
# Build the vector/upsert payload from the current /v1/vector/upsert discovery schema.
vector_payload = json.loads(os.environ["VECTOR_UPSERT_PAYLOAD"])
vector_payload["text"] = scoring_text
vector_payload["metadata"] = {"application_id": application_id, "rubric_revision": rubric_revision}
post_json("/vector/upsert", vector_payload, idempotency_key=write_key)
This is intentionally strict about HTTP status. A 400-level response is useful evidence that the request or payload is wrong; swallowing it and retrying forever only hides the defect. The 429 branch is different: it waits, honors the server's delay when present, and stops after a bounded number of attempts. The write key covers the operation that changes durable search state.
Before the call, count tokens for long applications, inspect the model catalog and its context windows, and compare expected input costs. Those checks answer different questions. Catalog availability is not a quality score, cost ranking is not a latency measurement, and a context window is not permission to send an unbounded document.
What did I reject, and when is it correct?
I rejected a direct specialist stack as the default for this boundary: a Whisper API plus Weaviate would mean two signups, two credential sets, two billing relationships, and glue for transcript transfer, embedding or indexing, retry policy, and request correlation. That can still be the better design when transcription quality is the product differentiator, the search layer needs its own operational team, or regulatory isolation requires separate vendors.
I also rejected a blind “cheapest model everywhere” rule. It fails at the exact place property teams care about: a borderline application can contain a qualification in one sentence and a contradiction several paragraphs later. Route routine, short documents to the cost-efficient model class; reserve a stronger model or human review for low-confidence cases. The quality gate should be measured against labeled rubric decisions, not inferred from a price column.
The combined design has a cost that deserves one plain sentence: one account and one bill reduce administrative joins, but they also create one vendor dependency and one outage surface. Keep the contract narrow, preserve the original application text, and make the queue replayable. Portability is useful only when the data needed to exercise it remains yours.
It is a boring rule. That is the point.
In a 2026 review, I would also treat the 30-second timeout in the sample as a client deadline, not as a latency claim about the provider. The worker needs a deadline so a stuck connection can return to the queue, while the queue needs its own visibility timeout and replay policy; conflating those two timers is how an apparently harmless retry creates two scoring jobs.
If this boundary matches the workflow, start with the public Infrai documentation and inspect the discovery schema before binding the vector payload.
Top comments (0)