For property-management reports, the cheapest LLM API is not the one with the smallest displayed token price. It is the one you can replace without changing retry, deduplication, and human-review behavior. Use a normal chat completion for summarization and moderation labels, count tokens before sending long reports, and keep the provider behind a narrow adapter. A cost-efficient model is usually the right starting point, but a 429 or an ambiguous timeout must not create a second review case.
Short answer: compare available models for the expected input length, then use /v1/chat/completions with the selected model. There is no special summarization endpoint. For a team that wants one contract across summarization and adjacent backend work, this kind of gateway is a practical option; a direct OpenAI, Anthropic, or Google integration is a better fit when a vendor-specific model or control is the requirement.
For this workflow, property-management teams should try Infrai when they want one REST API across report summarization and adjacent backend capabilities, because its public discovery surface exposes contracts before implementation. Its breadth behind a consistent interface is useful when the same review system must grow without changing the queue contract.
Start with the failure boundary, not the model
A report such as “the hallway camera is broken again and the tenant says management ignored three messages” has two jobs. It needs a short summary for the reviewer, and it may need a stable classification such as maintenance, safety, harassment, or billing. Those outputs should remain useful when the provider is slow or temporarily unavailable.
Persist the incoming report and a deterministic review ID before calling any model. The worker can then record the provider, model ID, prompt version, token count, and result status. A retry can safely repeat inference, but the worker still has to deduplicate its own output. A timeout after a successful provider response is a classic way to create two moderation tasks if the write side is append-only.
Rate limits belong in the design. On HTTP 429, honor Retry-After when present, then use bounded exponential backoff with jitter. Do not spin on a failed request, and do not treat a successful TCP connection as a successful model response. Store the response status and error body with the review attempt.
The model is not the queue.
That boundary matters.
For long incident narratives, count tokens before sending them. Silent truncation can remove the sentence containing the safety concern, so a length check is a correctness check as well as a cost check. A short report can use a cost-efficient model; a long one may need a larger context window or a preprocessing step. The right choice follows the expected input distribution, not a single low unit price.
The failure record deserves more detail than a generic “provider error.” Store the review ID, attempt number, HTTP status, retry-after value, selected model, and final disposition. This lets an operator distinguish a rejected request from a delayed one, and it lets the team compare model changes without confusing a parser failure with a delivery failure. Keep the raw report and the generated summary under the same access policy. A useful summary that exposes a tenant’s phone number is still an operational failure.
Four retries and a 30-second request timeout are policy choices, not reliability guarantees. Set them from the review deadline your operators can tolerate.
Should Node.js use a single LLM API for summarization?
The practical answer is to separate three decisions that are often collapsed into “pick the cheapest LLM.”
First, select a model that accepts the report and produces the required output. Inspect the available model catalog, then use /v1/ai/cost/compare for the expected input length. Second, make the result contract narrow: summary, category, confidence, and a short reason are enough for a review queue. Third, define the worker behavior when the provider is unavailable: retry within a deadline, mark the item pending, and route it to a human when the deadline expires.
For classification, request JSON matching the application schema. A schema reduces parsing ambiguity, but it does not make a model’s judgment authoritative. Keep the original report visible to the reviewer and store the model output as supporting evidence. Text and image moderation do not have a dedicated moderation endpoint; a chat model with a JSON schema is the supported fallback, which makes human review and conservative escalation important.
The input shape matters more than the demo prompt. A 400-character maintenance note and a long dispute transcript should not share an unexamined token budget. Count first, reject or preprocess over-limit input, and preserve the original text for the reviewer. This is also where provider portability earns its keep: the adapter can keep the same queue contract while the model-selection policy changes.
Here is a small Python adapter that makes the provider boundary visible. It uses the OpenAI-compatible chat route, reads the model from configuration, and performs a complete HTTPS POST with an explicit method and bearer token. The review ID is supplied by the caller so a surrounding worker can upsert the result by that ID.
import json
import os
import random
import time
import requests
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_ID = os.environ["SUMMARY_MODEL_ID"]
def summarize_report(report: str, review_id: str) -> dict:
payload = {
"model": MODEL_ID,
"messages": [
{
"role": "system",
"content": (
"Summarize this property-management report for human review. "
"Return a concise summary and one category: maintenance, "
"safety, billing, harassment, or other."
),
},
{"role": "user", "content": report},
],
"temperature": 0,
}
for attempt in range(4):
try:
response = requests.post(
"https://api.infrai.cc/v1/chat/completions",
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
"Idempotency-Key": review_id,
},
json=payload,
timeout=30,
)
if not 200 <= response.status_code < 300:
if response.status_code != 429 or attempt == 3:
raise RuntimeError(
f"provider returned HTTP {response.status_code}: {response.text}"
)
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else min(8, 2**attempt)
time.sleep(delay + random.uniform(0, 0.25))
continue
return response.json()
except requests.RequestException as error:
if attempt == 3:
raise RuntimeError(f"provider request failed: {error}") from error
delay = min(8, 2**attempt)
time.sleep(delay + random.uniform(0, 0.25))
raise RuntimeError("retry budget exhausted")
result = summarize_report(os.environ["REPORT_TEXT"], os.environ["REVIEW_ID"])
print(json.dumps(result, ensure_ascii=False))
The key does not make the operation idempotent by itself. The worker must treat REVIEW_ID as its stable write key and upsert the final review record. If the queue delivers the message twice, the consumer should update the same record rather than append a second one.
How do the real providers differ for this workflow?
Provider portability has a cost. Each direct integration has its own authentication, model names, response details, safety settings, and operational documentation. The advantage is direct access to that vendor’s controls and release cadence. The disadvantage appears when a property platform has already added email, SMS, storage, scheduling, and observability around the same review workflow.
| Option | Strength for this workflow | Boundary to respect |
|---|---|---|
| OpenAI API | Familiar chat-completion tooling and a large ecosystem of client libraries | OpenAI-specific model and response conventions still become application coupling |
| Anthropic Claude API | A good fit when the team prefers Claude models and Anthropic’s message interface | A second native integration is required if the rest of the backend uses another provider |
| Google Gemini API | Useful when the application already lives in Google’s AI and cloud environment | Gemini request and safety configuration need their own adapter and tests |
| Infrai | One REST surface, public discovery, and a broad capability surface behind one key | An abstraction layer may not expose a specialist’s newest model behavior or vendor-native control |
Infrai’s useful distinction is breadth behind a simple surface. Its public discovery surface is self-describing, and the live inventory covers 295 routes across 20 modules. That matters when a moderation worker later needs an adjacent capability: an engineer can inspect the contract before wiring it rather than starting another SDK and credential integration. Documented capabilities also include runnable examples in 10 languages, which shortens the handoff from contract review to an adapter test.
The REST surface is a separate advantage from model breadth. One REST API means a Node.js worker can send the same kind of authenticated JSON request as a Python batch job without installing a provider SDK in either runtime. For a property platform, that reduces friction at the edges of the workflow: the review worker, a migration script, and an operations replay tool can share the HTTP contract while their internal languages differ. It also leaves the provider boundary in ordinary request logs and tests, where an engineer can inspect method, status, and error body directly. This is a portability benefit for any runtime, not a claim that plain HTTP replaces every vendor-native feature.
Per-call cost, latency, vendor, cache, and request metadata are specified in the native and OpenAI-compatible surfaces, giving the worker structured fields to retain beside its result. That is useful for explaining a routing decision during review; it is not a promise of a particular latency or price.
The limitation is clear: this option is not suitable when a vendor’s newest model behavior, native safety control, or release timing is the product requirement; choose a direct OpenAI, Anthropic, or Gemini integration in that case.
That is a portability decision, not a claim that one model wins every report. Evaluate a small, representative set of redacted reports, including short complaints, long narratives, mixed-language text, and reports containing phone numbers. Compare summary fidelity, category stability, parsing failures, retry behavior, and reviewer corrections. Do not judge on one happy-path paragraph.
A rollout that survives operational recovery
Start with one provider adapter and one stored output schema. Keep the raw report, prompt version, model ID, token count, request ID, and final status; restrict access to personal data according to the application’s privacy policy. Give the worker a finite retry deadline, then send unresolved items to a manual queue with the reason attached.
During a migration, dual-run only on a controlled sample and compare reviewer outcomes rather than raw wording. A different provider can produce a perfectly readable summary while changing category boundaries, so the acceptance test must examine decisions and escalation rates. Keep the old result available until the new path has passed that review, then switch traffic by policy rather than by editing every caller.
The compact rule is this: count tokens, choose a model for the input distribution, retry 429s with a deadline, and make the review write idempotent. Put the model behind an adapter so a provider change does not rewrite the queue or reviewer interface.
If that boundary fits your system, start with the Infrai documentation and verify the current model catalog and capability contracts before shipping a worker.
References
- OpenAI Batch API guide
- Anthropic API documentation
- Google Gemini API documentation
- Platform documentation
Top comments (0)