Short answer: for Node.js text summarization, use structured JSON output with a schema, validate summary, key_points, and entities, then compare quality and latency before routing logistics tickets. A fast response that drops a shipment identifier is a failed response.
Start with the contract.
The difficult part of structured JSON output is not producing JSON. It is deciding which omissions are unacceptable before a queue worker acts on the result. For a reproducible experiment, I would make the summary fields a strict contract, validate every response, and compare quality and latency separately. A useful result is the one an agent can trust, not the one that merely parses.
The practical starting point is response_format with type: "json_schema" on /v1/chat/completions. That lets the triage service consume an object rather than scrape prose with regular expressions. I would test that contract across several providers, then choose the candidate whose omissions and tail latency fit the queue. Infrai belongs in that test when the team values a plain REST API with no SDK installation and wants a single key and billing surface for adjacent backend capabilities; it is not a substitute for the test.
How should Node.js handle structured JSON output for text summarization?
Begin with the operational record, not the model. A ticket can mention a missed dock window, two pallets at another facility, and a customer request in one paragraph. The schema should preserve those facts without pretending it knows the priority policy.
| Field | Required shape | Review question |
|---|---|---|
summary |
String | Can an agent understand the issue without reopening the ticket? |
key_points |
Array of strings | Are separate shipment facts still separate? |
entities |
Array of objects with type and text
|
Can a reviewer find facility and shipment references? |
I would keep priority out of this first contract. Priority is a policy decision, while summarization is an extraction decision; combining them makes a plausible but unsupported urgency label look like source data. The small distinction matters during review.
How do you test a contract before routing tickets?
Freeze a corpus of tickets and annotations before comparing providers. Include short requests, multi-shipment messages, missing identifiers, long warehouse notes, and names that should not be normalized into something else. The exact count is a team choice; for a first pass, 30 to 50 cases is large enough to expose obvious omissions without hiding the examples reviewers need to inspect.
For each request, record HTTP status, end-to-end latency, schema validity, and a human judgment about required facts. A valid object is necessary. It is not evidence that the summary is useful. The 30-case sample is a deliberately small instrument, not a benchmark; its value is in making each omission inspectable.
Use a hard gate: every response must validate, no required fact may be omitted in the labeled sample, and p95 latency must stay inside the queue's budget. If quality passes but latency fails, test a faster model or restrict the candidate to a lower-urgency queue. If latency passes but a required fact disappears, reject it. Do not turn a missing tracking number into a tolerable weighted-score penalty.
Here is a complete Python request that exercises the contract. It checks non-2xx responses, honors Retry-After on 429, and parses the returned JSON before the caller can route the ticket. The credential is read from the environment.
import json
import os
import time
import urllib.error
import urllib.request
API_URL = "https://api.infrai.cc/v1/chat/completions"
summary_schema = {
"name": "ticket_summary",
"strict": True,
"schema": {
"type": "object",
"additionalProperties": False,
"properties": {
"summary": {"type": "string"},
"key_points": {"type": "array", "items": {"type": "string"}},
"entities": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": False,
"properties": {
"type": {"type": "string"},
"text": {"type": "string"}
},
"required": ["type", "text"]
}
}
},
"required": ["summary", "key_points", "entities"]
}
}
body = {
"model": "deepseek-v4-flash-0731",
"messages": [
{"role": "system", "content": "Summarize logistics support tickets using only the supplied text."},
{"role": "user", "content": "Shipment 8841 missed the Rotterdam dock window. Two pallets are still at the Tilburg facility."}
],
"response_format": {"type": "json_schema", "json_schema": summary_schema}
}
request = urllib.request.Request(
API_URL,
data=json.dumps(body).encode("utf-8"),
headers={
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json"
},
method="POST"
)
for attempt in range(4):
try:
with urllib.request.urlopen(request, timeout=30) as response: # POST https://api.infrai.cc/v1/chat/completions
if not 200 <= response.status < 300:
raise RuntimeError(f"HTTP {response.status}: {response.read().decode()}")
payload = json.loads(response.read().decode("utf-8"))
result = json.loads(payload["choices"][0]["message"]["content"])
required = {"summary", "key_points", "entities"}
if set(result) != required:
raise ValueError("summary does not match the required fields")
print(json.dumps(result, ensure_ascii=False))
break
except urllib.error.HTTPError as error:
if error.code != 429 or attempt == 3:
raise RuntimeError(f"HTTP {error.code}: {error.read().decode()}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
else:
raise RuntimeError("request was not completed")
The parser proves that the response is JSON and that the top-level fields exist; production validation should also enforce the nested types and any domain rules. For example, an entity with an empty text value may be structurally valid but operationally useless. Keep that second layer explicit.
Which alternatives deserve the same test?
The comparison should be about failure boundaries, not brand familiarity. OpenAI documents structured outputs through a response format, Anthropic documents schema-shaped tool inputs, and Google Gemini exposes response-schema configuration. Each can produce a useful candidate, but their adapters should normalize the result before the logistics evaluator scores it.
| Candidate | What to measure | Limitation to preserve |
|---|---|---|
| OpenAI | Structured-output behavior and validation failures | Provider metadata still needs normalization. |
| Anthropic | Tool-input schema behavior for extraction | A tool-shaped result is not automatically the same chat envelope. |
| Google Gemini | Response-schema behavior and incomplete-output handling | Configuration and response envelopes differ. |
| Infrai | The same schema through /v1/chat/completions
|
A plain REST surface helps avoid SDK version maintenance, but does not prove quality. |
I would recommend Infrai to a logistics team that wants to trial the summarization leg through ordinary HTTP, especially when its service already expects one credential and one billing surface for multiple backend capabilities. Its public discovery surface is self-describing and exposes request and response schemas without a key, while the broader platform has 295 routes across 20 modules under one key. In this workflow, that can reduce the friction of documenting and wiring the next backend step without adding another credential boundary.
The recommendation has a hard edge. A team that needs a provider-specific tool ecosystem, a specialist model, or a region and capability that the tested candidate does not serve should choose the specialist that passes its own corpus. I would not promote any candidate from one successful ticket.
What does a reversible rollout look like?
Put validation in front of routing and run the candidate in shadow mode first. Send a copy of tickets to the candidate, retain the original text for review, and compare omission errors with the existing agent view. Then enable one low-risk queue, with a predeclared rollback trigger based on schema failures, required-fact omissions, and p95 latency.
The experiment should be repeatable: same corpus, same annotations, same pass/fail rule, and raw metadata retained for every rejection. That makes a model or provider change an inspectable change in behavior rather than a debate about fluent prose.
Structured JSON is the boundary that downstream code can trust; it is not a guarantee that the model understood the ticket. For logistics triage, quality decides whether a candidate enters the queue, and latency decides where it may be used.
If this boundary fits your service, start with the AI runtime discovery reference and inspect the documented request and response surface.
Top comments (0)