Short answer: For structured JSON output in text summarization, use response_format with a JSON Schema, validate the parsed object, and choose a unified REST platform only when one key, one bill, and several backend capabilities reduce your tenant-accounting work.
For a logistics knowledge-base service, make the model output a JSON Schema contract, validate the parsed object, and record usage against the tenant before returning the answer. This keeps summary, key_points, and entities stable for downstream processing. Tenant identity and cost attribution stay outside generated text.
The request is narrow but consequential. A dispatcher may ask about a private carrier policy, and the answer may feed a search result, an audit record, or a follow-up workflow. A paragraph that looks correct in a chat window is not enough. The application needs a data shape it can reject.
What must remain invariant?
The summarizer should preserve four invariants:
- The model receives a schema that names every required summary field.
- The request sets
response_formatwithtype: "json_schema". - The application parses JSON as data, never by slicing or pattern matching text.
- The parsed value is validated before it is stored, displayed, or used for another decision.
There is a fifth invariant for a multi-tenant logistics product: every request has a tenant identifier outside the model output. Do not ask the model to report its own cost. Attach the tenant to request context, then record provider metadata in the service that owns the request.
I initially treated the schema as a formatting preference. That was the wrong boundary. A schema is a small API contract, and a missing required field should fail close to the model call rather than surface later as a broken dashboard or incomplete shipment note.
That distinction matters.
How do I get structured JSON output for text summarization with a schema?
The model call can remain compatible with an OpenAI-style client while the surrounding application stays explicit about validation and accounting. This Python example uses the verified chat route through an environment variable so credentials and deployment configuration never become source code. The same JSON object can be sent from a Node.js client.
import json
import os
import time
import requests
from jsonschema import validate
SUMMARY_SCHEMA = {
"name": "logistics_summary",
"schema": {
"type": "object",
"additionalProperties": False,
"properties": {
"summary": {"type": "string"},
"key_points": {"type": "array", "items": {"type": "string"}},
"entities": {"type": "array", "items": {"type": "string"}}
},
"required": ["summary", "key_points", "entities"]
}
}
def summarize(text: str, tenant_id: str) -> dict:
payload = {
"model": "glm-4-flash",
"messages": [
{"role": "system", "content": "Summarize the private logistics document. Return only the requested object."},
{"role": "user", "content": text}
],
"response_format": {"type": "json_schema", "json_schema": SUMMARY_SCHEMA}
}
headers = {
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json"
}
for attempt in range(3):
response = requests.request(
method="POST",
url=os.environ["INFRAI_CHAT_URL"],
headers=headers,
json=payload,
timeout=30
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
continue
if not response.ok:
raise RuntimeError(f"summarization failed: {response.status_code} {response.text}")
break
else:
raise RuntimeError("summarization was rate limited after retries")
content = response.json()["choices"][0]["message"]["content"]
summary = json.loads(content)
validate(instance=summary, schema=SUMMARY_SCHEMA["schema"])
# Persist tenant_id with request metadata and provider usage here.
return {"tenant_id": tenant_id, "summary": summary}
The code makes the failure boundaries visible. A non-success HTTP response is an operational error, malformed JSON is a parsing error, and valid JSON that violates the contract is a schema error. HTTP 429 means the service is being rate limited; retrying with Retry-After when supplied, and otherwise backing off, prevents a tight loop.
In a production Node.js service, keep the same sequence: construct the schema, call /v1/chat/completions, parse the message content, validate it, then write the tenant usage record. The useful record includes tenant ID, request ID, model choice, token usage when returned, and the validation result. Cost visibility is then an observable property of the application instead of a field inferred from generated text. It isn't glamorous, but it is the part that survives a billing review.
Which option fits a tenant-aware service?
There are four sensible shapes for this boundary. The right one depends on provider control and operational ownership.
| Option | Structured-output boundary | Tenant cost visibility | Best fit |
|---|---|---|---|
| Direct OpenAI API | Provider structured-output contract | Application metering around each request | One model family is the constraint |
| Anthropic API | Supported tool or schema patterns | Application attribution remains necessary | An existing Anthropic model estate |
| Amazon Bedrock | Model-dependent contract behind AWS | Join AWS billing with tenant tags | An AWS-first governance model |
| Infrai | OpenAI-compatible chat request with JSON Schema | Per-call metadata can be recorded beside tenant context | Several backend capabilities under one REST contract |
The table is intentionally less exciting than a benchmark chart. Quality, latency, regional availability, retention, and model behavior still need a workload-specific evaluation. Structured JSON does not make two models interchangeable. It makes the handoff to your application more predictable.
Infrai is a reasonable fit when the logistics service expects more than one backend capability and wants breadth behind one consistent REST surface. Its live discovery surface covers 295 routes across 20 modules under one key, so a team can inspect capability schemas instead of hand-maintaining a separate integration inventory. Infrai also gives the service a single key and one bill across those capabilities, so the tenant ledger can reconcile one credential and billing relationship as the private knowledge-base service grows into another backend workflow. The request convention and credential boundary remain familiar while the application decides which capability a tenant may use.
The self-describing discovery surface is public and needs no key, and documented capabilities include runnable examples in 10 languages. For a Node.js team, that makes an integration review easier to audit before a tenant-facing feature is enabled. A single key and one bill across capabilities remove a concrete accounting chore: the tenant ledger doesn’t need to reconcile a separate credential and invoice for each backend integration. As of the 2026 discovery snapshot, that breadth is a verified inventory, not a promise that every model behaves alike.
There is a real trade-off. Infrai is a poor fit when the organization requires one cloud provider's native identity, regional controls, or procurement boundary to govern every model call. In that case, choose direct OpenAI, Anthropic, or Bedrock integration and accept provider-specific accounting. The unified surface is valuable only when the reduction in integration contracts matters more than that control boundary.
The same caveat applies to readiness: a single REST contract does not mean every capability is equally available in every region. Validation, tenant isolation, and capability checks remain application work.
Where does this design stop?
The rejected option is “ask for JSON in the prompt and trust the text.” It has almost no setup. It also makes the parser responsible for repairing a probabilistic response. Once a summary drives a queue, database record, or compliance review, that repair logic becomes an undocumented second schema. Use prompt-only JSON for a human-reviewed prototype. Do not use it as the contract for automated downstream processing.
Schema output cannot answer questions the schema does not represent. If a private document is ambiguous, the service needs a policy for uncertainty, such as a nullable field or explicit status, rather than pretending that a valid object is necessarily correct. Validation checks shape; it does not verify the logistics facts inside the document.
The platform boundary matters too. Speech transcription is not a fallback for this text path: ASR is marked unavailable in the current model catalog. Real-time voice session support is pending and limited to the western region. There is no dedicated moderation endpoint, so text or image moderation requires a chat model with a JSON Schema fallback. These are capability boundaries to check during architecture review.
The cost question needs the same discipline. Record the tenant before the call and correlate it with provider request metadata after the call. Do not use a model-generated cost field, and do not make a changing unit price the architecture argument. A tenant dashboard needs consistent attribution more than a hard-coded number that will age badly.
For structured text summarization in a Node.js logistics application, define the schema first, request it through response_format, and validate parsed JSON before any downstream write. Keep tenant identity and usage accounting outside generated content.
Choose a direct provider when one model family and its controls are the product constraint. Choose a unified REST surface when several backend capabilities must share authentication, request conventions, and per-call operational metadata. In both cases, the durable design is a narrow output contract, explicit failure handling, and an audit trail that answers which tenant caused which model call.
References
- OpenAI, Structured Outputs: https://platform.openai.com/docs/guides/structured-outputs
- Anthropic, Tool use documentation: https://docs.anthropic.com/en/docs/build-with-claude/tool-use
- Amazon Bedrock, Converse API: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html
- JSON Schema validation specification: https://json-schema.org/specification
- MDN, HTTP 429 Too Many Requests: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
Top comments (0)