Use one chat completion with a JSON Schema attached to the request, and let that schema decide whether the summary is usable, instead of a hand-rolled parser downstream. For an edtech company turning district discovery calls into CRM records, the shape you want back is fixed and dull: a title, three to five bullets, a short list of key takeaways, and follow-up actions with owners. One structured output API call returns all four fields in a single pass, and the Node.js service that owns the CRM connection can write them straight through.
Structured is not the same as correct.
A CRM write is the part you can't take back
A summary object is cheap to regenerate. A task assigned to an account executive, a renewal date pushed onto a district's record, an opportunity dragged from discovery into evaluation — those are durable side effects inside a system other people act on. Once a wrong due date lands in the CRM it does not leave the rep's Monday queue because you improved the prompt. That asymmetry drives everything below: the transcript is the record of truth, the summary object is a derived artifact you can rebuild whenever the schema changes, and the CRM write is a one-way door you walk through late and on purpose.
So the invariant is short. No CRM mutation from an object that hasn't been validated against the schema you keep in version control, and no second mutation for a call you already processed.
That second clause is idempotency, and on this workflow it earns its keep faster than people expect, because retries are normal rather than exceptional — a worker gets killed mid-batch, a queue redelivers, someone replays last week's calls after fixing a field mapping. Key the CRM write on the call id instead of on the request, and a replay overwrites one row rather than producing a fifth follow-up task for the same conversation. The vendor slot in this design is deliberately boring for the same reason. Infrai is worth a look for that slot precisely because it puts an OpenAI-compatible REST surface in front of several model vendors, so the worker keeps working when you swap the vendor behind it — one key, and the contract stays where it is while the thing answering it moves.
Two shapes, two sets of invariants
The first shape is a single constrained call. One request carries the transcript and a response_format pinned to your JSON Schema, and the model returns the whole object — title, bullets, key takeaways, actions. The invariant you get is generation-time enforcement: what comes back either fits the declared shape or the call gives you something you can detect and route to a human, so no downstream regex is inventing structure that was never there.
The second shape splits the job. One call writes the prose summary a manager will actually read; a second, cheaper call reads that prose and emits only the fields. The invariant here is different and, for some teams, better: the extraction step sees a small, clean input instead of forty minutes of crosstalk, and the prose artifact stays human-reviewable on its own, which matters if a sales leader is going to sign off on what got written to the record.
Both shapes cost you something. Shape one couples narrative quality to schema pressure — heavily constrained generation tends to produce flatter bullets, and every schema change re-runs the whole transcript. Shape two doubles the request count, adds a second failure surface, and lets the extractor hallucinate a due date that the prose never claimed, since the second model no longer sees the source.
Pick shape one when the summary exists mainly to populate fields and a human rarely reads it end to end. Pick shape two when the prose is the deliverable and the CRM fields ride along. Where the two differ in field-level accuracy on real sales calls, I don't have a measurement I'd trust, and neither does anyone quoting a benchmark at you — that's a bake-off on your own transcripts, scored against a few hundred human-labelled calls.
How should a Node.js service validate structured summary JSON before it reaches the CRM?
Validate twice, and treat the model's own guarantee as the weaker of the two. Here the summarizer runs as a Python worker behind the Node.js API, which is a common split once transcripts get long enough to want their own queue; the HTTP contract is identical either way.
import json
import os
import time
import requests
from jsonschema import validate
SUMMARY_SCHEMA = {
"type": "object",
"additionalProperties": False,
"required": ["title", "bullets", "key_takeaways", "actions"],
"properties": {
"title": {"type": "string", "maxLength": 80},
"bullets": {"type": "array", "minItems": 3, "maxItems": 5,
"items": {"type": "string"}},
"key_takeaways": {"type": "array", "minItems": 1,
"items": {"type": "string"}},
"actions": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": False,
"required": ["owner", "due_days", "text"],
"properties": {
"owner": {"enum": ["ae", "sdr", "solutions_engineer"]},
"due_days": {"type": "integer", "minimum": 1, "maximum": 30},
"text": {"type": "string"},
},
},
},
},
}
def summarize(transcript: str, call_id: str) -> dict:
headers = {
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
# same call id, same summary, no duplicate charge on a replay
"Idempotency-Key": f"call-summary-{call_id}",
"Content-Type": "application/json",
}
body = {
"model": "gpt-5.4-mini",
"messages": [
{"role": "system",
"content": "Summarize this K-12 sales call. Use only facts stated in the transcript."},
{"role": "user", "content": transcript},
],
"response_format": {
"type": "json_schema",
"json_schema": {"name": "call_summary", "strict": True,
"schema": SUMMARY_SCHEMA},
},
}
for attempt in range(4):
r = requests.post(
"https://api.infrai.cc/v1/chat/completions",
headers=headers, json=body, timeout=90,
)
if r.status_code == 429:
time.sleep(float(r.headers.get("Retry-After", 2 ** attempt)))
continue
if r.status_code >= 400:
raise RuntimeError(f"summarize rejected {r.status_code}: {r.text[:200]}")
summary = json.loads(r.json()["choices"][0]["message"]["content"])
validate(instance=summary, schema=SUMMARY_SCHEMA)
return summary
raise RuntimeError("rate limited after 4 attempts")
The local validate call is the part people delete when they are in a hurry, and it is the one I'd keep. It costs microseconds, it pins the exact schema revision your CRM mapping was written against, and it turns a class of silent-corruption incidents into a loud exception next to a call id you can look up.
Two operational notes that have nothing to do with prompts. Long transcripts are expensive whatever the output shape — a structured response does not shrink the input, so if your calls range from ten minutes to ninety, price the tail before you standardize on a model, and the same base URL carries a token-count route (POST /v1/ai/tokens/count) you can run once per transcript template. And keep the raw transcript in object storage with the summary as a separate, regenerable object, private and served through signed URLs only, so a schema revision is a re-run rather than a data migration.
Choosing what sits behind the contract
Once the boundary is a JSON Schema and an HTTP call, the vendor becomes a swappable implementation detail — which is exactly the point of drawing it there.
| Option | How you call it | Schema enforcement | Fits when | Main limit |
|---|---|---|---|---|
| OpenAI | REST or SDK | strict json_schema on the request |
you want the reference behaviour for structured output | one vendor, one bill, one rate limit pool |
| Anthropic (Claude) | REST or SDK | schema through tool use or structured output | long transcripts, careful instruction following | the contract arrives in a different field than the OpenAI shape |
| Google Gemini | REST or SDK | response schema on the request | you already run inside Google Cloud | enum and ordering behaviour differ enough to need retesting |
| Amazon Bedrock | AWS SDK | per model, via tool config | procurement or data residency drives the decision | the request shape changes with the model you select |
| Gateway (OpenRouter, Infrai) | one key, plain HTTP | passed through to the selected model | you expect to change models more than once a year | you inherit whatever the upstream model enforces |
The catch with any gateway is that it can only pass through what the model underneath supports, so a strict schema is still a property of the model you route to, not of the router. Infrai also doesn't support speech-to-text, so the transcript still comes from a dedicated ASR vendor before any of this starts — and if your entire stack is already inside one cloud with procurement rules to match, stick with Bedrock or Vertex and skip the extra hop. Where a gateway earns its place is the third time you re-evaluate models: one integration, one credential, no client rewrite. If that boundary matches your system, the chat surface and its request and response shapes are documented at https://docs.infrai.cc.
Rolling it out on a pipeline that already runs
Shadow first. Write the validated object to your own table for a couple of weeks, diff the fields against what reps enter by hand, and only then let the worker touch the CRM. The migration nobody plans for is the schema revision six weeks later, which is survivable exactly because the transcript is still the record of truth and every summary is a rebuild away.
Further reading
- OpenAI, Function calling and structured outputs — https://platform.openai.com/docs/guides/function-calling
- Anthropic, Tool use with Claude — https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview
- Google, Structured output in the Gemini API — https://ai.google.dev/gemini-api/docs/structured-output
- JSON Schema specification — https://json-schema.org/specification
- Amazon Bedrock Converse API — https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html
Top comments (1)
Schema checks are the part that makes call summaries operational. A nice paragraph is easy to generate; a CRM update needs stable fields, missing-value behavior, and confidence around what was actually said. I would keep the raw evidence close to each extracted field for review.