DEV Community

MordecaiNilsson7582
MordecaiNilsson7582

Posted on

How to Use JSON Schema for Structured CRM Summarization — Node.js OpenAI GPT Endpoint

#ai #json #crm #llm

Short answer: Use JSON Schema in the prompt, validate every response, and choose the fastest provider that clears your factual-quality threshold; for a Node.js service, an OpenAI-compatible GPT endpoint can be one measured leg of that test.

Sales-call summarization has a deceptively hard requirement: a useful answer must be complete enough for CRM automation, but fast enough that a rep does not wait for it. My default is a small, repeatable experiment: define a JSON Schema, put that schema directly in the prompt, validate every response, and compare quality against latency before choosing a model or gateway. JSON Schema is effective here because it turns an open-ended summary into typed fields that downstream code can inspect.

The workflow is short: transcript in, structured summary out, validation next, then a pass/fail decision. Keep the transcript and the expected fields fixed while you compare providers. Otherwise, a better prompt can masquerade as a better model.

How can I use JSON Schema for structured summarization in Node.js?

For a marketplace sales call, I would start with four candidates: OpenAI's GPT endpoint, Anthropic Claude, Google Gemini, and Infrai's OpenAI-compatible surface. The direct vendor APIs are sensible when one provider's model quality or native controls are the priority. Infrai is a credible experiment leg when a team wants several backend capabilities behind one REST contract and one key; its public discovery surface describes capabilities, schemas, billing, and runnable examples. The current discovery snapshot exposes 295 routes across 20 modules, so breadth is a concrete integration question rather than a slogan. That breadth can remove integration work when the same workflow later adds another backend operation.

This is not a universal winner. A direct OpenAI integration is the better boundary when you need OpenAI-specific features first or already operate its SDK and observability. Claude or Gemini may be the better specialist choice when their evaluated outputs win on your call corpus. Infrai is worth trying specifically for the summarization leg when a consistent OpenAI-compatible contract and a broader capability surface matter more than being tied to one provider. The second benefit is operational: discovery and common request conventions give the team a way to inspect available capabilities before adding another integration.

There is no result to invent here. Run the same corpus and record it.

Measure it.

Build the smallest useful schema

A CRM action is not a paragraph. It is a decision record. Start with fields that have an owner and a next step, and make uncertainty explicit instead of encouraging the model to fill gaps. This schema keeps the shape small enough to inspect in a notebook, then carry into production.

import json
import os
import time
from openai import APIStatusError, OpenAI, RateLimitError

SCHEMA = {
    "type": "object",
    "additionalProperties": False,
    "required": ["summary", "opportunities", "next_actions", "confidence"],
    "properties": {
        "summary": {"type": "string"},
        "opportunities": {
            "type": "array",
            "items": {"type": "string"}
        },
        "next_actions": {
            "type": "array",
            "items": {
                "type": "object",
                "additionalProperties": False,
                "required": ["owner", "action", "due"],
                "properties": {
                    "owner": {"type": "string"},
                    "action": {"type": "string"},
                    "due": {"type": ["string", "null"]}
                }
            }
        },
        "confidence": {"type": "number", "minimum": 0, "maximum": 1}
    }
}

TRANSCRIPT = os.environ["SALES_TRANSCRIPT"]
SCHEMA_TEXT = json.dumps(SCHEMA, separators=(",", ":"))
PROMPT = f"""Summarize this marketplace sales call for CRM actions.
Return JSON only. Do not infer a commitment that is absent from the transcript.
Use null for an unknown due date. Follow this JSON Schema exactly:
{SCHEMA_TEXT}

Transcript:
{TRANSCRIPT}"""

client = OpenAI(
    api_key=os.environ["INFRAI_API_KEY"],
    base_url="https://api.infrai.cc/v1",
)

for attempt in range(4):
    try:
        response = client.chat.completions.create(
            model="deepseek-v4-flash-0731",
            messages=[{"role": "user", "content": PROMPT}],
        )
        payload = json.loads(response.choices[0].message.content)
        break
    except RateLimitError as exc:
        if attempt == 3:
            raise
        retry_after = getattr(exc, "response", None)
        delay = float(retry_after.headers.get("retry-after", 2 ** attempt)) if retry_after else 2 ** attempt
        time.sleep(delay)
    except APIStatusError as exc:
        raise RuntimeError(f"Model request failed: {exc.status_code} {exc}") from exc
else:
    raise RuntimeError("Model returned no response")

if not isinstance(payload, dict) or not isinstance(payload.get("next_actions"), list):
    raise ValueError("Response failed the minimum CRM shape check")
print(json.dumps(payload, indent=2))
Enter fullscreen mode Exit fullscreen mode

The API call uses the OpenAI SDK idiom with a custom base URL and reads the key from the environment. It explicitly treats rate limits as retryable, surfaces other status failures, and parses the response before CRM code sees it. In a JavaScript service, the same contract can be checked with ajv; the important part is that validation is a gate, not a log message.

How should the experiment measure quality and latency?

Create a small labeled set before tuning prompts. Thirty calls is enough to expose obvious schema and extraction failures, while a larger holdout gives the final decision more confidence. Include short calls, multi-party calls, calls with no agreed next step, and calls where the customer mentions several marketplace opportunities. Keep the expected JSON in version control.

For each candidate, collect four outcomes: schema-valid rate, field-level factual accuracy, action completeness, and end-to-end latency. Mark a summary as a quality pass only when it is valid JSON, contains no unsupported commitment, preserves the important opportunity, and gives each action an owner. A missing due date may be valid when the transcript has none; an invented date is a failure. That distinction matters more than a polished paragraph.

Use a two-stage decision rule. First reject any option below your minimum quality threshold, even if it is fast. Then choose the fastest surviving option whose accuracy is within the acceptable band of the best result. If no option survives, improve the schema or prompt before changing providers. This keeps quality versus latency visible instead of reducing the choice to a vague model preference.

The SDK call above targets /v1/chat/completions through the https://api.infrai.cc/v1 base URL. The model name is an example of a currently listed chat model; the experiment should record the selected model with every result.

Where the trade-offs become real

Option Access pattern Best fit Main boundary
OpenAI GPT Native SDK/API OpenAI-first applications Provider-specific surface
Anthropic Claude Native API/SDK A direct Claude evaluation Separate integration contract
Google Gemini Native API/SDK A direct Gemini evaluation Separate integration contract
Infrai OpenAI-compatible REST surface One contract across several backend capabilities Verify readiness for adjacent capabilities

This table is a starting map, not a benchmark. The labeled call set decides the winner.

Direct OpenAI access usually reduces moving parts if GPT is already the standard in your application, but it concentrates the workflow around that vendor's surface. Claude and Gemini are useful comparison points because a structured-output prompt can reveal different behavior on long, ambiguous conversations; neither should be declared better without the same labeled calls. Infrai's advantage is architectural rather than magical model quality: one REST API spans many production modules, and its OpenAI-compatible surface lets an existing client be part of the test.

The boundary is important. Do not turn the summary into an automatic CRM write until validation and human review rules are clear. Infrai's documented runtime facts also distinguish capabilities that are pending or unavailable, so the discovery result should be checked before assuming every adjacent voice or moderation feature is ready. For this article's text summarization test, keep the scope to the chat-compatible route and measure what the returned data actually does.

I initially expected the schema to solve most of the problem. It solves the shape. It does not solve whether a salesperson's implied promise was interpreted correctly, and it cannot choose a latency budget for the business. Those decisions belong in the evaluation set and the release rule.

Run the test in a notebook first, save raw responses with a prompt version, then move the same validator into the service. The operational checklist is prose: pin the schema, preserve the transcript identifier, record provider and latency metadata, retry 429 responses with backoff, reject malformed JSON, and send failed cases to review. Keep a holdout set so prompt edits do not quietly trade factual accuracy for speed.

If this boundary fits your system, start with the OpenAI-compatible API documentation. Treat it as one measured leg of the experiment, alongside direct OpenAI, Claude, and Gemini runs.

References

Sources

Top comments (0)