DEV Community

MaximilianNilsson7568
MaximilianNilsson7568

Posted on

2026 Node.js Summarize Text API Chat Completions Example for Support Answers

#node #ai #customersupport #jsonschema

Short answer: for a support agent answering from a private knowledge base, the difficult part is not producing a shorter paragraph. It is producing a valid, bounded answer that preserves the facts the agent is allowed to use. Put that contract in the prompt and application validation, call a chat-completions endpoint, use a temperature around 0.3 to 0.5, and reject output that cannot be parsed. For basic summarization, gpt-4o-mini is a reasonable starting point.

A summary that drops a refund restriction is a data-layer defect wearing a language-model costume.

How should a Node.js text summarization API use chat completions?

The model should summarize only supplied passages, preserve restrictions and dates, distinguish an explicit answer from missing information, and return fields that a Node.js service can validate before the result reaches an agent. A useful shape is answer, key_facts, missing_information, and source_ids; it is more useful than asking for “a concise summary” and hoping the prose is safe.

The system prompt should say what to do when retrieval does not answer the question. “I do not have enough information in the supplied knowledge base” is a successful result when the alternative is an invented policy. Determinism helps, but it is not correctness. Validation and retrieval boundaries still carry the load.

The request pattern below uses the OpenAI-compatible chat-completions shape, reads the key and endpoint from the environment, checks HTTP status, parses choices[0].message.content, and retries 429 responses with Retry-After. The same body can be sent from a Node.js openai client or with fetch; the contract is the part worth preserving.

import json
import os
import time
from urllib import request, error


def summarize(question, passages):
    payload = {
        "model": "deepseek-v4.1-flash",
        "temperature": 0.4,
        "messages": [
            {"role": "system", "content": (
                "Summarize only the supplied knowledge-base passages. "
                "Preserve restrictions, dates, and exceptions. Return JSON "
                "with answer, key_facts, missing_information, and source_ids. "
                "If the passages do not answer the question, say so."
            )},
            {"role": "user", "content": json.dumps({
                "question": question, "passages": passages
            })},
        ],
    }
    req = request.Request(
        os.environ["CHAT_COMPLETIONS_URL"],
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
            "Content-Type": "application/json",
        },
        method="POST",
    )
    for attempt in range(4):
        try:
            with request.urlopen(req, timeout=30) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"unexpected HTTP status: {response.status}")
                result = json.loads(response.read())
            summary = json.loads(result["choices"][0]["message"]["content"])
            required = ("answer", "key_facts", "missing_information", "source_ids")
            for field in required:
                if field not in summary:
                    raise ValueError(f"missing output field: {field}")
            return summary
        except error.HTTPError as exc:
            if exc.code != 429 or attempt == 3:
                raise
            time.sleep(float(exc.headers.get("Retry-After", 2 ** attempt)))
    raise RuntimeError("summary request exhausted retries")
Enter fullscreen mode Exit fullscreen mode

In production, keep the parsed object behind a schema validator and attach retrieved document identifiers before displaying the answer. Do not treat a model-generated source_ids field as proof of citation; compare it with identifiers actually passed to the model.

The storage-style decision: where does the contract live?

I think about this as a storage problem before I think about it as a prompt problem. The passages are durable inputs, the summary is a derived record, and the contract is the index that makes the record usable. Store the question, retrieval IDs, model name, prompt version, temperature, raw response, parsed response, and validation result together. A text blob called summary will not explain what was supplied when an answer is challenged.

There is a trade-off. Raw responses improve auditability and replay, but increase retention and access exposure; parsed fields reduce exposure, but make prompt regressions harder to investigate. For customer support, retain both under the same access policy and apply retention to the support record. The knowledge base is private, so a public object URL or public-read storage policy is out of bounds.

Streaming improves perceived responsiveness, but partial JSON is not a valid final record. Buffer the stream, parse it once complete, and publish only after validation. A half-emitted answer is a UI event, not a support record.

How do the main API choices differ?

The vendor is only one part of the decision. The output contract, data boundary, operational controls, and client portability matter just as much. These are real options, but their semantics differ, so an adapter should own the differences.

Option Useful fit Constraint
OpenAI chat completions Familiar message API and a practical starting point for basic summaries Validate structured output in the application
Anthropic Messages API Existing Anthropic integration or prompt format Different message contract requires an adapter and separate tests
Google Gemini API Teams already using Gemini models and response configuration Request configuration is not a drop-in chat-completions swap
Infrai OpenAI-compatible surface One key and one bill for backend services, with a plain REST surface Transcription is unavailable, voice sessions are pending and Western-region only, and there is no dedicated moderation endpoint

The last row is a fit judgment, not an endorsement. One credential and billing surface can reduce operational sprawl, but it does not remove capability-readiness checks. For summarization, keep the integration narrow and verify the selected model through the model listing.

I would not choose a provider because one manual response sounds polished. Run a fixed corpus containing refusal cases, missing-policy cases, and exception-heavy answers. Compare schema-valid rate, preserved constraints, and reviewer corrections. Those signals map to the job.

Roll out the summary without corrupting the record

Begin in shadow mode: retrieve the same passages, generate a candidate, validate it, and show it only to reviewers. Log invalid JSON, missing fields, unsupported claims, omitted restrictions, and retrieval mismatches. Five categories make a first dashboard useful without pretending one score explains the system.

Version the prompt and output schema. When either changes, compare new results with stored cases before changing the default. If a request is retried after a timeout, use an idempotency key derived from the question, retrieval snapshot, and prompt version; otherwise one question can produce several records that appear independent.

Keep a human path for incomplete results. The model can summarize evidence; it cannot turn absent policy into policy.

Sources

References used:

Top comments (0)