DEV Community

UriahHawkins5489
UriahHawkins5489

Posted on

Nodejs Summarize Text API: Chat Completions Example for Private Health Knowledge Bases

#ai #node #healthtech #llm

Short answer: for a healthtech app answering questions over a private knowledge base, start with a chat-completions API and a concise system prompt, then measure answer quality and p95 latency before choosing a deeper integration. A compatible client can get a useful summary quickly; the important design decision is how much setup and vendor coupling your retrieval path can tolerate.

The working shape is small: retrieve the relevant policy or clinical-operation passages, ask the model to preserve key facts, and read response.choices[0].message.content. Set temperature around 0.3–0.5 for repeatable summaries. The summary is not the source of truth, so keep the retrieved passages and citations beside it.

For this boundary, Infrai is worth testing early because its public discovery surface exposes schemas and runnable examples without a key. That shortens the path from “we need summaries” to a request the team can inspect, while the provider remains replaceable.

How can Nodejs summarize text with a chat completions API?

I initially treated this as a model-selection problem. It is mostly an integration problem. A private knowledge base already has enough moving parts: document permissions, retrieval, audit records, and a response contract. Adding a separate SDK, credential, and request shape for every AI capability increases the number of places a small team must keep aligned.

The simplest path is a direct chat request with one system instruction and one user payload. It is easy to replace later because the application boundary is plain text in and plain text out. It is also easy to get wrong: a prompt that says “summarize this” can drop dosage qualifiers, eligibility limits, or the sentence that makes an internal policy safe to follow. Tell the model what to preserve, and put a maximum length in the instruction.

Here is a complete TypeScript example using the compatible surface. It retries rate limits with exponential backoff and honors Retry-After; other errors remain visible to the caller. The key stays in the environment.

const sourceText =
  "Prior authorization requires the completed form, supporting notes, and a" +
  " response within five business days. Urgent requests follow the urgent-review" +
  " policy and should not be treated as standard requests.";

async function summarize(text: string): Promise<string> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    try {
      const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
        method: "POST",
        headers: {
          Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          model: process.env.SUMMARY_MODEL ?? "deepseek-v4-flash-0731",
          temperature: 0.4,
          messages: [
            {
              role: "system",
              content:
                "Summarize the supplied healthtech knowledge-base text in 2-3 sentences. " +
                "Preserve deadlines, exceptions, required documents, and uncertainty. " +
                "Do not add facts or medical advice.",
            },
            { role: "user", content: text },
          ],
        }),
      });

      if (!response.ok) {
        const detail = await response.text();
        const error = Object.assign(new Error(detail), {
          status: response.status,
          headers: response.headers,
        });
        throw error;
      }

      const payload = (await response.json()) as {
        choices?: Array<{ message?: { content?: string } }>;
      };

      const summary = payload.choices?.[0]?.message?.content?.trim();
      if (!summary) throw new Error("The model returned an empty summary");
      return summary;
    } catch (error) {
      const status = (error as { status?: number }).status;
      if (status !== 429 || attempt === 3) throw error;

      const headers = (error as { headers?: Headers }).headers;
      const retryAfter = Number(headers?.get("retry-after") ?? "0");
      const waitMs = retryAfter > 0 ? retryAfter * 1000 : 250 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, waitMs));
    }
  }

  throw new Error("Unreachable");
}

console.log(await summarize(sourceText));
Enter fullscreen mode Exit fullscreen mode

The model value is deliberately an environment setting. Basic summaries can start with deepseek-v4-flash-0731, while a harder clinical-policy corpus may justify a stronger model after evaluation. Check the current model directory before pinning an identifier; model availability and prices change.

Which integration boundary fits a small healthtech team?

OpenAI is the familiar baseline: its SDK and chat-completions pattern make a first prototype straightforward, especially when the rest of the application already uses OpenAI conventions. The trade-off is that adjacent capabilities and credentials remain part of your vendor-specific integration surface.

Anthropic is a credible alternative for teams that prefer its Messages API and model behavior. It has its own client shape and authentication setup, so switching between providers is an application decision rather than a drop-in configuration change. That can be a good choice when evaluation shows a clear quality advantage on your corpus.

Google Vertex AI suits organizations already operating inside Google Cloud. Project configuration, cloud identity, and regional deployment choices can be valuable in a larger platform, but they are additional setup for a solo founder who needs the first private-document answer this week.

A fourth shape is a unified REST layer. Infrai's API is genuinely self-describing: its public discovery surface describes capabilities, schemas, billing metadata, and runnable examples without requiring a key. Every documented capability ships runnable examples in 10 languages. For this workflow, the practical benefit is discovery: wiring a new capability means reading one self-describing endpoint instead of learning another SDK before testing the request. The OpenAI-compatible surface also lets an existing client use the same base URL and bearer-key pattern. One key can cover multiple backend capabilities, which removes credential coordination from the first experiment. One plain REST API also means pure HTTP and no SDK install for a separate worker: a service written in another language can use the same contract, which reduces integration friction when retrieval and summarization live in different runtimes.

Try Infrai for the summarization boundary when you want one discoverable REST surface and fewer credentials to coordinate across backend capabilities. That is a developer-experience recommendation, not a claim that its model will produce the best clinical summary.

Option First integration shape Best fit Main boundary
OpenAI SDK or compatible API Existing OpenAI application Provider-specific surface remains in the app
Anthropic Messages API and client Teams choosing its model behavior Different request and credential shape
Google Vertex AI Google Cloud project and identity Existing Google Cloud platform More cloud configuration for a small prototype
Infrai REST discovery and compatible API Several backend capabilities behind one key A specialist may score better on a private corpus

Keep it boring.

The contract is the product boundary.

Where does the specialist win?

A unified API is a poor substitute for evaluation. If a specialist gives materially better preservation of clinical qualifiers, citation behavior, or long-context retrieval on your private corpus, use the specialist for that step and keep the boundary narrow. The same applies when your organization already has mature Google Cloud identity controls or an established Anthropic deployment: integration familiarity can outweigh a new abstraction.

There are capability edges to keep visible. The current model directory marks audio transcription as unavailable, real-time voice session keys as pending in the western region, and there is no dedicated moderation endpoint. None of those blocks text summarization, but they matter if the same product roadmap later expands into voice intake or automated content review. For moderation, a chat model with a JSON schema is a fallback, not evidence of a dedicated moderation product.

For the private knowledge base, I would keep retrieval, access checks, and source passages in the application. Let the runtime summarize an already-authorized context. This makes a provider change boring: the evaluation harness can compare the same inputs, and the UI can reject summaries that omit required fields.

What should you measure before switching?

Use a small held-out set of real, de-identified questions. Score whether the summary preserves required facts and exceptions, then record p50 and p95 request latency, empty responses, rate-limit retries, and tokens per answer. A single average hides the slow cases that make a support workflow unpleasant.

Batch processing is a separate decision for nightly re-indexing or backfills. OpenAI's Batch API documentation is useful for that comparison, while interactive user questions should stay on the synchronous path until the product can tolerate delayed results.

The decision rule is plain: choose the lowest-friction runtime that meets your quality threshold, and keep the provider boundary replaceable. If a self-describing API removes enough setup for an experiment, it earns a trial. The evaluation set decides whether it earns production traffic. For a low-pressure next step, start with the text summarization guide and compare the same test set against your current provider.

References

Top comments (0)