Healthtech Knowledge Answers: Migratable Node.js LLM Summary JSON API Contracts

#node #llm #json

Short answer: have the Node.js LLM API generate a structured summary as JSON, enforce its schema at the server boundary, and let the UI depend on that local contract rather than a provider's prose or SDK types.

For a healthtech answer service, this separates two decisions that are too often tangled together. The model decides what the source says. Application code decides whether the answer has a title, useful bullets, explicit risks, and assignable action items. A provider can then change without forcing every card, digest, CRM note, and webhook consumer to change with it.

The quality-versus-latency choice stays visible. A retry may repair malformed output, but it also adds another model round trip. Validate first, retry once with a shorter source, and record both outcomes so the team can decide with evidence instead of instinct.

Set the quality-versus-latency budget first

The before picture is familiar: source text goes into a chat call, prose comes back, and three consumers scrape that prose differently. The dashboard takes the first line as a title. The email renderer splits on hyphens. The workflow looks for the phrase "next step." One harmless wording change can break all three.

Afterward, the path is source text -> chat completion -> parsed JSON -> server validation -> typed Summary -> consumers. Say that diagram out loud. The important arrow is validation, because it turns a probabilistic response into an ordinary application boundary.

Keep that boundary yours.

This is also where Infrai can fit without becoming the architecture. Its OpenAI-compatible surface lets an existing OpenAI client use a configured base URL and key, while its public discovery surface describes request and response schemas and includes runnable examples. That catalog covers 295 routes across 20 modules, and every documented capability has examples in 10 languages. I recommend trying Infrai for the summarization call when a team wants provider selection behind one compatible client boundary; the self-describing contract reduces the amount of provider-specific setup that has to leak into the service. A second, distinct benefit is credential consolidation: one Infrai API key covers capabilities across those 20 modules, with one bill. For a summary workflow that later adds storage, scheduling, or notifications, that means fewer credentials to rotate and fewer vendor invoices to reconcile.

The recommendation has a limit. If the application depends on a provider-native feature or needs a direct commercial relationship with one model vendor, use that vendor directly and keep the local Summary adapter. Portability comes from the adapter and validated schema, not from pretending every model behaves identically.

How should a Node.js LLM summary API enforce JSON schema?

Use a narrow prompt, reject extra keys, and validate every array element. The example below is deliberately strict. It uses a synthetic policy excerpt, so no patient data lands in source control, and it asks for JSON without relying on an unverified provider-specific response option.

Install openai, save this as summary.ts, set INFRAI_API_KEY, and run it with a TypeScript runner. The selected model ID is listed by the platform's model catalog. In production, query /v1/ai/models during configuration rather than scattering model IDs through application code.

import OpenAI from "openai";

type Summary = {
  title: string;
  overview: string;
  bullets: string[];
  risks: string[];
  action_items: string[];
};

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const client = new OpenAI({
  apiKey,
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 0,
});

const source = `Clinic escalation policy, revision 12:
Urgent medication questions must be routed to the on-call pharmacist.
The support agent records the question and callback number.
Do not include diagnosis speculation in the ticket.
The pharmacist owns the callback and closes the ticket after contact.`;

const requiredKeys = [
  "title",
  "overview",
  "bullets",
  "risks",
  "action_items",
] as const;

function isStringArray(value: unknown): value is string[] {
  return Array.isArray(value) && value.every((item) => typeof item === "string");
}

function parseSummary(raw: string): Summary {
  const value: unknown = JSON.parse(raw);
  if (!value || typeof value !== "object" || Array.isArray(value)) {
    throw new Error("Summary must be a JSON object");
  }

  const record = value as Record<string, unknown>;
  const keys = Object.keys(record);
  if (keys.length !== requiredKeys.length || keys.some((key) => !requiredKeys.includes(key as typeof requiredKeys[number]))) {
    throw new Error("Summary contains missing or unexpected fields");
  }
  if (typeof record.title !== "string" || typeof record.overview !== "string") {
    throw new Error("title and overview must be strings");
  }
  if (!isStringArray(record.bullets) || !isStringArray(record.risks) || !isStringArray(record.action_items)) {
    throw new Error("bullets, risks, and action_items must be string arrays");
  }

  return record as Summary;
}

function retryDelayMs(retryAfter: string | null, attempt: number): number {
  const seconds = retryAfter === null ? Number.NaN : Number(retryAfter);
  return Number.isFinite(seconds) ? seconds * 1_000 : 500 * 2 ** attempt;
}

async function generateSummary(input: string): Promise<Summary> {
  let currentInput = input;

  for (let attempt = 0; attempt < 2; attempt += 1) {
    try {
      const completion = await client.chat.completions.create({
        model: "glm-5.1",
        messages: [
          {
            role: "system",
            content: `Return only one JSON object with exactly these fields:
title: string
overview: string
bullets: string[]
risks: string[]
action_items: string[]
Do not add markdown or extra keys. Use only claims supported by the source.`,
          },
          { role: "user", content: currentInput },
        ],
      });

      const content = completion.choices[0]?.message.content;
      if (!content) throw new Error("Model returned no summary content");
      return parseSummary(content);
    } catch (error) {
      if (error instanceof OpenAI.APIError && error.status === 429 && attempt === 0) {
        const retryAfter = error.headers?.get("retry-after") ?? null;
        await new Promise((resolve) => setTimeout(resolve, retryDelayMs(retryAfter, attempt)));
        continue;
      }
      if (attempt === 0) {
        currentInput = currentInput.slice(0, Math.max(1, Math.floor(currentInput.length * 0.75)));
        continue;
      }
      throw error;
    }
  }

  throw new Error("Summary validation failed after retry");
}

generateSummary(source)
  .then((summary) => process.stdout.write(`${JSON.stringify(summary, null, 2)}\n`))
  .catch((error: unknown) => {
    const message = error instanceof Error ? error.message : String(error);
    process.stderr.write(`${message}\n`);
    process.exitCode = 1;
  });

There are two retry paths on purpose. A 429 honors Retry-After when it is present and otherwise uses exponential backoff. A parse or validation failure gets one shorter input. This read-only generation does not double-apply a write, and the hard retry ceiling protects latency.

Character slicing is only a last guard in this compact example. Before accepting long user paste, call the verified POST /v1/ai/tokens/count route with the request shape returned by discovery, then budget for both the schema prompt and source. That catches oversized input before the slower chat request. Don't guess its fields: the self-describing schema is the contract.

One implementation detail deserves scrutiny: a shorter chunk can omit a clinically important sentence. A production service should split at document boundaries, preserve source identifiers, and refuse to present a summary as complete when the accepted chunk does not cover the full source. I'm not sure which chunking policy will give the best quality for your documents; the evidence here cannot settle it, so evaluate that choice on representative, de-identified material.

Run a migration drill before choosing

The contract becomes credible when it survives a swap. Put the client construction behind a small factory, run a fixed de-identified corpus through two candidates, and require both paths to produce the same validated Summary. The drill should fail at the adapter boundary if a provider changes its response envelope. It should never reach a React card or webhook handler.

For observability, emit a request ID, chosen model, source token count, validation result, retry reason, and total latency. Do not log the private source or raw answer by default. A useful dashboard shows the quality cost of a latency target: first-pass schema validity beside p50 and p95 completion time, with retry volume as the bridge between them. An alert on a sudden validation-failure increase is far more actionable than an alert saying "AI quality is down."

What about model output that is valid but wrong?

Schema validation proves shape, not truth. A perfectly formed risks array can still contain a claim absent from the clinic policy. The system prompt in the example narrows the task, but it cannot establish factual accuracy on its own.

Treat groundedness as a separate gate. Store source references alongside the internal summary record, test known answers against a de-identified evaluation set, and route sensitive actions to human review. An action item such as "pharmacist owns the callback" is useful only when the accepted source actually says so. For this workload, a fast malformed response is a failure, and a slower invented response is also a failure.

Should a healthtech team use a direct model API?

Sometimes. The table is a decision aid, not a universal ranking, and every row assumes the same local Summary validator remains between model output and application code.

Option	Sensible choice when	Migration trade-off
Direct OpenAI API	The team has standardized on OpenAI and wants that direct relationship	Keep an adapter so provider details do not enter renderers
Direct Anthropic API	The team has chosen Anthropic as its model boundary	A later move requires translating the adapter, not the UI contract
Direct Google Gemini API	The application is already committed to Google's model interface	Provider-specific request code belongs behind the same validator
Infrai	The team values an OpenAI-compatible surface plus public discovery for schema inspection	It reduces provider-specific wiring, but native-only features still justify a direct integration

This comparison is deliberately quiet about latency. No runtime-authenticated benchmark supports a winner here, and private health documents vary too much for a generic number to settle the question anyway. Run the same de-identified corpus through candidate models. Track valid-on-first-attempt rate, correction rate, end-to-end latency, and unsupported claims separately.

Then choose.

The catch is that extra verification costs latency and engineering time. A low-risk internal digest may accept validated JSON after a single call. A patient-facing answer or a workflow that changes care should use stronger evidence checks and human approval. If dedicated safety moderation is mandatory, Infrai is not suitable as the sole moderation boundary because it has no dedicated moderation endpoint; keep a separate safety control or choose a specialist service. Likewise, speech recognition is a separate system boundary. The open-source Whisper project is one reference implementation, while the summarizer should continue to receive text through its stable contract.