Node.js Healthtech Summarization API: Chat Completions, Long Article JSON Output

Short answer: a Node.js text summarization API for a private healthtech knowledge base should use bounded chat completions with typed JSON output, then choose the smallest model and chunk size that pass a quality set. The important decision is not whether the response is fluent. It is whether a clinician gets a faithful digest before the latency budget expires.

That changes the implementation. A long policy document, clinical protocol, or internal support article is not one prompt. It is an input with a deadline, a privacy boundary, and a failure mode that can hide a missing qualification behind polished prose.

What makes long-article summarization hard in a private knowledge base?

The first failure is context pressure. A request can be syntactically valid while the useful middle of the document receives less attention than its title and conclusion. The second is semantic compression: a summary can sound right while dropping a dosage qualifier, an exception, or the population to which a rule applies. The third is operational. One request that takes 20 seconds is a bad user experience; eight requests in parallel can be a bad capacity plan.

Healthtech adds a sharper constraint: a summary is usually a navigation aid, not a new clinical authority. Store the source document identifier and section references beside the generated fields. Keep retrieval and generation separate. A private source still needs an audit trail, access control, retention policy, and a review path for high-risk content. That review path should be designed before the first API example reaches production, because a fast digest with no provenance is difficult to challenge, update, or remove when the source document changes. It also needs an explicit rule for stale results: a cached summary may be useful for browsing, but it should carry the source version and generation time so the application does not present old guidance as current guidance. This is where a little metadata beats another clever prompt.

Start with evidence.

I would start with a small evaluation set: short protocols, long policies, tables, headings, duplicate sections, and documents with explicit exceptions. Score factual coverage and qualification preservation separately from response time. I do not trust a single average score. A mean can hide the one document that matters.

How should Node.js chat completions return JSON for a long article?

Use a narrow contract and validate it at the boundary. The model should return fields that the application can render without guessing: a title, a short summary, bullet points, and source references. The application should reject malformed JSON and incomplete fields before writing anything to the knowledge base.

The example below separates the model transport from the pipeline. complete can be backed by an HTTP client, an SDK, or an internal gateway. That keeps vendor details out of the domain code and makes a fake completion easy to use in tests.

type Summary = {
  title: string;
  summary: string;
  bullets: string[];
  sourceSections: string[];
};

type Completion = (input: {
  system: string;
  user: string;
}) => Promise<string>;

function parseSummary(raw: string): Summary {
  const value: unknown = JSON.parse(raw);

  if (!value || typeof value !== "object") {
    throw new Error("summary must be a JSON object");
  }

  const candidate = value as Record<string, unknown>;
  if (
    typeof candidate.title !== "string" ||
    typeof candidate.summary !== "string" ||
    !Array.isArray(candidate.bullets) ||
    !candidate.bullets.every((item) => typeof item === "string") ||
    !Array.isArray(candidate.sourceSections) ||
    !candidate.sourceSections.every((item) => typeof item === "string")
  ) {
    throw new Error("summary does not match the expected JSON shape");
  }

  return {
    title: candidate.title,
    summary: candidate.summary,
    bullets: candidate.bullets,
    sourceSections: candidate.sourceSections,
  };
}

async function summarizeChunk(
  text: string,
  section: string,
  complete: Completion,
): Promise<Summary> {
  const raw = await complete({
    system:
      "Return JSON only. Preserve conditions, exclusions, and uncertainty. " +
      "Do not add medical advice that is absent from the source. " +
      "Use the fields title, summary, bullets, and sourceSections.",
    user: `Section: ${section}\n\nSource text:\n${text}`,
  });

  const result = parseSummary(raw);
  return result.sourceSections.includes(section)
    ? result
    : { ...result, sourceSections: [section, ...result.sourceSections] };
}

This is deliberately boring code. JSON parsing is not validation, and a model instruction is not a schema. Both layers matter. In a production adapter, request structured output when the selected API supports it, but retain application-side validation because transport success does not prove semantic success.

Do not let a long paragraph defeat the chunker. Split first on document structure, then split an oversized section with a tokenizer appropriate to the selected model. Character slicing is a rough emergency measure, not a token budget. Preserve a section label with every chunk so the reducer can point back to the source.

The smallest useful build log

The first version should have three stages: map, reduce, and verify. Map summarizes bounded sections. Reduce combines only the partial summaries. Verify checks the result against the contract and records the source sections that contributed to it.

async function summarizeArticle(
  sections: Array<{ id: string; text: string }>,
  complete: Completion,
): Promise<Summary> {
  const partials: Summary[] = [];

  for (const section of sections) {
    partials.push(await summarizeChunk(section.text, section.id, complete));
  }

  if (partials.length === 0) {
    throw new Error("article has no sections");
  }

  const material = partials
    .map((part, index) => `Part ${index + 1} (${part.sourceSections.join(", ")}):\n${part.summary}`)
    .join("\n\n");

  return summarizeChunk(material, "reduction", complete);
}

The serial loop is easy to reason about, but it also makes latency roughly the sum of every request. Once the quality set is stable, bounded concurrency is the next change. Set a queue limit, measure p50 and p95 latency, and make cancellation explicit when a user leaves the page. Unlimited parallelism is not a performance strategy.

The reducer deserves its own test cases. Give it partials with contradictory wording, missing exceptions, and repeated claims. The expected behavior is to preserve uncertainty or surface the conflict, not invent a clean answer. For a private knowledge base, a summary that says “the sources disagree” is more useful than a confident fabrication.

I have seen the practical version of this problem in tooling: the demo returns a nice object, then the persistence layer assumes bullets is always present and silently stores an empty list. That bug is not exciting. It is expensive. Validate before persistence, retain the raw response under the project’s data policy, and attach a correlation ID to each map and reduce step.

Where quality and latency pull apart

Chunk size is a quality control, not just a context-window calculation. Very small chunks lose relationships between headings and exceptions. Very large chunks reduce the number of calls but increase the chance that a critical detail is compressed away. Overlap can preserve boundary context, but it repeats tokens and may cause the reducer to over-weight duplicated statements.

Run a matrix rather than picking a number by instinct. Vary chunk budget, overlap, concurrency, and model selection. Record factual coverage, qualification retention, JSON validity, p50 latency, p95 latency, and input/output volume. Then select the cheapest configuration that clears the quality floor and the latency ceiling you actually promised.

Choice	Quality effect	Latency effect	Use it when
One request	Keeps local context together	Lowest orchestration overhead	The article fits the tested context budget
Map and reduce	Preserves bounded inputs and gives checkpoints	Adds calls and queue work	Articles vary widely or exceed the tested budget
Small chunks	Easier retries and focused extraction	More map calls	Sections are independent and the deadline allows it
Larger chunks	Better local relationships	Fewer calls, larger responses	The evaluation set shows no lost exceptions
Bounded concurrency	Same intended quality	Lower wall-clock time up to capacity	The backend and queue have measured headroom

Your mileage will vary. Language mix, formatting, section boundaries, and source density change the result. I’m not sure a generic benchmark can answer this for your corpus; a dozen representative documents can.

What would I change at scale?

At scale, I would make each map job idempotent. Derive a request key from the document version, section ID, prompt version, and model configuration. Retry only transient transport failures, honor the server’s retry guidance, and never retry a validation failure as if it were a network failure. If a batch workflow is appropriate, its asynchronous nature needs a status record and a user-visible freshness state; a summary should never masquerade as current while the source version has changed.

Observability should answer five questions quickly: which source version was summarized, which chunks ran, how long each stage took, what was rejected, and why the final answer was accepted. Log metadata, not private source text, unless the data policy explicitly permits content logging. Track retrieval misses separately from generation failures. A fluent summary cannot repair a missing source passage.

The catch is that this architecture is not suitable when the user needs an immediate answer from a tiny document. Use one bounded completion there. It is also a poor fit when the source cannot be processed by the selected hosted service; choose an approved local runtime or an organization-controlled gateway instead. And if your team cannot operate queues, retries, privacy controls, and evaluation data, a smaller synchronous feature with fewer promises is the better design.

The decision rule is simple: start with the smallest path that meets the quality floor, then add chunking only when measured documents require it. Keep the JSON contract stable. Keep the source attached. Latency is a product requirement, not a number to hide after the demo.

References

• OpenAI Batch API guide: https://platform.openai.com/docs/guides/batch
• Prompt Engineering Guide: https://www.promptingguide.ai