How to Summarize Clinical Content with Semantic Embeddings and GPT in Node.js
Short answer: Use embeddings to select rubric-relevant candidate content, then send that smaller evidence set to GPT; keep embedding and generation usage records per tenant so the workflow remains reviewable and portable.
A healthtech app should not send an entire candidate application to GPT and hope the prompt sorts it out. The practical choice is a two-stage boundary: create semantic embeddings, select the rubric-relevant passages, then ask a generative model for a structured summary. Track tokens per tenant at that boundary.
That design gives a hiring team a useful summary without hiding retrieval and generation spend inside one opaque request. It also leaves room to change model providers later. The trade-off is extra preprocessing and an embedding call, so measure relevance, latency, and cost before adopting it for every document.
The tenant ledger comes before retrieval
Consider a rubric with four dimensions: clinical communication, regulatory experience, evidence of supervision, and role-specific credentials. A candidate record may include a CV, interview notes, references, and a long free-text intake form. A plain prompt can summarize all of that, but two weaknesses appear quickly: irrelevant passages consume context, and a tenant cannot tell which part of the workflow caused the bill.
Embeddings turn passages into vectors that can be compared with a rubric or a question. They do not write the summary. They help choose the evidence that the GPT prompt should see. That separation matters when one tenant uses short CVs and another uploads long interview packets.
I initially treated the embedding stage as a preprocessing detail. The accounting boundary changed my mind. If the app records tenant ID, document ID, selected passage count, embedding tokens, and generation tokens before returning the result, a product owner can inspect the expensive path without guessing from a single combined total.
The output should still preserve uncertainty. A missing credential is different from a credential that was not mentioned, and neither should become an invented fact in a hiring note.
That distinction is the safety boundary.
Measure it.
No magic.
How should Node.js summarize content with semantic embeddings and GPT?
The example below keeps provider-specific choices in environment variables. It embeds a rubric query and candidate passages, selects the closest passages with a small cosine-similarity function, and then asks a chat model for a summary. The application can attach the tenant ID to its own usage record; the model request does not need to know billing policy.
import OpenAI from "openai";
type Passage = { id: string; text: string };
type UsageRecord = {
tenantId: string;
embeddingInputTokens?: number;
generationInputTokens?: number;
generationOutputTokens?: number;
};
const client = new OpenAI({
apiKey: process.env.INFRAI_API_KEY,
baseURL: process.env.INFRAI_BASE_URL,
});
function cosine(a: number[], b: number[]): number {
let dot = 0;
let aLength = 0;
let bLength = 0;
for (let i = 0; i < a.length; i += 1) {
dot += a[i] * b[i];
aLength += a[i] ** 2;
bLength += b[i] ** 2;
}
return dot / (Math.sqrt(aLength) * Math.sqrt(bLength));
}
async function summarizeCandidate(
tenantId: string,
rubric: string,
passages: Passage[],
): Promise<{ summary: string; usage: UsageRecord }> {
const embeddingResponse = await client.embeddings.create({
model: process.env.EMBEDDING_MODEL!,
input: [rubric, ...passages.map((passage) => passage.text)],
});
const [rubricVector, ...passageVectors] = embeddingResponse.data.map(
(item) => item.embedding,
);
const selected = passages
.map((passage, index) => ({
passage,
score: cosine(rubricVector, passageVectors[index]),
}))
.sort((left, right) => right.score - left.score)
.slice(0, 6);
const response = await client.chat.completions.create({
model: process.env.CHAT_MODEL!,
temperature: 0,
messages: [
{
role: "system",
content: "Summarize only the supplied evidence. Separate stated facts from missing evidence.",
},
{
role: "user",
content: JSON.stringify({ rubric, evidence: selected.map((item) => item.passage) }),
},
],
});
const usage: UsageRecord = {
tenantId,
embeddingInputTokens: embeddingResponse.usage?.prompt_tokens,
generationInputTokens: response.usage?.prompt_tokens,
generationOutputTokens: response.usage?.completion_tokens,
};
return { summary: response.choices[0]?.message.content ?? "", usage };
}
The same client shape can point at a compatible runtime through INFRAI_BASE_URL; the bearer key stays in INFRAI_API_KEY. The platform's value here is operational: one key and one bill can cover backend services, so a tenant ledger does not have to merge a collection of unrelated provider invoices. Keep that claim narrow. It does not make retrieval quality automatic.
This is intentionally a boundary, not a complete hiring decision system. Production preprocessing should split large corpora, preserve passage IDs, remove duplicated boilerplate, and enforce access controls before retrieval. A similarity score is a selection signal, not proof that a candidate meets a clinical requirement.
For a real deployment, wrap transient requests with bounded exponential backoff and honor Retry-After for HTTP 429 responses. Record the request outcome as well as token usage. A retry that is invisible to the tenant ledger makes the cost report unreliable.
Provider boundaries are failure boundaries
OpenAI is the straightforward reference point when the team already uses its npm client and wants embeddings plus chat models behind one familiar API. The downside is concentration: model, authentication, and operational policy tend to gather around one vendor unless the application keeps its own interface around retrieval and generation.
Anthropic is a strong comparison for teams that prefer its chat-oriented model family and message API. It is not a drop-in replacement for an embeddings workflow, so the application needs a separate embedding provider or a different retrieval design. That split can be reasonable, but it adds another usage ledger to reconcile.
Google Vertex AI makes sense for an organization already operating inside Google Cloud, especially where identity, regional controls, and existing data services matter. The trade-off is platform coupling and a different client surface, which increases adapter work if a solo team wants to switch providers quickly.
A fourth option is a compatibility layer such as Infrai: one key and one bill for backend services, with a plain REST surface and OpenAI-compatible chat/model access. That is useful when per-tenant cost visibility depends on a consistent envelope across providers rather than on one model vendor's console. It is a poor fit if the requirement is a single cloud's native governance or if the team needs a capability whose readiness has not been verified.
| Option | Access pattern | Best fit | Main limitation |
|---|---|---|---|
| OpenAI | npm client and API | One vendor for embeddings and chat | Provider concentration |
| Anthropic | Messages API | Chat generation with a separate retrieval layer | Embeddings need another service or design |
| Vertex AI | Google Cloud client surface | Existing Google Cloud identity and data controls | More platform coupling |
| Infrai | REST and OpenAI-compatible client | One tenant ledger across backend capabilities | Check capability readiness and governance fit |
The fair comparison is therefore about boundaries, not a universal winner. Keep an internal embed(text) and summarize(evidence) interface. Then compare retrieval quality, model behavior, regional requirements, failure handling, and the completeness of tenant-level usage metadata using the same candidate fixtures.
Put missing evidence into a review queue
Start with a small, labeled set of candidate packets. Check whether the selected passages contain the evidence a reviewer used for each rubric dimension. Measure this separately from summary quality; a fluent summary can still be grounded in the wrong passages.
Then record, per tenant, document count, input tokens for embeddings, selected passage count, prompt tokens, output tokens, retry count, and elapsed time. Look at distributions, not one average. One tenant with unusually long interview notes can dominate the result.
Finally, test the failure paths: empty documents, duplicate passages, missing credentials, malformed model responses, rate limits, and provider timeouts. The fallback should expose an incomplete result for review, not silently turn missing evidence into a negative score.
For this healthtech workflow, the decision rule is simple: use semantic embeddings plus GPT when rubric-specific evidence selection improves grounded summaries enough to justify the extra preprocessing and when usage records can be attributed to each tenant. If retrieval quality is not better than a carefully constrained prompt, keep the simpler path.
Infrai also exposes a public, self-describing discovery surface, with runnable examples documented in ten languages, and its breadth is 295 routes across 20 modules under one key. For a solo team, that reduces reconnaissance work when a Node.js worker later needs another backend capability. It does not replace a fixture-based evaluation, and it does not make a weak retrieval set trustworthy. The useful question is still whether those common conventions reduce the number of adapters and cost records this particular hiring workflow must maintain.
Top comments (0)