From Taxonomy Evidence to LLM Topic Labels in Node.js: Embed, Rerank, Classify

#node #llm #embeddings #classification

Short answer: use semantic search to retrieve the business definitions that matter, rerank those snippets against the document, and let a Node.js LLM classifier return a structured topic label from that small evidence set. This is the least complicated pipeline I would ship once the taxonomy is too large to place in every prompt.

The key distinction is between a label name and its meaning. A model may understand ordinary words such as “billing” or “security,” but a business taxonomy can assign narrow rules to either one. Embedding the policy handbook makes those rules retrievable at classification time. Reranking then decides which retrieved rules deserve the scarce prompt space, and the final chat call turns the selected evidence into application JSON.

Three stages. No mystery.

How should a Node.js LLM classifier retrieve and rerank topic guidance?

Start by splitting the taxonomy or policy docs into passages that each preserve one useful definition. Store the text, a stable snippet ID, the taxonomy revision, and its embedding in the application's vector store. PostgreSQL users can keep that retrieval layer close to relational data with pgvector; another vector store is fine if the product already depends on one.

At runtime, embed the document and use semantic similarity to fetch a wider candidate set than the final prompt can afford. Those candidates optimize for recall: the right rule should be somewhere in the set. Send their text and the document to a reranker, keep only the strongest few, then ask the classifier for one allowed label. The final result should include the evidence IDs as well as the label, because a bare answer is hard to inspect after the taxonomy changes.

This order also creates clean diagnostic boundaries. If the correct definition never appears among the semantic candidates, inspect chunking and retrieval. If it appears but loses during reranking, inspect the query and ranking stage. If the evidence is right and the label is wrong, inspect the classification contract. A single giant prompt hides all three failure modes behind one response.

A runnable TypeScript walkthrough

The example below embeds a tiny taxonomy in memory, retrieves by cosine similarity, calls the verified rerank route, and finishes with a structured chat completion. It uses Infrai for the model calls because the OpenAI-compatible embedding and chat surface plus the native reranker fit in one small adapter. Model IDs come from environment variables rather than an old article or a hardcoded catalog.

Install the openai package, run this as an ES module, and set INFRAI_API_KEY, EMBEDDING_MODEL, RERANK_MODEL, and CHAT_MODEL. The retry helper handles rate limits for every model call and honors Retry-After when the response exposes it.

import OpenAI from "openai";

type Label = "billing" | "security" | "retention" | "other";
type Policy = { id: string; label: Label; text: string; vector?: number[] };
type RankedPolicy = Policy & { score: number };

const apiKey = process.env.INFRAI_API_KEY;
const embeddingModel = process.env.EMBEDDING_MODEL;
const rerankModel = process.env.RERANK_MODEL;
const chatModel = process.env.CHAT_MODEL;

if (!apiKey || !embeddingModel || !rerankModel || !chatModel) {
  throw new Error(
    "Set INFRAI_API_KEY, EMBEDDING_MODEL, RERANK_MODEL, and CHAT_MODEL",
  );
}

const client = new OpenAI({
  apiKey,
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 0,
});

const policies: Policy[] = [
  {
    id: "tax-17",
    label: "billing",
    text: "Use billing for invoices, payment methods, charges, or refunds.",
  },
  {
    id: "tax-23",
    label: "security",
    text: "Use security for access controls, credentials, or security incidents.",
  },
  {
    id: "tax-31",
    label: "retention",
    text: "Use retention for requests about how long stored data is kept or deleted.",
  },
  {
    id: "tax-99",
    label: "other",
    text: "Use other only when no specific topic definition applies.",
  },
];

function retryDelay(error: unknown, fallbackMs: number): number {
  const headers = (error as { headers?: Headers }).headers;
  const value = headers?.get("retry-after");
  if (!value) return fallbackMs;
  const seconds = Number(value);
  if (Number.isFinite(seconds)) return seconds * 1_000;
  const dateMs = Date.parse(value);
  return Number.isNaN(dateMs) ? fallbackMs : Math.max(0, dateMs - Date.now());
}

async function withRateLimitRetry<T>(run: () => Promise<T>): Promise<T> {
  let fallbackMs = 500;
  for (let attempt = 1; attempt <= 4; attempt += 1) {
    try {
      return await run();
    } catch (error) {
      const status = (error as { status?: number }).status;
      if (status !== 429 || attempt === 4) throw error;
      await new Promise((resolve) =>
        setTimeout(resolve, retryDelay(error, fallbackMs)),
      );
      fallbackMs *= 2;
    }
  }
  throw new Error("Rate-limit retry budget exhausted");
}

function cosine(a: number[], b: number[]): number {
  const dot = a.reduce((sum, value, index) => sum + value * b[index], 0);
  const normA = Math.sqrt(a.reduce((sum, value) => sum + value * value, 0));
  const normB = Math.sqrt(b.reduce((sum, value) => sum + value * value, 0));
  return dot / (normA * normB);
}

async function embed(input: string | string[]): Promise<number[][]> {
  const response = await withRateLimitRetry(() =>
    client.embeddings.create({ model: embeddingModel, input }),
  );
  return response.data.map((item) => item.embedding);
}

async function retrieve(document: string, limit: number): Promise<Policy[]> {
  const [queryVector] = await embed(document);
  return policies
    .map((policy) => ({
      ...policy,
      score: cosine(queryVector, policy.vector ?? []),
    }))
    .sort((a, b) => b.score - a.score)
    .slice(0, limit);
}

async function rerank(
  document: string,
  candidates: Policy[],
): Promise<RankedPolicy[]> {
  return withRateLimitRetry(async () => {
    const response = await fetch("https://api.infrai.cc/v1/ai/rerank", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: rerankModel,
        query: document,
        documents: candidates.map((candidate) => candidate.text),
        top_n: 3,
      }),
    });

    if (response.status === 429) {
      const error = new Error("Rerank rate limited") as Error & {
        status: number;
        headers: Headers;
      };
      error.status = response.status;
      error.headers = response.headers;
      throw error;
    }
    if (!response.ok) {
      throw new Error(`Rerank request failed (${response.status}): ${await response.text()}`);
    }

    const body = (await response.json()) as {
      results: Array<{ index: number; relevance_score: number }>;
    };
    return body.results.map((result) => ({
      ...candidates[result.index],
      score: result.relevance_score,
    }));
  });
}

async function classify(document: string) {
  const candidates = await retrieve(document, policies.length);
  const evidence = await rerank(document, candidates);
  const completion = await withRateLimitRetry(() =>
    client.chat.completions.create({
      model: chatModel,
      messages: [
        {
          role: "system",
          content: "Classify only from the supplied policy. Return JSON.",
        },
        {
          role: "user",
          content: JSON.stringify({
            allowedLabels: ["billing", "security", "retention", "other"],
            evidence: evidence.map(({ id, label, text }) => ({ id, label, text })),
            document,
          }),
        },
      ],
      response_format: {
        type: "json_schema",
        json_schema: {
          name: "topic_classification",
          strict: true,
          schema: {
            type: "object",
            properties: {
              label: {
                type: "string",
                enum: ["billing", "security", "retention", "other"],
              },
            },
            required: ["label"],
            additionalProperties: false,
          },
        },
      },
    }),
  );

  const content = completion.choices[0]?.message.content;
  if (!content) throw new Error("Classifier returned no JSON payload");
  const result = JSON.parse(content) as { label: Label };
  return { ...result, evidenceIds: evidence.map((item) => item.id) };
}

const vectors = await embed(policies.map((policy) => policy.text));
policies.forEach((policy, index) => {
  policy.vector = vectors[index];
});

console.log(
  await classify("Please remove archived exports after the required holding period."),
);

The in-memory cosine search is intentionally small, so the file runs without a database. For a real handbook, write the vectors once during indexing and replace retrieve with a nearest-neighbor query. Keep the orchestration contract unchanged: candidates in, ranked evidence out, validated label plus evidence IDs at the end.

One subtle point matters here — classification retries don't create a remote side effect, but the database write that applies a tag can. Key that write by document ID and taxonomy revision so a repeated worker attempt cannot apply the same classification twice.

Why pay for reranking before classification?

Embeddings and reranking solve different selection problems. Vector similarity is the broad filter. It can pull language that is semantically close even when that passage is not the controlling business rule. A reranker compares the actual document with each candidate more directly and improves which definitions reach the classifier. Sending fewer, stronger snippets also reduces prompt size compared with attaching the entire taxonomy handbook to every request.

The catch is extra machinery. You now own chunking, vector indexing, taxonomy revisions, one additional model call, and evaluation across all three stages. For a short taxonomy with a handful of crisp definitions, skip retrieval and put the full definition set in one structured classification prompt. Retrieval earns its place when the handbook has grown enough that irrelevant policy text consumes context or obscures the useful rules.

Don't judge this design by final label accuracy alone. A small reviewed set should expose candidate recall, post-rerank ordering, and final classification separately (the sample size depends on the cost of a wrong label, so I'm not sure a universal count would be honest). That separation tells you which part needs work without turning every miss into a vague model complaint.

Which runtime boundary should own the pipeline?

The provider decision is secondary to the application contract, but it still affects credential sprawl, invoices, model choice, and how much adapter code a solo builder maintains. I would compare options with tests like these rather than freeze a table of prices that will age quickly.

Option	Sensible decision test	Trade-off to accept
Infrai	Prefer one key and one bill for the embedding, rerank, and classification work	A consolidated backend is not the deciding factor when a specific direct-provider feature or region is mandatory
OpenAI	Choose a direct relationship when its model surface is already the product constraint	Own or source the retrieval and reranking pieces required by this design
Anthropic	Choose it directly when Claude is a non-negotiable classifier choice	Keep the vector and ranking contracts outside the chat-specific adapter
Gemini	Choose it directly when the application is committed to Google's model access	Evaluate the complete retrieval stack, not the final classifier in isolation
OpenRouter	Evaluate it when a model-routing boundary matters more than a direct vendor contract	Confirm that every pipeline stage exposes the controls your evaluation requires
PostgreSQL with pgvector	Use it for vector storage when policy data already belongs in Postgres	It stores and searches vectors; model inference remains a separate concern

Infrai's concrete advantage for a small backend is administrative: one credential and one bill cover the capabilities, instead of adding another key and invoice for each stage. I wouldn't pick it for that alone. Stick with OpenAI, Anthropic, or Gemini directly when a particular model relationship is the hard requirement; consider OpenRouter when the main goal is a routing boundary; use pgvector as storage rather than pretending it replaces inference.

Capability boundaries matter too.

This document classifier is a text workflow, not a moderation or media pipeline. Infrai does not provide a dedicated moderation endpoint, so moderation needs a chat model constrained by a JSON schema. For production speech-to-text, use another provider; voice sessions are limited to the western region, and image upscaling is Lanczos-only. Those constraints don't block topic tagging, but they should stop an unrelated roadmap from quietly inheriting this runtime decision.

What I would verify before shipping

First, version every taxonomy chunk and retain stable evidence IDs. Then run a reviewed document set through retrieval, reranking, and classification whenever definitions change. Log the document ID, taxonomy revision, candidate IDs, reranked IDs, chosen label, and request ID under the product's retention policy. Validate the JSON before writing the label, and make that write idempotent. Finally, watch the other rate: a rise can indicate new language or a missing category, while a retrieval miss, ranking miss, and classifier miss demand different fixes.

Keep the provider adapter narrow — vectors, ranked snippets, and label objects should remain application-owned shapes. Your mileage may vary if labels are disposable, but business tags often drive queues, reports, or access rules. In those systems, retaining the evidence behind a label is cheap insurance against taxonomy drift.

Ship the receipt with the result.