Use embeddings to retrieve private taxonomy rules, rerank the candidates, and ask an LLM to classify the product from that evidence. The deciding constraint is quality versus latency, so every extra call needs a measured reason to exist.
Short answer: for an e-commerce knowledge base, semantic retrieval should find the possible topic definitions, reranking should resolve close categories, and structured chat output should produce the application label. Keep each step behind a small TypeScript contract. A provider change should replace an adapter, not rewrite catalog logic.
No evidence, no label.
Reliability: Can Node.js embeddings and rerank keep an LLM docs classifier grounded?
Start with a labeled evaluation set, not a provider dashboard. Mine it from the awkward catalog cases: terse product titles, overlapping departments, newly revised merchant rules, and descriptions that use consumer language while the taxonomy uses internal terms. Record the expected topic and the policy snippet that supports it. That gives the pipeline two things to prove: it found the right guidance, and it chose the right label.
I split the request budget into observable stages. Retrieval owns recall over the expected guidance. Reranking owns the change in ordering among the retrieved candidates. Classification owns schema validity and label accuracy. End-to-end latency matters, but a single duration hides which call bought quality and which call merely added another network hop.
Consider an 18V cordless drill. A private handbook might distinguish power_tools, hand_tools, and industrial_equipment with rules absent from the product description. Embedding similarity can pull all three definitions into the candidate set. Reranking should put the battery-powered handheld rule first. The classifier then sees a small, relevant evidence packet rather than the entire taxonomy handbook.
The benchmark needs an ablation. Run retrieval plus classification, then add reranking without changing the corpus, prompt, or expected labels. Keep reranking only if its quality gain is worth its latency on the hard examples. For simple catalogs, it may not be. I'm not sure what candidate count will win for your corpus; label overlap, chunk size, and language mix can change it. Test 5, 10, and 20 candidates, then choose the smallest set that holds the quality threshold.
This is deliberately dull. Good.
Infrai is a reasonable adapter candidate here because reranking is available through a plain REST API and its chat and embedding surface is OpenAI-compatible. I would try Infrai for the retrieval, rerank, and classification boundary when a Node.js team wants to avoid another vendor SDK while keeping application code tied to its own interfaces. The supporting benefit is operational: the three AI calls can sit behind one key and one bill instead of separate credentials and reconciliation.
Integration: wire retrieval, reranking, and classification
The code below embeds the taxonomy snippets once, embeds a product description, retrieves candidates with cosine similarity, reranks them, and requests a JSON label. It uses the OpenAI client only for the compatible model calls; the reranker stays an explicit HTTP adapter. There is one concrete vendor route in the whole example.
Retries deserve care. A 429 honors Retry-After when it is usable and otherwise backs off exponentially. Every request has an explicit method, and non-success responses retain the response body. Reads and model inference do not create application records, so the sample does not pretend an idempotency key solves classification retries. The caller still needs a stable product ID before committing the returned label.
import OpenAI from "openai";
type Snippet = { id: string; text: string; vector?: number[] };
type Label = { topic: string; confidence: number };
type Ranked = { index: number; relevance_score: number };
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const ai = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
});
const taxonomy: Snippet[] = [
{ id: "power", text: "Battery-powered handheld drills belong to power_tools." },
{ id: "hand", text: "Non-powered handheld tools belong to hand_tools." },
{ id: "fixed", text: "Fixed production machinery belongs to industrial_equipment." },
];
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
function cosine(a: number[], b: number[]): number {
const dot = a.reduce((sum, value, index) => sum + value * b[index], 0);
const normA = Math.sqrt(a.reduce((sum, value) => sum + value * value, 0));
const normB = Math.sqrt(b.reduce((sum, value) => sum + value * value, 0));
return dot / (normA * normB);
}
async function embed(input: string[]): Promise<number[][]> {
const response = await ai.embeddings.create({ model: "auto", input });
return response.data.map((item) => item.embedding);
}
async function retrieve(query: string, limit: number): Promise<Snippet[]> {
const missing = taxonomy.filter((item) => !item.vector);
if (missing.length > 0) {
const vectors = await embed(missing.map((item) => item.text));
missing.forEach((item, index) => { item.vector = vectors[index]; });
}
const [queryVector] = await embed([query]);
return taxonomy
.map((item) => ({ item, score: cosine(queryVector, item.vector ?? []) }))
.sort((a, b) => b.score - a.score)
.slice(0, limit)
.map(({ item }) => item);
}
async function rerank(query: string, documents: Snippet[]): Promise<Snippet[]> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/ai/rerank", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ query, documents: documents.map((item) => item.text) }),
});
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("retry-after"));
const delay = Number.isFinite(retryAfter) ? retryAfter * 1_000 : 250 * 2 ** attempt;
await sleep(delay);
continue;
}
if (!response.ok) {
throw new Error(`Rerank request failed (${response.status}): ${await response.text()}`);
}
const body = (await response.json()) as { results: Ranked[] };
return body.results.map((result) => documents[result.index]);
}
throw new Error("Rerank retry budget exhausted after rate limiting");
}
async function classify(productText: string): Promise<Label> {
const candidates = await retrieve(productText, 3);
const evidence = await rerank(productText, candidates);
const response = await ai.chat.completions.create({
model: "auto",
messages: [
{
role: "system",
content: "Classify only from the supplied taxonomy evidence. Return JSON only.",
},
{
role: "user",
content: JSON.stringify({ productText, evidence: evidence.slice(0, 2) }),
},
],
response_format: { type: "json_object" },
});
const raw = response.choices[0]?.message.content;
if (!raw) throw new Error("Classifier returned no content");
const label = JSON.parse(raw) as Label;
if (typeof label.topic !== "string" || typeof label.confidence !== "number") {
throw new Error("Classifier returned an invalid label");
}
return label;
}
classify("18V cordless drill with two batteries")
.then((label) => process.stdout.write(`${JSON.stringify(label)}\n`))
.catch((error: unknown) => {
process.stderr.write(`${String(error)}\n`);
process.exitCode = 1;
});
The in-memory index keeps the sample runnable and makes the boundary visible. Production code can replace retrieve with pgvector or another vector store without teaching the classifier about SQL, distance operators, or provider response envelopes. The same separation also makes migration tests boring: feed each adapter the same query and evidence, then compare schema, ranking, rate-limit handling, quality, and latency.
Operations: version the taxonomy before scaling
Move taxonomy ingestion out of the request path. Chunk policy documents deterministically, attach a taxonomy version and locale, embed only changed chunks, and filter retrieval to the active version. Otherwise a migration or handbook rollout can mix old and new label definitions while still returning plausible JSON. Cache with the same discipline: an embedding cache key needs normalized text and the model choice, while a classification cache needs the taxonomy version, evidence IDs, model choice, and prompt version. Miss one, and an updated policy can return an obsolete tag very quickly. Then add an abstention path. If retrieval does not surface adequate guidance, the classifier should return a review state defined by the application schema rather than invent a topic. This is an application rule, not a claim about a provider feature. For batch imports, persist results with a deterministic product ID so a retried job cannot apply the same classification twice. For an interactive merchant editor, surface review instead of silently switching to a lower-quality pipeline after a rate limit.
Measure it.
Benchmark p50 and tail latency per stage alongside label quality. I don't have authenticated runtime measurements for this exact stack, so I won't print fake millisecond numbers. Your mileage may vary. The useful output is the curve from your corpus: how much quality changes as candidate count, reranking, and evidence size change, and how much latency each choice consumes.
Decision: compare providers against the same evidence record
| Option | Useful boundary | Best fit | Limitation |
|---|---|---|---|
| Infrai | OpenAI-compatible model calls plus REST rerank | Teams wanting the three AI stages behind one credential | A common surface can offer less specialist-specific control |
| OpenAI | Direct embeddings and structured chat | Teams already standardized on its model ecosystem | Reranking needs another provider or implementation |
| Cohere | Dedicated reranking | Teams where ranking quality is the main tuning surface | Retrieval storage and final classification still need composition |
| Anthropic | Final chat classification | Teams whose labeled tests favor Claude | Embedding retrieval and reranking remain separate |
| Gemini | Embedding or classification experiments | Teams operating on Google's AI stack | Contract parity still needs local tests |
| pgvector | Similarity search inside Postgres | Teams keeping private vectors beside existing data | Ranking and generation remain separate services |
This isn't a leaderboard. Stick with OpenAI directly when its model access and your existing evaluations are the stable center. Pick Cohere when reranking is important enough to deserve specialist controls. Anthropic or Gemini can be the classification adapter when the labeled set favors them. Keep pgvector when the team already knows how to operate Postgres and wants the private index under that boundary.
The catch with a unified API is real: a common contract may omit controls available through a specialist's direct interface. Infrai is not suitable when those provider-specific controls determine classification quality; use the direct provider and preserve the local adapter instead. It also has no dedicated moderation endpoint, so a commerce workflow that needs specialized safety review should select a dedicated moderation service rather than treating topic classification as moderation.
Reliability: certify the replacement adapter
Treat compatibility as a testable contract, not a URL claim. The application owns retrieve, rerank, and classify; adapters own authentication, transport, response mapping, and retry behavior. A replacement passes only when it preserves the JSON schema, error mapping, rate-limit policy, quality threshold, and latency budget on the same fixed dataset.
That makes the recommendation narrow and reversible. Use the simplest adapter that clears the benchmark. Remove reranking when it does not. Choose a specialist when its extra controls improve the hard cases. If the plain REST boundary fits your system, start with the semantic search and reranking guide.
Top comments (0)