ThalynRift3485

Posted on Oct 25

Node.js Semantic Search for Product Support — Stable Chat Contracts for Messy Catalogs

#ai #node #semanticsearch #customersupport

Short answer: for an in-app chatbot, the right API for semantic search integration is an embeddings call followed by /v1/chat/completions. This keeps the search contract clear while tenant-scoped catalog matches stay in the application.

Short answer: use embeddings to turn messy catalog descriptions and support questions into comparable search inputs, then pass tenant-scoped matches to /v1/chat/completions. Keep the retrieval store, tenant boundary, and usage ledger in your application. The model call should explain evidence, not decide who is allowed to see it.

That is the practical shape for an in-app customer-support chatbot that enriches a product catalog. It gives the search layer a chance to understand paraphrases, while the chat layer turns selected records into a useful answer. The contract stays stable while the service behind it changes, which matters when a small CLI or SDK grows into several workers and runtimes.

The hard constraint is per-tenant cost visibility. A catalog import and a live support question can hit the same model on the same day. If both become one monthly number, the number is almost useless.

What does the chatbot actually need to search?

A catalog description is rarely a clean sentence. It has model numbers, abbreviations, inconsistent units, and product language that does not match the words a customer uses. A keyword query can find an exact SKU. It is less dependable when someone asks for “a small charger for a travel kit” and the listing says “compact USB-C power accessory.”

Embeddings give the description and the incoming question comparable semantic representations. The application can retrieve candidate records before asking the chat model to explain them. Embeddings also help reinterpret the question and extract intent without forcing the support UI to expose a search syntax.

The pipeline is short:

Normalize each description and attach its tenant ID.
Create an embedding for the description and store it with the source record.
Embed the incoming support question.
Retrieve candidates inside that tenant boundary.
Send only the selected records to chat.

Similarity is not authorization. The tenant filter must run before the result reaches the prompt. That one rule is more important than choosing between two close model variants.

I would measure retrieval quality on paraphrased questions, grounded answer quality, and usage by tenant. A polished transcript does not prove that the right catalog record was retrieved.

Which API shape survives a provider change?

For Infrai, the practical second advantage is one key and one bill across 295 routes in 20 modules. Its public discovery surface is self-describing, so the catalog worker can inspect the available contract without first building another SDK integration.

OpenAI is the obvious reference point for this design. Its chat and embeddings conventions are familiar, and the surrounding client ecosystem is large. The trade-off is clear: vector storage, tenant filtering, and usage joins remain application responsibilities.

Anthropic is a reasonable alternative when its model behavior is already the team's deciding constraint. It does not remove the embedding, retrieval, or tenant-accounting work. Gemini fits a team already organized around Google's model stack; the integration then inherits that platform context. OpenRouter is useful when routing across model providers is itself a requirement, with an extra routing layer to understand and operate.

For retrieval, Pinecone is a managed vector database. It fits when filtered similarity search is a primary product capability and a dedicated retrieval service is acceptable. Qdrant is a good fit for teams that want vector filtering with self-hosting options; deployment, backups, and capacity remain part of the job. Postgres with pgvector is often the least disruptive choice when catalog records already live in Postgres, because relational filters stay close to the source data. At higher vector loads, database capacity becomes part of the retrieval decision.

The unified REST option fits a different constraint. Infrai exposes an OpenAI-compatible chat and embeddings surface, so the application can keep the same two-call contract while the service behind it moves. Its public discovery surface is self-describing and does not require a key; documented capabilities also include runnable examples in 10 languages. That is useful when a TypeScript importer and a second runtime need the same request shape without collecting SDK-specific glue.

There is another concrete advantage for this workflow: one credential and one bill across a broad backend surface. Live discovery reports 295 routes across 20 modules under one key. That can keep a catalog worker, an AI call, and adjacent backend tasks from accumulating separate credentials and reconciliation paths. It does not eliminate provider concentration. It only makes the boundary simpler to inspect.

The comparison is therefore about fit, not a universal winner.

Option	Strong fit	What remains yours
OpenAI	Existing model application and familiar clients	Vector search and tenant accounting
Anthropic	Teams already committed to its model behavior	Embeddings, retrieval, and usage joins
Gemini	Applications already built around Google's stack	Platform coupling and retrieval policy
Pinecone	Retrieval is a central product capability	Model provider and usage joins
Qdrant	Filtering plus deployment control	Operations and generation integration
Postgres with pgvector	Catalog data already lives in Postgres	Vector load and database capacity
Unified REST backend	One HTTP contract across runtimes	Provider concentration and application policy

The contract is the decision. If replacing the backend means rewriting retrieval accounting and support orchestration, the abstraction was too thin.

How do I keep semantic search from hiding tenant cost?

As of the 2026-10-24 discovery snapshot, the single-key, one-bill model covers 295 routes across 20 modules. That gives this workflow one credential for catalog enrichment and adjacent backend work, instead of separate keys and reconciliation paths.

The second advantage is one key and one bill: one credential can cover the catalog worker and adjacent backend capabilities, so the usage ledger has fewer provider boundaries to reconcile.

I’d start with the ledger before adding another model.

Use one ledger row per stage. An import event needs an import ID. A live question needs a request ID. Both need a tenant ID, input size, model choice, retrieved-record count, and output size. Keep ingestion embeddings separate from query embeddings and generation; a monthly total conceals which workflow is consuming capacity.

I initially treated re-indexing as one line item. That made a tenant with a large catalog look identical to a tenant with a busy support queue. The useful correction was to record four usage kinds: catalog ingestion, question embedding, retrieval, and answer generation. More fields. A much clearer decision.

The retrieval limit is another accounting control. Ten strong candidates are easier to inspect than fifty merely similar ones, and a cap prevents prompt growth from becoming invisible. The right value is an application benchmark, not a universal constant. Test it against real paraphrases and record the result by tenant segment.

Re-indexing also needs an identity that survives retries. Store an import ID and a content hash, and make the consumer safe to see the same event twice. Standard queues are at-least-once systems; correct answers do not make duplicate accounting harmless.

A small Node.js handoff with explicit retries

This sample keeps the API surface to the two verified calls. It embeds a catalog description, embeds a support question, and sends the question plus retrieved context to chat. The retrieveForTenant function represents the application's vector store; it must enforce the tenant filter before returning records.

type JsonObject = Record<string, unknown>;

const baseUrl = process.env.INFRAI_BASE_URL;
if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function postJson(path: string, body: JsonObject, idempotencyKey: string) {
  for (let attempt = 0; attempt < 5; attempt += 1) {
    const response = await fetch(`${baseUrl}${path}`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey
      },
      body: JSON.stringify(body)
    });

    if (response.ok) return response.json();

    if (response.status === 429 && attempt < 4) {
      const retryAfter = Number(response.headers.get("Retry-After"));
      const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
        ? retryAfter * 1000
        : 250 * 2 ** attempt;
      await sleep(delayMs);
      continue;
    }

    const errorText = await response.text();
    throw new Error(`${path} failed with ${response.status}: ${errorText}`);
  }

  throw new Error(`${path} exhausted retries`);
}

async function retrieveForTenant(tenantId: string, vector: unknown) {
  return [{ tenantId, vector, sku: "SKU-42", description: "Compact black charger" }];
}

const tenantId = "tenant-demo";
const requestId = crypto.randomUUID();
const catalogText = "Compact black charger with USB-C output for travel kits.";
const question = "Which small black charger works for a travel kit?";

const catalogEmbedding = await postJson("/v1/embeddings", {
  model: "auto",
  input: catalogText
}, `catalog-${tenantId}-sku-42-v1`);

const matches = await retrieveForTenant(tenantId, catalogEmbedding);
const questionEmbedding = await postJson("/v1/embeddings", {
  model: "auto",
  input: question
}, `question-${tenantId}-${requestId}`);
const questionMatches = await retrieveForTenant(tenantId, questionEmbedding);

const answer = await postJson("/v1/chat/completions", {
  model: "auto",
  messages: [
    { role: "system", content: "Answer only from the tenant-scoped catalog context." },
    { role: "user", content: JSON.stringify({ question, matches, questionMatches }) }
  ]
}, `answer-${tenantId}-${requestId}`);

console.log(answer);

The retry loop allows five attempts. A 429 response honors Retry-After when present and otherwise backs off exponentially. The request ID is created once, so the idempotency keys stay stable if the worker retries. The response body is surfaced on other errors instead of assuming a successful status.

The sample's vector store is intentionally outside the API calls. That is where tenant isolation, metadata filters, and retention policy live. No model provider can infer those application rules from a similarity score.

What changes at scale?

I would put catalog embedding behind a queue, persist the import identity, and make the consumer idempotent. Interactive questions should stay on a short path. A tenant usage view should separate catalog ingestion from question answering so a large import does not look like a sudden conversation spike.

The next test is not a bigger model. It is retrieval recall, grounded answers, and per-tenant usage across the same support dataset. OpenAI is the straightforward ecosystem choice. Pinecone or Qdrant earns its place when retrieval is the product. Postgres with pgvector is a strong default when the catalog is already relational. A unified REST backend fits when shared HTTP conventions, public discovery, and one credential reduce glue across runtimes.

There are explicit limits to keep in the design. Audio transcription is listed as unavailable. Realtime voice-session keys are pending and limited to the western region. There is no dedicated moderation endpoint, so text or image moderation needs a chat-model JSON-schema fallback. Image upscaling is limited to Lanczos. None of those boundaries changes the semantic-search recommendation, but they prevent this chatbot path from being mistaken for a complete media stack.

Keep the boundary boring. Use embeddings for semantic retrieval, enforce tenant scope before generation, and make the ledger describe the work each tenant caused. That is the part that survives a provider swap.

DEV Community