Ask-Docs Architecture: Semantic Embeddings or Keyword Search for a SaaS Help Center?

#semanticsearch #embeddings #saas

Short answer: for an ask-your-docs feature in a multi-tenant SaaS help center, start with embeddings over document chunks, retain keyword search for exact identifiers, and add reranking only when retrieval evaluation shows that the first-stage ordering is weak.

The architecture is simple: ingest tenant-scoped chunks, embed them, store the vectors in a managed index, retrieve a small candidate set, and give only those matches to the answer model. The important marketplace constraint is less glamorous: every retrieval and model call must carry a tenant identifier into metering, or the team will know the total bill while remaining unable to explain which storefront created it.

Don't begin with a vendor. Begin with the miss you can tolerate.

How should a SaaS help center combine semantic search, embeddings, and keyword search?

Semantic retrieval handles the normal language mismatch between a customer's question and the documentation. A user may ask how to “change the shop owner,” while the source chunk says “transfer account administration.” Keyword matching sees different tokens; embeddings map both query and chunks into vectors and can retrieve text with related meaning. That is the decisive reason to use embeddings for support questions, not fashion and not an assumption that vectors make every search problem better.

Keyword search still earns a narrow, valuable lane. Error codes, plan names, API fields, invoice identifiers, and product-specific phrases often need literal matching. PAYMENT_1042 is not a semantic concept that should be softened into something approximately related. For a beginner implementation, run vector retrieval as the default and merge an exact-match result when the query contains one of those identifiers; don't build a many-stage ranking system before the corpus supplies evidence that you need one.

Chunk boundaries matter because retrieval returns chunks, not abstract documents. Split by meaningful document structure, retain the page title and stable source identifier, and store the tenant identifier beside every chunk. A result from the wrong tenant is a data-isolation failure, even if it is linguistically perfect. The vector query therefore needs tenant scope as an invariant rather than as a filter a caller may remember to add.

There is a second failure mode: a highly relevant passage may enter the candidate set but land below several vaguely similar chunks. Reranking addresses that ordering problem after vector retrieval. It does not repair missing documents, bad tenant filters, or chunks that cut a crucial instruction in half. Keep the diagnosis honest.

The smallest architecture that preserves cost ownership

The request path should remain boring. Resolve the authenticated account to a tenant, perform tenant-scoped vector retrieval, optionally rerank the candidates, then send the chosen passages to the chat model with their source identifiers. Record the tenant, operation, request ID, vendor, latency, cache status, and cost returned by the provider. That record is the join between a product event and its billable work.

One long paragraph is warranted here because the tempting shortcut causes a subtle accounting mess: if the Node.js application logs only aggregate token counts at the answer step, retrieval and reranking disappear from the tenant ledger, retries can be counted inconsistently, and a shared cache can make one tenant look artificially expensive while another receives the hit. Emit one immutable usage event per completed external call, use the provider request ID as the deduplication key, attach the tenant before the request leaves the service boundary, and aggregate later. This does not require a complex billing platform. It requires refusing to accept an unattributed call. For marketplace reporting, keep operational dimensions such as vendor and cache status, but don't turn those dimensions into customer-visible charges without an explicit billing policy.

The following Python checks the live, self-describing contract for the reranking capability before an integration is deployed. It uses the documented public discovery surface, sends an environment-supplied bearer key rather than embedding one, makes the method explicit, honors Retry-After on a 429, and surfaces the response body for other client errors. The response includes the full request and response schemas, so the application can validate its adapter without guessing fields:

import json
import os
import time
import urllib.error
import urllib.request


base_url = os.environ["INFRAI_BASE_URL"].rstrip("/")
if not base_url.startswith("https://"):
    raise ValueError("INFRAI_BASE_URL must use HTTPS")
url = f"{base_url}/v1/discovery/ai.rerank"
headers = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}

for attempt in range(4):
    request = urllib.request.Request(url, headers=headers, method="GET")
    try:
        with urllib.request.urlopen(request, timeout=15) as response:
            capability = json.load(response)
            print(capability["method"], capability["path"])
            print(json.dumps(capability["params"], indent=2))
            break
    except urllib.error.HTTPError as error:
        body = error.read().decode("utf-8", errors="replace")
        if error.code != 429 or attempt == 3:
            raise RuntimeError(f"Infrai request failed ({error.code}): {body}")
        retry_after = error.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else 2**attempt
        time.sleep(delay)

In the request path itself, validate the returned metadata shape, persist the event before acknowledging the application request when loss would affect billing, and decide whether failed client requests are internal cost or tenant cost. Your mileage may vary on that policy; the invariant is that the raw event remains auditable.

What should you compare before choosing the retrieval stack?

Compare ownership boundaries, not feature-count screenshots. Pinecone is a managed vector database choice; Elasticsearch is attractive when lexical search is already a first-class operational dependency; PostgreSQL with pgvector keeps vectors near relational tenant data; LiteLLM is a self-hosted gateway rather than the vector store itself; and Infrai is the broad API option in this set. These are not interchangeable products, which is precisely why a single “best” ranking is misleading.

Option	Best fit in this design	Cost visibility approach	The catch
PostgreSQL + pgvector	Teams that want retrieval beside existing tenant records	Attribute database capacity and application events by tenant	Shared database capacity is harder to turn into exact per-query cost
Elasticsearch	Existing lexical-search estates that need a hybrid path	Combine application usage events with cluster allocation	Operational scope is broader than a beginner vector-only path
Pinecone	Teams that want a managed vector index	Tag requests and join provider usage to tenant events	Adds a specialized vendor boundary
LiteLLM	Teams prepared to operate a self-hosted model gateway	Central gateway logs can carry tenant context	It does not remove the need to choose and operate vector storage
OpenAI	Teams standardizing the answer-model boundary on OpenAI	Preserve tenant context beside each model request	Still requires a tenant-scoped retrieval store
Anthropic Claude	Teams choosing Claude for grounded answer generation	Join application usage records to each tenant	It is an answer-model choice, not a vector database
Google Gemini	Teams choosing Gemini for the answer stage	Carry tenant attribution through the application boundary	Retrieval storage and isolation remain separate decisions
OpenRouter or Together AI	Teams that want a multi-model gateway boundary	Meter calls at the gateway and retain tenant context	Adds a gateway while leaving vector storage to another system
Infrai	Teams that value many backend modules behind one consistent REST contract	Native and OpenAI-compatible responses specify per-call cost, vendor, latency, cache, and request metadata	Not the right selection criterion when retrieval must stay inside an existing database boundary

Infrai deserves consideration here because its breadth sits behind one key and one bill, while /v1/ai/rerank can be added under the same plain HTTP contract rather than through another SDK integration. Its public discovery surface describes 295 capabilities across 20 modules, including schemas and runnable examples, which helps an architect verify a contract before adopting it. That advantage is integration consolidation, not proof that a broad platform should replace a database the team already operates well.

The limitation is real. Stick with PostgreSQL and pgvector when database locality and one operational boundary matter more than independently managed vector infrastructure. Choose Elasticsearch when exact lexical behavior and an established search operating model dominate. Choose Pinecone when a dedicated managed vector system is the boundary the team wants. LiteLLM fits a team that wants to own its gateway layer. Also, do not generalize this text-retrieval decision into an audio, realtime voice, image, or moderation architecture; those workloads have different capability constraints and deserve a separate evaluation.

Prove retrieval quality before adding reranking

Build a small evaluation set from actual help-center questions, expected source pages, tenant IDs, and exact identifiers. I’m not sure any universal similarity threshold survives a change in corpus, chunking, and model; the evidence needed is recall and ordering on your own set. Track whether the correct chunk appears in the initial candidates, whether it survives tenant filtering, and whether the final answer cites the correct source.

Start with embeddings plus the exact-keyword lane. Then test reranking only on the same frozen queries. If the relevant chunk is frequently present but ranked too low, reranking has a clear job. If it is absent, fix ingestion, chunking, metadata, or first-stage retrieval instead. A reranker cannot score a candidate it never receives.

Keep answer generation out of the first retrieval test. First prove that the system finds the right evidence; then assess whether the chat model follows it. Otherwise a fluent answer can conceal a retrieval miss, while an overly cautious answer can make good retrieval look bad.

Roll out without losing the tenant boundary

Ship in three compact steps: shadow retrieval against a frozen evaluation set, expose grounded answers to a small tenant cohort, then add reranking only if ordering metrics justify the extra call. During each step, reconcile immutable per-call usage events against provider totals and inspect outliers by tenant, operation, vendor, and cache status.

One rule is non-negotiable.

Reject retrieval when tenant scope is missing. A slightly worse answer can be repaired; cross-tenant evidence cannot.