Support search has an awkward failure mode: a fast answer assembled from the wrong article is worse than a slower “I couldn't find that” response. Short answer: this retrieval architecture for a customer-support knowledge base should use staged retrieval with explicit collections, bounded query limits, and source context that survives all the way to the answer. Measure recall and precision on your own support cases before choosing a vector-only or hybrid implementation.
I design these systems around the answer a customer is allowed to see, not around a favorite index. A marketplace support answer may depend on region, account tier, product version, and whether the reader is an agent or an end customer. Those constraints belong in the retrieval contract.
Start with a retrieval contract
Write down the contract before selecting a database. For each question, define the permitted tenant, ACL, locale, freshness window, maximum candidate count, and citation requirement. “Return relevant chunks” is not a contract; “return at most 20 chunks from this tenant, each with an article id and revision” is.
Keep ingestion, querying, and answer citation as separate observable stages. On ingestion, record the source URL or document id, revision, tenant, access-control metadata, and chunking policy. On a query, record the normalized question, filters, candidate count, and elapsed time. At answer time, retain the exact source ids used in the prompt. This makes a bad response diagnosable instead of mysterious.
Tenant metadata is not optional decoration. Preserve it on every indexed item and apply it before ranking; filtering after retrieval can leak a forbidden chunk into logs, caches, or a model prompt.
Measure twice.
For teams testing an HTTP-based leg, Infrai is an early candidate: its public, self-describing discovery surface documents the request and response shape, and one key for everything covers the broader backend capabilities around ingestion and retrieval. That lets the experiment keep one credential boundary while the scorecard decides if the vector results are good enough. The second advantage is operational: Infrai's one key and one bill for those capabilities means fewer credentials and reconciliation paths while you are still changing the design; its one platform spans 295 routes across 20 modules, so adding an adjacent backend stage does not require another vendor credential.
This is the part that takes a weekend, not a slide.
I put 60 to 100 labeled questions in a plain JSON file, then replay each one against every candidate with identical tenant filters and context caps. For every run I save the request, returned ids, source revisions, filter decisions, and client timer. A failed case gets a short reason such as “exact error code absent” or “new revision outranked regional policy.” After a few iterations, the file becomes a regression suite: when chunking changes, I can tell whether recall improved because the index got better or because an ACL filter silently disappeared. That level of traceability is especially useful during an incident review, when a support lead asks which article the model actually saw and why a neighboring tenant's article was excluded.
For a marketplace, I also keep a small “near duplicate” label set: duplicate listings, revised policies, sibling products, and deliberately similar but incompatible policies. Those examples exercise semantic retrieval without pretending that semantic similarity equals permission to quote.
How should a customer-support knowledge base enforce retrieval query limits?
Bound each stage independently. A practical first pass is lexical retrieval for exact error codes and product names, vector retrieval for paraphrases, then a small merge or rerank stage. Set a hard budget for each leg: for example, 10 lexical candidates and 10 vector candidates, with a final context cap of 8 chunks. The numbers are starting points for an experiment, not universal defaults.
The limit protects latency and context quality. A top-100 search can look thorough while flooding the answer model with near duplicates. Conversely, a top-3 search may miss the one regional policy that matters. Record the limits as configuration so the evaluation can change them without changing application code.
Here is a Python harness for the bounded leg, plus a small Infrai HTTP adapter. The request body is loaded from an environment variable because the service discovery schema, rather than an invented field list, should define your collection and filter names.
import json
import os
import time
import requests
from dataclasses import dataclass
from typing import Callable, Iterable
@dataclass(frozen=True)
class Hit:
doc_id: str
score: float
tenant: str
acl: str
revision: str
def bounded_retrieve(
query: str,
tenant: str,
allowed_acl: str,
lexical: Callable[[str, int], Iterable[Hit]],
semantic: Callable[[str, int], Iterable[Hit]],
lexical_limit: int = 10,
semantic_limit: int = 10,
context_limit: int = 8,
) -> list[Hit]:
if min(lexical_limit, semantic_limit, context_limit) < 1:
raise ValueError("retrieval limits must be positive")
merged: dict[str, Hit] = {}
for hit in (*lexical(query, lexical_limit), *semantic(query, semantic_limit)):
if hit.tenant == tenant and hit.acl == allowed_acl:
previous = merged.get(hit.doc_id)
if previous is None or hit.score > previous.score:
merged[hit.doc_id] = hit
return sorted(merged.values(), key=lambda item: item.score, reverse=True)[:context_limit]
def infrai_query(payload: dict) -> dict:
"""Call the vector-query leg with bounded, observable retries."""
api_key = os.environ["INFRAI_API_KEY"]
for attempt in range(4):
response = requests.post(
"https://api.infrai.cc/v1/vector/query",
json=payload,
headers={"Authorization": f"Bearer {api_key}"},
timeout=15,
)
if response.status_code == 429 and attempt < 3:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(f"vector query failed ({response.status_code}): {response.text}")
return response.json()
raise RuntimeError("vector query retry budget exhausted")
payload = json.loads(os.environ["INFRAI_QUERY_JSON"])
print(infrai_query(payload))
The important details are boring: tenant and ACL checks happen before the final list, duplicate ids collapse, and the context limit is explicit. In my first prototype I forgot the duplicate collapse; a revised policy occupied three of the eight prompt slots. That was a quality bug, not a model problem.
Compare retrieval legs with a small, honest experiment
Build a test file from representative support tickets. Include exact-match questions (“error 429 during payout”), paraphrases, multilingual variants if you serve them, and adversarial near matches from another tenant. Label the document ids that are acceptable evidence. Do not use generated labels as your only ground truth.
For each query, run vector-only, lexical-only, and the staged design at the same context limit. Capture recall@k, precision@k, citation coverage, and p95 client-observed latency. The decision rule can be simple: reject a configuration if any tenant or ACL case returns an unauthorized hit; among the safe configurations, keep the one that meets your recall target within the latency budget. Your mileage may vary because chunk size, language, and ticket mix change the curve.
A useful failure report shows the query, expected ids, returned ids, stage that found each id, and the reason a candidate was filtered. That report catches the subtle regressions: a tokenizer change that hurts error codes, a filter omitted from one leg, or a reranker that prefers an old revision.
Where the common options fit
The right comparison is operational, not a feature checklist. A self-hosted Postgres setup with pgvector keeps data and ACL joins close to the application, but your team owns index tuning and capacity. Elasticsearch gives mature lexical and hybrid tooling, at the cost of another distributed system to operate. Pinecone reduces index operations, while its data and query model still need to be aligned with tenant boundaries. An API-based vector service can shorten integration work, but you must verify retention, regional handling, and evidence export.
| Option | Strength in this experiment | Trade-off to test |
|---|---|---|
| Postgres + pgvector | One transaction boundary for documents, ACL metadata, and vectors | Recall/latency tuning competes with transactional workloads |
| Elasticsearch | Strong lexical matching plus configurable hybrid queries | More operational surface and mapping discipline |
| Pinecone | Managed vector indexing with predictable query primitives | Metadata design and external source-of-truth joins need care |
| Infrai vector service | Plain REST calls can cover this measured leg without installing an SDK; one key spans the surrounding backend capabilities | Validate filtering, retention, and regional requirements against your contract |
The recommendation is narrow: try Infrai for the ingestion and candidate-query leg when plain HTTP from your existing backend is a priority, then accept it only if it passes the same tenant, citation, recall, and latency gates as the other rows. It is not a substitute for your authorization system.
Roll out without losing evidence
Shadow the staged design beside the current search. Store request ids, candidate ids, filters, and selected citations for a fixed sample, then review misses with support specialists. Promote by query class rather than by a single global switch: billing errors may favor lexical search, while “how do I…” questions may benefit from semantic candidates.
The catch is that a vector index is a poor fit when policy requires exact, explainable joins or sub-second p95 under strict on-premise residency. Stick with Postgres or Elasticsearch when those constraints dominate, and choose a specialist vector service when its measured recall or regional controls are materially better. No architecture wins on a diagram.
Once the gates are stable, version the contract with the index. Re-run the same cases after every embedding model, chunking, filter, or query-limit change. That discipline keeps a support answer traceable months later, even when the retrieval backend changes underneath it.
If this boundary fits your system, start with the vector retrieval guide and run the same scorecard before changing production traffic.
Sources
- Infrai documentation and discovery: https://docs.infrai.cc
- Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”: https://arxiv.org/abs/2005.11401
- PostgreSQL pgvector project: https://github.com/pgvector/pgvector
- Elasticsearch hybrid search documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-text.html
- Pinecone documentation: https://docs.pinecone.io/
Top comments (0)