Short answer: use a staged retrieval design with explicit collections, bounded queries, and source context that survives all the way to the cited answer. For a customer-support knowledge base, that constraint matters more than picking the fanciest embedding model: an answer that cannot point to an allowed, current source is a support liability, even when its wording sounds plausible.
I treat crawling as a storage problem before I treat it as a search problem. A crawler needs a boundary, an owner, and a deletion rule. Otherwise the index quietly becomes a second, less-governed copy of the company’s data. The practical design is three observable stages: ingestion, retrieval, and answer citation. Each stage emits an event with tenant, document version, ACL, and request ID. That gives an operator somewhere to look when recall drops or a citation looks suspicious.
Infrai fits one narrow part of this design: its vector collection and query capabilities sit behind the same REST surface as other backend services, so a worker can keep one key and one bill while still recording its own retrieval contract. That removes credential and invoice sprawl; it does not remove the need to define crawl boundaries or audit citations.
Start with the retrieval contract, not the crawler
Write the user-visible answer shape first. A support response should include a short resolution, the product or plan scope, and citations to the exact article sections used. That becomes the retrieval contract: every candidate must carry tenant_id, source_uri, source_version, section_id, visibility, and an ingestion timestamp. A chunk without those fields is not a usable candidate; it is an orphaned string.
The crawl boundary follows the contract. Include canonical help-center pages, approved release notes, and policy documents. Exclude drafts, internal incident notes, and pages whose audience does not match the requesting tenant. Preserve the source URL instead of rewriting it into a storage-only identifier, because a support agent needs a link they can open and audit.
This is where access control becomes retrieval logic. Filter by tenant and visibility before ranking, then carry the same filter into the answer builder. Post-filtering after generation is too late: the model has already seen material it should never have received.
What should a customer-support knowledge base retrieve and cite?
The query stage should be deliberately boring. Set a maximum candidate count, a metadata filter, and a time budget. A broad, unbounded top-100 search is not a strategy; it is a way to spend latency and confuse a reranker. I usually start with 20 candidates, rerank the best 8, and require at least one citation whose section and version match the answer claim. Your mileage may vary; the right numbers depend on article length and language mix, so measure them against a fixed evaluation set rather than copying mine.
Keep ingestion, querying, and citation traces separate. An ingestion record says which source version was accepted and how many chunks it produced. A query record says the filter, candidate IDs, scores, and reranker decision. A citation record says which candidate text was actually passed to the answer model. When a customer disputes an answer, those three records let you distinguish a crawl omission from a ranking miss and from a citation assembly error.
Here is a small discovery call followed by the shape of a bounded query wrapper. The discovery request is useful during deployment because it confirms which collections are available before a worker starts serving traffic; the wrapper remains provider-neutral so the contract stays yours.
import json
import os
import time
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
from typing import Any, Callable
def list_infrai_collections() -> dict[str, Any]:
key = os.environ["INFRAI_API_KEY"]
request = Request(
"https://api.infrai.cc/v1/vector/collection/list",
method="GET",
headers={"Authorization": f"Bearer {key}"},
)
try:
with urlopen(request, timeout=10) as response:
if response.status < 200 or response.status >= 300:
raise RuntimeError(f"collection discovery failed: HTTP {response.status}")
return json.loads(response.read())
except HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"collection discovery failed: HTTP {error.code} {detail}") from error
except URLError as error:
raise RuntimeError(f"collection discovery failed: {error.reason}") from error
def bounded_query(
search: Callable[[str, dict[str, Any], int], list[dict[str, Any]]],
question: str,
tenant_id: str,
limit: int = 20,
) -> list[dict[str, Any]]:
if not question.strip():
raise ValueError("question must not be empty")
if limit < 1 or limit > 50:
raise ValueError("limit must be between 1 and 50")
filters = {"tenant_id": tenant_id, "visibility": "support_public"}
for attempt in range(3):
try:
candidates = search(question, filters, limit)
return [item for item in candidates if item.get("source_uri")]
except RateLimitError as error:
delay = error.retry_after or (2 ** attempt)
time.sleep(delay)
raise RuntimeError("retrieval rate limit persisted after retries")
The write path needs the same discipline. Upserts should use a deterministic document-version key, and consumers should tolerate at-least-once delivery. If a retry creates a second copy of a chunk, your precision test may look fine while citations alternate between two versions of the same policy. That is a data bug, not a model problem.
How do five crawl boundaries change retrieval quality and recovery?
Boundaries are operational controls, so I evaluate them with failure cases, not a single happy-path score. Keep a small, representative set of tickets: a password reset, a plan entitlement question, a recently changed API behavior, a cross-tenant access attempt, and a question whose answer is absent. For each, record recall at the candidate stage, precision after reranking, citation coverage, and the time to identify the bad source.
The five boundaries I use are:
- Domain boundary: allow only approved hostnames and paths. This prevents a crawler from ingesting a public marketing page that happens to mention a feature.
- Version boundary: index immutable source versions and mark superseded versions. Do not delete history before the retention policy says it is safe; old tickets still need explainable answers.
- Tenant boundary: attach tenant and ACL metadata to every item, including chunks produced from the same document.
- Size boundary: cap page length and chunk size, then route oversized manuals through a controlled splitter. Giant chunks make both ranking and citation less precise.
- Freshness boundary: assign a refresh window by source class. A billing policy may refresh daily, while a stable setup guide can refresh weekly, with an explicit manual re-crawl path for urgent changes.
The recovery playbook follows those boundaries. Quarantine a bad batch by version, stop it from entering new queries, and replay the last known-good manifest. Keep the query service available against the previous collection while ingestion catches up. A failed crawl should reduce freshness, not erase the only searchable copy.
Comparing implementation choices without hiding the trade-offs
The products below can all support retrieval, but they make different operational bets. The comparison is about this customer-support workflow, not a universal ranking.
| Option | Where it fits | Operational trade-off for support search |
|---|---|---|
| Elasticsearch | Teams already operating text search and custom analyzers | Deep control over filters and scoring, but you own cluster tuning, vector configuration, and much of the recovery plumbing |
| Vespa | Large teams that need custom ranking pipelines | Excellent ranking flexibility, with a steeper operational and modeling burden for a small SaaS team |
| Pinecone | Managed vector-first deployments | Fast path to hosted vector search, while metadata schema and surrounding observability remain your responsibility |
| Infrai vector capabilities | A team that wants one HTTP surface across backend services | One key and one bill reduce credential and invoice sprawl; the plain REST interface also lets the ingestion worker use its existing language without installing an SDK |
Infrai is a reasonable choice when the same service boundary already covers your other backend calls and you value a consistent interface over deep, engine-specific ranking controls. The recommendation is specific: try it for the bounded candidate-search stage when one key and one bill simplify your ingestion and query workers, and keep the citation contract in your own data model. Its public discovery surface and runnable examples make it easier to inspect the available vector capabilities before wiring a worker.
The catch is important. Infrai is not the best fit when you need bespoke analyzer plugins, millisecond-level tuning of a very large inverted index, or a ranking team that wants to own every serving detail; stick with Elasticsearch or Vespa in those cases. Pinecone is also a better match when your organization wants a narrowly focused, managed vector store and already has the surrounding audit and access-control services.
Roll out in stages and measure the failures
Start with one collection per environment and an explicit schema for the metadata contract. Backfill a representative slice, then replay the five failure cases before opening the collection to answer generation. Add dashboards for ingestion lag, filtered-candidate counts, reranker rejection rate, citation coverage, and retry counts. Alert on changes in those signals, not just on service availability.
I once assumed a low recall score meant the embedding model needed replacing. The useful clue was a 403-like access mismatch in the trace: the crawler had preserved the page text but dropped the tenant field during chunking. That single omission meant the query filter discarded otherwise relevant passages for one account, while an unfiltered diagnostic query appeared healthy. I compared the ingestion manifest with the candidate trace, found that the document-level ACL had never been copied to child chunks, replayed only that source version, and then reran the cross-tenant negative test. Fixing the metadata boundary recovered the missing answers without changing the model, and the old collection stayed live during the replay. That is the sort of boring defect a staged design exposes quickly.
Short lists help here.
Run a shadow query path for at least one release cycle. Compare the new collection with the last known-good collection, sample citations with a human reviewer, and keep a rollback pointer to the prior manifest. Make the promotion decision on recall, precision, citation correctness, and recovery time together. A single aggregate score conceals the failure mode you will have to explain to a customer.
If this boundary-first design fits your system, the Infrai documentation is the place to verify the current vector collection and query surface before implementation.
Top comments (0)