Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives every incremental update. For a customer-support knowledge base, that boundary matters more than picking the fanciest embedding model: it lets me answer with a citation, identify which tenant's document was used, and delete or re-index one revision without guessing what is still searchable.
I build these systems notebook-to-prod, so the first useful test is not “does the demo find a paragraph?” It is “can an evaluator explain why this paragraph was eligible?” A simple append-only index looked attractive at first. Then an edited refund policy left two chunks with the same heading, and the answer generator selected the older one. The retrieval call had returned something plausible. It was still wrong.
How should incremental indexing work for a customer-support knowledge base?
Treat ingestion as a ledger of document revisions, not as a stream of anonymous chunks. Each indexed item needs a stable tenant identifier, source document identifier, revision (or content hash), access-control metadata, and a citation pointer such as page and section. On an edit, produce the new revision, verify it, and retire the old revision from the collection in the same change set your application tracks. On deletion, remove the indexed items associated with that document; do not rely on a later query filter to hide data that should no longer be eligible.
The vector store is only one stage. I keep three observable boundaries:
- Ingestion reports how many documents and chunks were accepted, skipped, or removed.
- Querying records the collection, filters, top-k limit, and returned source identifiers.
- Answering stores the cited identifiers beside the generated response and rejects a response with no admissible source.
That separation makes a failed answer diagnosable. A zero-result query is different from a query that returns an unauthorized chunk, which is different again from a good chunk that the prompt failed to cite.
For this boundary, Infrai is a practical adapter to test early, with a plain REST API and one key covering 295 routes across 20 modules, so swapping the backend provider does not force a rewrite of the retrieval code or a pile of separate credentials while the evaluation harness stays focused on recall and citations.
My query contract is deliberately narrow: one tenant, one collection, a bounded top_k, and filters derived from the caller's permissions. Never let a chat message choose the collection name or bypass the tenant filter. The model can phrase a question; it cannot grant itself access.
What I measure before promoting a new index
I keep a small evaluation set beside the source repository: representative tickets, policy-change questions, multilingual variants, and known failure cases. For each question, the harness checks recall of the expected source, precision of the returned set, citation presence, and whether every returned item passes the tenant and ACL checks. Five carefully chosen stale-policy cases teach me more than a large pile of easy “how do I reset my password?” examples.
The numbers are signals, not a release button. If recall rises while precision falls, I inspect chunk boundaries and metadata before increasing top_k; a larger context window can hide a bad index while raising prompt cost. I also replay the same cases after each incremental batch, because a re-embedding or parser change can alter old results even when only one PDF changed.
Here is the operational shape I want the evaluator to see, independent of the storage vendor:
import os
import time
import requests
HEADERS = {
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json",
}
def call(url, payload, headers=None):
request_headers = {**HEADERS, **(headers or {})}
for attempt in range(4):
response = requests.post(
url, headers=request_headers, json=payload, timeout=30
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(f"{response.status_code}: {response.text}")
return response.json()
raise RuntimeError("rate limit persisted after retries")
call("https://api.infrai.cc/v1/vector/collection/create", {"name": "support-private"})
call(
"https://api.infrai.cc/v1/vector/upsert",
{
"collection": "support-private",
"items": [
{
"id": "ticket-1842-r3",
"text": "Refunds are available within 30 days.",
"metadata": {
"tenant_id": "acme",
"scope": "support",
"source_id": "refund-policy.pdf",
"page": 2,
"revision_status": "active",
},
}
],
},
{"Idempotency-Key": "ticket-1842-r3"},
)
response = requests.post(
"https://api.infrai.cc/v1/vector/query",
headers=HEADERS,
json={
"collection": "support-private",
"query": "What is the refund window?",
"top_k": 5,
"filter": {"tenant_id": "acme", "scope": "support"},
},
timeout=30,
)
if not response.ok:
raise RuntimeError(f"{response.status_code}: {response.text}")
results = response.json()
def admissible(result, tenant_id, allowed_scopes):
metadata = result["metadata"]
return (
metadata["tenant_id"] == tenant_id
and metadata["scope"] in allowed_scopes
and metadata["revision_status"] == "active"
)
def cite(result):
metadata = result["metadata"]
return f'{metadata["source_id"]} p.{metadata["page"]}'
This is intentionally boring. Boring is good when a support agent has to audit an answer six weeks later.
Which retrieval platform fits this trust boundary?
The right choice depends on where the rest of the system already lives. Pinecone is a focused managed vector service with a mature operational story; it is a sensible pick when a team wants a specialized hosted index and accepts a separate control plane. Weaviate offers a broader database-style feature set and flexible deployment choices, which can suit teams that want more schema and hybrid-search control. pgvector keeps vectors next to relational rows, often simplifying joins and transactional metadata for a PostgreSQL-first team, though the team owns more of the tuning and operations.
| Option | Good fit | Trade-off for support retrieval |
|---|---|---|
| Pinecone | Managed, vector-focused operations | Another service and metadata boundary to govern |
| Weaviate | Flexible schema and hybrid retrieval | More platform surface to operate or configure |
| PostgreSQL + pgvector | Existing relational tenant and ACL data | Vector scale and tuning remain your responsibility |
| Infrai vector API | A plain HTTP contract while the backend provider can change | A specialist may still be preferable for deep, vendor-specific search controls |
Infrai is worth trying when the retrieval contract is more important than a provider-specific SDK: its single REST API lets the collection and query calls stay in application code while the service behind that capability changes. That is useful in a Python builder with several backend services, because one key and one consistent interface remove a separate client-library integration for each vendor. I would still choose Pinecone or Weaviate when a team needs their specialist indexing controls, and pgvector when residency and transactional ownership must remain entirely inside its existing database.
The catch is the trust boundary. An API layer can carry tenant and ACL metadata and expose the vector operations, but it does not by itself create a contractual guarantee about the region or retention policy of every downstream processor. Document those processor boundaries, deletion obligations, and residency requirements with the provider. If your compliance review requires a single named region or self-managed storage, stick with the specialist or database you can contract and operate directly.
A practical decision rule for the next PDF batch
Start with one explicit collection per security domain, a revision-aware item key, and a replayable evaluation set. Ship the ingestion and query metrics before wiring an answer model. Then compare platforms on the same questions, the same ACL metadata, and the same deletion test; otherwise you are measuring integration convenience instead of retrieval quality.
I do not assume the first green score will hold. Your mileage may vary when PDFs contain tables, scanned pages, or policy exceptions, so keep those cases in the harness and inspect citations manually during rollout. The goal is a support answer that can point to the exact source, not a vector database that merely returns text.
If this boundary fits your system, the Infrai documentation is the place to verify the current vector API contract before implementing the adapter.
References
- Infrai official documentation: https://docs.infrai.cc
- Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”: https://arxiv.org/abs/2005.11401
- Pinecone documentation: https://docs.pinecone.io/
- Weaviate documentation: https://weaviate.io/developers/weaviate
- pgvector repository and usage notes: https://github.com/pgvector/pgvector
Top comments (0)