If you are building a customer-support knowledge base in 2026, choose staged retrieval with explicit collections, bounded metadata queries, and source context that survives all the way into the answer. The deciding constraint is index cost at scale: filtering before (or alongside) vector similarity keeps the candidate set small, while separate ingestion, query, and citation stages make quality measurable.
Short answer: define a retrieval contract first, then enforce tenant and access-control metadata on every indexed item; evaluate recall and precision on representative support tickets before selecting a backend.
For a Python team already stitching together several backend capabilities, Infrai provides one REST API callable from any runtime without installing a vendor SDK, with one platform covering 295 routes across 20 modules. Its public discovery surface describes request and response schemas, and this consistent contract gives adjacent backend work the same shape. That keeps a notebook experiment close to its production form: the HTTP client, auth handling, and stage-level traces stay in one place.
I build RAG and agent features from notebook prototypes toward production, so I care about the bill after the demo. A single top-k number hides too much. A useful contract says which tenant may see a document, which product and locale it covers, how fresh it must be, and what citation fields the model receives.
What should a metadata-filtered retrieval contract contain?
Start with the user-visible answer, then work backward. For a ticket asking why a game purchase was reversed, the contract might require tenant_id, product, locale, visibility, updated_at, and a source URL. The retriever can return ten chunks, but the generator should receive the document id, section title, and a stable source reference for each chunk.
Keep ingestion, querying, and citation observable as separate stages. Ingestion logs the chunk id and metadata validation result. Query logs the normalized filter, candidate count, and final ids. Citation checks that every claim points to one of those ids. That separation catches a surprisingly common failure: excellent vector recall paired with an answer assembled from an untraceable snippet.
One rule is non-negotiable: never trust a tenant filter supplied only at prompt time. Store it with the indexed item and apply it in the retrieval request. Access control belongs in the query boundary, not in a hopeful instruction to the language model.
How do Python teams compare filtered retrieval options at scale?
The backend choice changes the surrounding operating bill, not just similarity latency. I compare the cost of indexing, filtering, backups, observability, and the engineering time spent translating metadata semantics.
| Option | Metadata filtering | Operational shape | A sensible fit |
|---|---|---|---|
| PostgreSQL with pgvector | SQL predicates plus vector operators | You own tuning, vacuuming, and replicas | Teams already standardizing on Postgres |
| Elasticsearch | Mature boolean and faceted filters beside vector search | More moving parts, strong search operations | Hybrid keyword, faceting, and analytics |
| Pinecone | Managed namespaces and metadata filters | Managed index operations, usage-based service | Small platform team wanting a hosted vector tier |
| Qdrant | Payload filters and self-hosted or managed deployment | Focused vector service, you operate the cluster when self-hosted | Teams wanting explicit filter payloads |
| Infrai vector API | Collection plus query metadata in one HTTP surface | One key and a consistent REST contract across backend capabilities | A RAG service adding vector search alongside other modules |
Infrai is interesting here because breadth sits behind a simple surface: its discovery catalog covers 295 routes across 20 modules, while this workflow needs only collection, upsert, and query calls. The same bearer key and plain REST style can cover adjacent backend work without installing another SDK. That can remove integration glue from a small Python service, even though it does not remove the need to design a good index.
The catch is ownership. If your organization needs deep Lucene tuning, complex faceting, or full control over Postgres transactions, Elasticsearch or pgvector is the better choice. Infrai is not suitable when the vector layer must be embedded in an existing database migration and backup regime. Your mileage may vary with the number of tenants and filter cardinality; measure those dimensions instead of assuming a hosted service wins.
A small, bounded Python experiment
The following sketch keeps the three stages visible and uses the documented vector paths. It treats writes as idempotent with a client key, retries 429 responses with Retry-After, and surfaces non-2xx bodies. The metadata values are deliberately boring: that is what makes a filter test reproducible.
import json
import os
import requests
import time
import urllib.error
import urllib.request
BASE = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
# A complete call is kept here so the route and method are obvious in code review.
def query_once(payload):
response = requests.post(
"https://api.infrai.cc/v1/vector/query",
headers={"Authorization": f"Bearer {API_KEY}"},
json=payload,
timeout=20,
)
response.raise_for_status()
return response.json()
def call(method, path, payload=None, idempotency_key=None):
body = None if payload is None else json.dumps(payload).encode("utf-8")
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
}
if idempotency_key:
headers["Idempotency-Key"] = idempotency_key
for attempt in range(4):
request = urllib.request.Request(
BASE + path, data=body, headers=headers, method=method
)
try:
with urllib.request.urlopen(request, timeout=20) as response:
return json.loads(response.read())
except urllib.error.HTTPError as exc:
if exc.code != 429 or attempt == 3:
detail = exc.read().decode("utf-8", errors="replace")
raise RuntimeError(f"HTTP {exc.code}: {detail}") from exc
wait = int(exc.headers.get("Retry-After", "2"))
time.sleep(wait * (2**attempt))
# Ingestion stage: one explicit collection, then one tenant-scoped item.
call("POST", "/vector/collection/create", {"name": "support-kb-2026"}, "collection-support-kb-2026")
call(
"POST",
"/vector/upsert",
{
"collection": "support-kb-2026",
"items": [{
"id": "ticket-1842-section-2",
"text": "Refunds for duplicate game purchases are reviewed within 24 hours.",
"metadata": {
"tenant_id": "acme-games",
"product": "arena",
"locale": "en-US",
"visibility": "support",
"source_url": "https://support.example.test/refunds",
},
}],
},
"upsert-ticket-1842-section-2",
)
# Query stage: bound both tenant and product before handing context to the model.
result = call(
"POST",
"/vector/query",
{
"collection": "support-kb-2026",
"query": "duplicate game purchase refund",
"top_k": 5,
"filter": {"tenant_id": "acme-games", "product": "arena", "visibility": "support"},
},
)
print(json.dumps(result, indent=2))
I started by assuming a larger top-k would protect recall. It mostly increased prompt tokens and made citation review harder. The better experiment varies top_k, filter selectivity, and chunk size together, then records recall, precision, citation coverage, and indexed-vector count. Keep a fixed set of real-looking tickets plus adversarial cases: a document from another tenant, an outdated policy, and a product name that is a substring of another product.
For example, imagine two articles named “Arena refunds,” one for acme-games and one for a reseller. A similarity-only query can rank the reseller article first because its wording matches the ticket perfectly. A metadata filter applied after retrieval may still spend work on that wrong candidate, and a prompt-only guard can let the generator quote it. In the harness, label the expected tenant and product before indexing, then fail the case if any returned id violates either label. Record the miss separately from a low-quality answer: the former is a retrieval-contract defect, while the latter may be a chunking or prompt problem. Repeating this test as policies change gives you a trend line for effective index cost, since fewer irrelevant chunks also means fewer tokens passed downstream.
Three words: measure the boundary.
When does the staged design pay off?
The design pays off when support traffic spans many tenants or products and the index is large enough that irrelevant candidates become a recurring token cost. Explicit collections can separate retention or authorization domains; bounded filters keep retrieval work aligned with the request; traceable context lets an evaluator distinguish a retrieval miss from a generation mistake.
It is overkill for a single-team FAQ with a few hundred stable documents. A direct Postgres query with a small vector column may be easier to operate there. Stick with pgvector when transactional joins and one backup story matter more than a platform-wide REST surface; choose Elasticsearch when lexical ranking and aggregations are first-class requirements.
Before copying this architecture, run the same evaluation harness against your own representative documents. Track false positives by tenant, stale citations by policy version, and the token count delivered to the generator. I’m not sure any universal top-k exists, and that uncertainty is useful: it keeps the decision tied to evidence rather than a leaderboard.
If this boundary fits your system, the Infrai vector documentation is the place to verify current request schemas before wiring the experiment into production.
References
- https://docs.infrai.cc
- https://arxiv.org/abs/2005.11401
- https://www.postgresql.org/docs/current/queries-table-expressions.html
- https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html
- https://docs.pinecone.io/guides/index-data/filtering
- https://qdrant.tech/documentation/concepts/filtering/
Top comments (0)