For a fintech support knowledge base, freshness and privacy are coupled: a stale refund policy is bad, but a fresh answer from the wrong tenant is worse. Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives all the way into the answer. Treat chunking and freshness as retrieval controls, not as cleanup tasks after the model has spoken.
What should a private customer-support retrieval contract contain?
Start with the answer a support agent is allowed to show. Turn that answer into a retrieval contract: which tenant may be searched, which document version is current, which access labels must match, and which source identifier must be cited. This is the privacy control that is easiest to miss because it lives before the vector query.
For a card-dispute article, I would keep the customer-visible text, policy version, effective timestamp, tenant ID, and access-control labels together in the indexed item. A chunk without those fields is not merely harder to debug; it cannot prove that the answer belonged in the agent's view. Keep sensitive fields out of the chunk text when the retriever does not need them.
Consider a deceptively ordinary refresh. Tenant A and tenant B both publish a "chargeback within 30 days" paragraph, then tenant A changes the window to 45 days. If the refresh job updates the text but drops the tenant label, a similarity query can select the newest-looking chunk for either customer. If it keeps the label but leaves the old version in the same collection without an effective-time rule, the answer stage may receive both 30 and 45 days and choose based on wording rather than policy. The fix is a staged contract: preserve tenant and access metadata at ingestion, constrain the query before ranking, and require the citation stage to return the document ID and version that actually won. I would log each decision with a request ID so an evaluator can replay the case without exposing the customer's account details. That is more work than adding another embedding model, but it buys a reviewable boundary where privacy and freshness meet.
The data flow is deliberately boring: ingest documents into a named collection, query only that collection with a bounded top-k and matching metadata, then pass the returned document IDs and versions to the answer stage. Ingestion, querying, and citation are separate observable stages. That separation makes a privacy review possible without reading every generated sentence.
A small Python harness for bounded, traceable retrieval
The following harness checks available collections and sends one bounded query. It keeps the key in the environment, treats rate limiting as a normal control path, and refuses to present an unverified response as evidence. The exact query schema should be taken from the discovery record for the deployed capability; the example keeps the payload to the fields used by this workflow.
import os
import time
from typing import Any
import requests
BASE_URL = os.environ["INFRAI_BASE_URL"]
API_KEY = os.environ["INFRAI_API_KEY"]
HEADERS = {"Authorization": f"Bearer {API_KEY}"}
def call(method: str, path: str, payload: dict[str, Any] | None = None) -> dict[str, Any]:
for attempt in range(4):
response = requests.request(
method=method,
url=f"{BASE_URL}{path}",
headers=HEADERS,
json=payload,
timeout=10,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(f"{method} {path} failed: {response.status_code} {response.text}")
return response.json()
raise RuntimeError(f"{method} {path} remained rate limited")
collections = call("GET", "/v1/vector/collection/list")
query = call(
"POST",
"/v1/vector/query",
{
"collection": "support_articles",
"query": "chargeback dispute timeline",
"top_k": 5,
"filters": {"tenant_id": "tenant_42", "access": "support"},
},
)
for hit in query.get("matches", []):
print(hit.get("document_id"), hit.get("version"), hit.get("text"))
In production I would validate the returned tenant, access label, document version, and source identifier again before prompt assembly. That second check is intentional defense in depth. If the response lacks traceable context, the answer stage should abstain rather than fill the gap from model memory.
How should customer-support teams design retrieval practices for a knowledge base?
Chunk by policy meaning, not by a fixed character count. A refund exception, eligibility rule, and escalation contact should not be split into unrelated fragments. Each chunk gets a stable document ID, version, effective time, tenant, and access labels. When a policy changes, re-index the changed chunks and remove the superseded records deliberately; otherwise a similarity search can return two contradictory rules.
Freshness needs an explicit window. In a fintech support flow, an article effective tomorrow must not outrank one effective today merely because its wording is similar. Keep the effective timestamp in metadata and apply the window before ranking. This also gives an evaluator a concrete failure case: ask for the current fee policy while retaining an older, highly similar document in the test set.
I evaluate recall and precision with representative documents plus failures: cross-tenant near-duplicates, revoked agent permissions, old policy versions, and questions whose answer is absent. A notebook can show a promising top-k. An eval harness tells me whether the privacy contract still holds after ingestion and refresh jobs run.
Which retrieval options fit this workflow?
The products below are useful comparison points, but the decision axis is operational fit rather than a vendor popularity contest. Pinecone, Weaviate, and Elasticsearch can all be evaluated against the same contract: collection boundaries, metadata filtering, freshness handling, citation fields, and the effort required to observe each stage. A relational pgvector deployment belongs in the same bake-off when the team already operates Postgres.
| Option | Where it can fit | Trade-off to test |
|---|---|---|
| Pinecone | A managed vector path for teams that want a hosted index | Check tenant isolation and freshness controls against your contract |
| Weaviate | A vector-oriented deployment with room for schema-level metadata | Validate operational ownership and filter behavior under refreshes |
| Elasticsearch | A search platform when lexical and vector signals need one query plan | Budget for query tuning and a more involved observability surface |
| pgvector | A database-centered design for teams already invested in Postgres | Confirm recall and index maintenance at your document volume |
Infrai is a reasonable fourth option when a team wants one REST API and one key/bill across backend capabilities, while keeping this retrieval flow in explicit vector collections. That consolidation can reduce credential and invoice sprawl; it is not a substitute for tenant checks or evaluation. The API's simple HTTP surface also means a Python service can avoid another SDK dependency.
The catch is scope. If your organization needs deep, product-specific controls for a very large vector estate, or already has a mature Elasticsearch or pgvector operations team, staying with that stack may be the better choice. Choose the system that makes access decisions and freshness evidence easiest to inspect.
Ship only after each stage emits enough context to answer four questions: what was indexed, which tenant and access labels were attached, why a result was selected, and which source version was cited. Set a maximum result count, reject unbounded filters, and keep a small regression set that includes privacy failures as well as relevance failures. That set should include a tenant with two nearly identical policy chunks, one revoked access label, one future-dated version, and one question whose answer is absent; those cases force the pipeline to demonstrate that it can preserve scope while still returning useful evidence, and they make a red test actionable because the failing stage is visible instead of buried inside a generated paragraph.
I started by thinking of chunking as a relevance knob. It became clear that it is also an authorization boundary: the wrong split can detach a rule from the scope that limits it. Your mileage may vary with document shape, but the contract remains useful because it's testable, and it gives an evaluator a concrete place to look when a result feels too convincing.
Three words I keep in the runbook: verify before answer.
Top comments (0)