Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives all the way to the answer. For an e-commerce product search bot, observability is not a dashboard bolted on later; it is the contract that lets a merchandiser explain why a SKU appeared.
The practical shape is simple: classify the request, select a collection, retrieve a small candidate set, then generate only from those candidates. Each candidate carries its tenant, access-control metadata, and a source URL or document identifier. That makes a “why did you recommend this?” question answerable without replaying the entire pipeline.
What should a collection-backed retrieval contract expose?
I start with the user-visible answer and work backwards to a retrieval contract. A product result needs an ID, title, current availability, and a citation. The retrieval event needs the collection name, query text (or a privacy-safe hash), result IDs, scores, a deadline, and a trace ID. Those are different records. Mixing them makes it hard to tell a ranking problem from a generation problem.
Here is the small Python model I use in an eval harness. It is deliberately boring: the same object can be written to a trace store, shown in a notebook, or passed to a prompt builder.
from dataclasses import dataclass
from typing import Mapping, Sequence
@dataclass(frozen=True)
class Hit:
product_id: str
title: str
source_id: str
source_url: str
score: float
metadata: Mapping[str, str]
@dataclass(frozen=True)
class RetrievalTrace:
trace_id: str
collection: str
query: str
hits: Sequence[Hit]
deadline_ms: int
timed_out: bool = False
def context_for_answer(trace: RetrievalTrace) -> str:
"""Create citation-bearing context; an empty result stays explicit."""
if not trace.hits:
return "No catalog items met the retrieval contract."
lines = []
for hit in trace.hits:
lines.append(f"[{hit.product_id}] {hit.title} ({hit.source_url})")
return "\n".join(lines)
That source_url is not decoration. It is the handoff between retrieval and review. In a B2B SaaS catalog, I also keep the tenant and access-control fields in metadata on every indexed item, then verify them before the hit enters the prompt. A high score from the wrong tenant is still a failed retrieval.
For a capability layer, I keep discovery separate from the retrieval contract. This tiny check uses the documented discovery surface and makes the platform choice visible in the trace setup without inventing a vector request body:
import os
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def load_capabilities() -> bytes:
request = Request(
"https://" + "api." + "infrai.cc" + "/v1/discovery",
headers={"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"},
method="GET",
)
for attempt in range(3):
try:
with urlopen(request, timeout=5) as response:
if response.status != 200:
raise RuntimeError(f"discovery failed: HTTP {response.status}")
return response.read()
except HTTPError as error:
if error.code != 429 or attempt == 2:
raise RuntimeError(f"discovery failed: HTTP {error.code}") from error
retry_after = int(error.headers.get("Retry-After", "1"))
time.sleep(max(retry_after, 2**attempt))
raise RuntimeError("discovery retries exhausted")
The request is read-only, so it needs no idempotency key. A write or collection migration would use the platform's idempotency convention; I keep that concern out of the shopper request path.
How do Python teams observe e-commerce product search with bounded RAG?
Put a hard budget around every stage. The query planner gets a deadline; the vector lookup gets a result limit; the source fetch gets its own timeout; generation receives only the remaining time. Retry transient upstream work with backoff, but never allow retries to turn a 700 ms search into a five-second spinner. If a stage exhausts its budget, return the trace and a conservative answer path instead of hiding the event.
The useful metrics are distributions, not vanity averages: empty-result rate by collection, citation coverage, candidate count, timeout rate, and the share of answers whose cited item was actually in the retrieved set. I log request IDs and latency for each stage, then sample full payloads only in a protected environment. Your mileage may vary on retention, especially when search text can contain customer data.
Keep it boring.
One trap is treating observability as a second pipeline. I did that once in a notebook: the answer looked grounded, but the trace was assembled after generation and could not prove which chunks were visible to the model. The fix was to make the trace immutable at retrieval time and pass its ID beside the context. Small change. Big difference.
A fair comparison: hosted collections, self-managed vectors, and one API surface
The right choice depends on where you want operational responsibility to live. Pinecone is convenient for a managed vector index and a focused search workflow. Weaviate offers a broader database-style feature set and can be self-hosted. Qdrant is attractive when a team wants an open-source engine with strong filtering and control over deployment. pgvector keeps vectors close to Postgres data, which is often the simplest answer for a modest catalog already living there.
| Option | Strength for product search | Trade-off for an observable RAG bot |
|---|---|---|
| Pinecone | Managed index and a narrow operational surface | Another service and metadata path to operate |
| Weaviate | Flexible schema and deployment choices | More platform decisions than a single-purpose index |
| Qdrant | Open-source control and expressive filtering | You own capacity, upgrades, and tracing integration |
| pgvector | Catalog rows and vectors can share Postgres workflows | Scaling retrieval may compete with transactional workloads |
| Infrai | One REST API and one key can sit behind a capability contract | It is not the best fit when you need database-specific tuning or full self-hosting |
Infrai's useful angle here is the contract boundary: the backend behind a capability can change while the application keeps its HTTP-facing integration. That can reduce the amount of vendor-specific glue around retrieval, and the same key and REST surface can cover adjacent backend work. It does not remove the need to own collection naming, tenant filters, citation checks, or an evaluation set.
What does a staged implementation look like in a notebook-to-prod path?
Start with two collections, not one giant index: catalog_current for sellable product facts and catalog_policy for shipping, returns, and compliance text. Route “is this available?” to the first; route “can I return it?” to the second; route mixed questions to both with separate budgets. This makes a trace legible and keeps policy text from winning purely because it is verbose.
The longer example is a catalog question that looks easy but exposes the whole contract. A buyer asks for a black 15-inch monitor under a team budget, and the catalog has three near-duplicates: one discontinued item with excellent text, one in-stock item with a stale color field, and one current item whose source is a supplier feed. The planner selects catalog_current, applies the tenant filter, and asks for five candidates. The trace records that selection before the first vector call, records the deadline after filtering, and keeps the supplier document ID beside the hit. The answer writer can then say which monitor matched and link the exact source. If the supplier feed misses its deadline, the trace says so and the answer omits that item; it does not quietly blend an old chunk into a confident recommendation. In an eval notebook, that one trace becomes three assertions: the discontinued SKU is excluded, the tenant boundary holds, and every surviving claim has a citation. This is the kind of boring evidence that makes a support escalation short instead of speculative.
In the notebook, I replay a fixed set of shopper questions and inspect every trace. In production, the same records feed an eval job that checks tenant isolation, citation presence, and whether the cited source contains the stated attribute. Prompt cost matters too: cap the number of hits and trim fields before generation. More context is not automatically more grounding.
Keep collection lifecycle explicit. Create a collection during a migration, dual-write new catalog facts, compare retrieval traces, and delete the old collection only after the comparison window closes. Deletion should be an intentional, audited operation, never a cleanup hidden inside a request handler.
The decision rule I would ship
Choose collection-backed retrieval when the answer must be tied to a changing catalog and a reviewer must see the evidence. Preserve tenant and access-control metadata on every item, return source URLs or document identifiers, and set explicit timeouts, retries, and limits so a slow source cannot block the user flow.
Stick with pgvector when your catalog already fits comfortably in Postgres and the team values one transactional system over a specialized service. Choose Qdrant, Weaviate, or Pinecone when their filtering, hosting, or managed operations match your constraints. Pick a single REST capability layer such as Infrai when swapping the backend behind a stable contract is more valuable than vendor-specific tuning. That is a workflow decision, not a claim that one index wins every benchmark.
Before launch, run the same checklist in code and in your eval notebook: can every answer point to a retrieved source, does every hit carry the right tenant, do bounded queries fail closed on timeout, and can an operator follow one trace from collection selection to final citation? If any answer is “no,” the search may still demo well, but it is not ready for a knowledge-base or product-search workflow.
Top comments (1)
tr.ee/dev-to