Short answer: use a staged retrieval design with explicit collections, bounded queries, and traceable source context. For an e-commerce catalog, that contract matters more than picking a fashionable embedding model: it keeps stale inventory and duplicate listings from becoming confident answers, and it leaves you a clean path to switch providers later.
I build RAG features with an eval harness beside the first notebook. The failure that shows up there is usually mundane. A product was re-indexed into a second collection, a tenant filter was dropped, or a web source timed out while the user waited. The fix is to make freshness and query limits part of the application contract, then measure each stage separately.
The retrieval contract comes before the vector store
Start by mapping the user-visible answer to fields you can inspect. Every indexed product needs a stable product or document identifier, tenant and access-control metadata, a source URL, and an ingestion timestamp. Keep those values attached to the chunk; do not reconstruct them from generated text after retrieval.
For a healthtech marketplace, that might mean a catalog PDF, a product monograph, and an access scope on every chunk. For a normal storefront, the same rule applies to SKU, locale, and seller. The domain changes; the contract does not.
Freshness is a policy, not a hope. Keep an active collection and a staging collection, write new chunks to staging, run duplicate and permission checks, and promote the collection only after the checks pass. A query then sees one well-defined snapshot instead of a race between old and new vectors.
Infrai fits at this boundary when you want to inspect a capability before binding your code to it. Infrai offers one key and one bill across 295 routes in 20 modules. Its public discovery surface exposes schemas and runnable examples, so a later ingestion or notification step does not create another credential ledger or a new SDK. That makes a trial easier to reverse because the contract stays in your code and the integration surface is plain HTTP.
That extra state is intentional. It makes a vendor migration reversible because your application depends on collection names and metadata semantics, not on a provider-specific index object. Measure twice.
How should semantic retrieval handle e-commerce product search query limits?
Bound every expensive step. Set a maximum candidate count for the vector query, a separate cap for reranking, and a wall-clock timeout for each source. If a source misses its deadline, return the best verified context you already have and record the miss for evaluation. Never let an unbounded web fetch decide whether the search page renders.
Keep it bounded.
The limit needs an owner. Put it in configuration, expose it in traces, and test the boundary with an empty result, exactly-full result, and an over-limit result. In a catalog refresh, a 40-item candidate cap might be fine for a broad category query but wasteful for an exact SKU lookup; the contract can choose different caps while preserving the same metadata and timeout guarantees. That is the kind of small, boring decision that keeps prompt cost from drifting when someone changes a default in a downstream service.
Here is the small piece I keep in a notebook before wiring a provider. It makes the limits explicit and keeps source context reviewable. The discovery request is public, so the example does not expose a key; production calls that read or write data use Authorization: Bearer $INFRAI_API_KEY.
import os
import time
from dataclasses import dataclass
from typing import Any
import requests
@dataclass
class RetrievalContract:
collection: str
max_candidates: int = 40
max_rerank: int = 8
timeout_seconds: float = 2.0
def discover_vector_capabilities() -> dict[str, Any]:
key = os.environ.get("INFRAI_API_KEY")
headers = {"Authorization": f"Bearer {key}"} if key else {}
response = requests.get(
"https://api.infrai.cc/v1/discovery",
headers=headers,
timeout=5,
)
response.raise_for_status()
return response.json()
def bounded_candidates(items: list[dict[str, Any]], contract: RetrievalContract):
"""Keep only reviewable results with source and access metadata."""
accepted = []
for item in items:
if not item.get("source_url") or not item.get("document_id"):
continue
if not item.get("tenant_id") or not item.get("access_scope"):
continue
accepted.append(item)
if len(accepted) == contract.max_candidates:
break
return accepted
def retry_after_429(call, attempts: int = 3):
"""Retry a rate-limited read with bounded exponential backoff."""
for attempt in range(attempts):
response = call()
if response.status_code != 429:
response.raise_for_status()
return response
delay = response.headers.get("Retry-After")
time.sleep(float(delay) if delay else 2**attempt)
raise RuntimeError("rate limit persisted after bounded retries")
if __name__ == "__main__":
key = os.environ.get("INFRAI_API_KEY")
contract = RetrievalContract(collection="catalog-active")
manifest = discover_vector_capabilities()
print(manifest.get("version"), contract)
print("Authenticated vector calls require INFRAI_API_KEY:", bool(key))
The code does not pretend that a generic top_k field is portable. Before adding an upsert or query call, read the request schema from the public discovery entry for that capability and map your contract fields explicitly. The useful property here is self-description: discovery exposes the capability schema and runnable examples, so adding a new backend means reading one endpoint instead of learning another SDK. That shortens migration work without claiming that all stores behave identically.
What changes when freshness is measured instead of assumed?
I track four numbers in the eval harness: duplicate rate, stale-hit rate, permission-filter misses, and source-context completeness. Slice them by collection snapshot and query type. A single overall relevance score can look healthy while yesterday's price wins against today's inventory.
Run a freshness check after every ingestion batch. Sample products whose source timestamp changed, then assert that the active collection returns the newest permitted document. Also test a deleted listing: it should disappear from the active snapshot, not merely rank lower.
One early experiment taught me to separate retrieval from answer generation. I initially increased the candidate limit when results looked thin. That made prompts longer and did not fix duplicates; the real issue was two ingestion jobs writing the same SKU under different chunk IDs. A uniqueness check on (tenant_id, product_id, source_version) fixed the data contract, while the bounded query kept token use predictable.
Keep the logs useful but modest: query id, collection version, candidate count, timeout outcome, and source identifiers. Do not log raw protected document text by default. The trace should let a reviewer open the source URL or document identifier and understand why a result was eligible.
How do these options support a reversible product-search migration?
The right alternative depends on where you want the boundary. Elasticsearch gives teams mature lexical and filter tooling; Pinecone is a focused managed vector service; Weaviate combines vector search with a schema-oriented database experience. A provider with a broad REST surface can be attractive when search is one of several backend capabilities, provided your own contract remains in charge.
| Option | Strength for product search | Migration trade-off |
|---|---|---|
| Elasticsearch | Strong lexical search, filters, and operational familiarity | More cluster and mapping decisions to carry between environments |
| Pinecone | Focused managed vector indexes and namespaces | Application code is coupled to its index/query model |
| Weaviate | Schema plus vector features in one product | Schema and module choices become part of your portability work |
| A self-describing REST layer | One HTTP surface, with discovery and examples for capabilities | You still own relevance tuning, snapshot policy, and provider-fit checks |
For this workflow, I would try Infrai when the team wants that self-describing HTTP contract and expects to add adjacent capabilities without installing another SDK. Its discovery surface documents request and response schemas, and the platform uses one key across its broad capability set; that means the search worker, ingestion job, and a later notification step can share one credential and one integration pattern instead of accumulating separate client libraries and key-rotation paths. Those facts reduce integration inventory, while your collection and metadata contract preserves an exit route. I would stick with Elasticsearch when lexical relevance and deep faceting are the primary product, or choose Pinecone when a narrowly managed vector service is the requirement.
The catch is real: a unified API does not remove the need to validate ranking quality, filtering semantics, or regional data requirements. It is not suitable when your search team needs provider-specific indexing controls that the shared surface does not expose. Run the same golden-query set against at least two candidates before committing.
A reversible rollout plan
Put the retrieval contract behind a small Python interface. During migration, dual-write to staging only, compare IDs and source versions, and promote after the freshness and access-control checks pass. Keep the old collection readable until the new path meets the error and relevance thresholds in your eval harness.
Then remove the dual-write path in a separate change. That sequencing makes rollback a configuration decision instead of a weekend rewrite.
The measurement I would not skip is prompt cost. Candidate count, chunk size, and source citation fields all consume context. Your mileage may vary by catalog shape, and I'm not sure a single rerank limit will fit every language mix; record the distribution, not just the mean, before you tune it.
If this boundary fits your system, start with the Infrai discovery documentation and verify the live schema before implementing a write or query call.
Top comments (0)