Short answer: this e-commerce product search retrieval architecture should use three freshness windows backed by one canonical catalog: seconds for price and stock, minutes for searchable attributes, and hours for slow-changing copy. Route each query to the newest completed snapshot, attach a source timestamp, and fail closed when a result is older than its promise.
This is a retrieval architecture decision, not a slider setting. Product search has two truths that drift at different speeds: “can I buy it now?” and “what does this item say?” A useful design makes that drift visible instead of pretending one index refresh policy fits both.
What should an e-commerce retrieval architecture promise?
Start with an invariant: every hit carries catalog_version, updated_at, and a freshness class. The answer layer may summarize a description, but it must never turn stale inventory into a confident availability claim. I have spent enough time around OTP delivery gaps and rate limits to distrust an unmeasured timeout; the same instinct applies here. Put the boundary in data.
Define windows as service-level promises, for example:
| Class | Typical fields | Target age | Acceptable behavior when late |
|---|---|---|---|
| Hot | price, stock, promotion end | 30 seconds | suppress availability claims; show “verify at checkout” |
| Warm | category, brand, attributes | 10 minutes | return the last complete snapshot and expose its age |
| Cold | long description, manuals, rich text | 6 hours | answer from the previous snapshot with a citation timestamp |
Those numbers are starting points, not universal facts. Your merchandising calendar, warehouse lag, and legal wording should set them. The important part is that the promise is testable and attached to a field, not hidden in an index job.
How do freshness windows change product search ranking and citations?
Keep ingestion and retrieval separate. An event stream writes immutable product records; a validator rejects incomplete records; then independent materializers build hot, warm, and cold views. Each view publishes a monotonically increasing version only after its batch is complete. Query fan-out can read the views in parallel, but the response chooses one consistent version per field group.
The citation should point to the product record (or document chunk) and include the version and timestamp used. In a RAG response, that lets a reviewer answer “which catalog state produced this sentence?” The RAG paper describes retrieving passages to condition generation; the same discipline works for product facts when retrieval metadata travels with the passage.
Here is the critical path in Python. It is deliberately boring: freshness is checked before generation, and an old hot value is not silently promoted.
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
@dataclass(frozen=True)
class FieldValue:
value: str
updated_at: datetime
source_id: str
version: int
WINDOWS = {
"hot": timedelta(seconds=30),
"warm": timedelta(minutes=10),
"cold": timedelta(hours=6),
}
def usable(field: FieldValue, freshness_class: str, now: datetime) -> bool:
age = now - field.updated_at
return age <= WINDOWS[freshness_class]
def cite(field: FieldValue) -> dict[str, str | int]:
return {
"source_id": field.source_id,
"catalog_version": field.version,
"updated_at": field.updated_at.isoformat(),
}
At query time, rank candidates using text relevance and business rules only after the freshness gate. If stock is stale, return the product for discovery but mark stock as unverified; do not bury that caveat in prose generated later. A three-word label can prevent a costly promise.
Where do these designs fail under load?
The common failure is a partial publish: hot data advances to version 812 while warm attributes remain at 811, and a join creates a product that never existed. Require a publish manifest listing the versions for every class, then atomically switch the manifest pointer. Readers either see the old complete set or the new complete set.
Another trap is clock ambiguity. Compare timestamps in UTC, record the producer clock source, and monitor ingestion delay separately from index refresh delay. A queue can be healthy while a materializer is minutes behind. Alert on the age users can observe, not only on worker status.
Test with replayed catalog events: out-of-order updates, duplicate events, deletes followed by re-adds, and a promotion that expires during a long query. Assert both ranking and citation metadata. Your mileage may vary on window sizes; the replay should tell you where the user-visible error rate crosses the business tolerance.
I would reject one global index refresh interval for a mixed catalog. It is easy to operate, but it makes the fastest field wait for the slowest pipeline or makes every field pay the cost of hot refreshes. A single view is still suitable for a small catalog with one update cadence, no generated availability claims, and a team that values operational simplicity over tight freshness guarantees.
The trade-off table is explicit: multiple views add storage, version bookkeeping, and more paths to observe. They earn that complexity when stale price or stock creates support, compliance, or checkout risk. They are not suitable when your source system cannot provide reliable update timestamps; fix provenance first.
Treat freshness as part of the answer contract. A search result without age and source identity is only a guess, no matter how polished the generated sentence sounds.
That boundary matters.
Top comments (0)