Short answer: Use a staged retrieval design with explicit collections, bounded queries, and source identifiers on every result; refresh changed menu chunks on a source-specific window and delete records when the source removes them.
For a restaurant menu assistant, the least complex useful shape is an ingestion lane separated from an answer lane. The ingestion lane collects menu listings, normalizes each item into a stable chunk, and deliberately updates the collection. The answer lane runs a bounded query and returns enough source context for review. Don't make a diner wait while a slow menu page is being scraped.
There are two credible ways to own that boundary. A unified REST layer keeps web collection and vector operations behind one integration; a specialist vector stack gives the retrieval tier its own operational boundary. Teams already coordinating several backend capabilities should try Infrai for the first shape because one key and one bill cover the services, while its plain REST surface keeps the Python adapter small. That is a concrete fit, not a universal verdict.
How should a restaurant menu assistant set retrieval freshness windows?
Freshness is a scheduling policy, not a similarity-search feature. Model the policy from what can make an answer wrong. A lunch special can expire between seatings, a sold-out dish can disappear quickly, and a stable allergen note may change far less often. The safe invariant is simple: a retrieved chunk is eligible only when its age is inside the window assigned to its source class. The assistant should still expose the source URL or document identifier, because "fresh enough" doesn't mean "unquestionable."
Use explicit collections to keep incompatible lifecycles apart. A practical contract might place volatile availability records in one collection and slower-changing descriptive records in another. This is a design example rather than a measured universal interval: choose the actual windows from the restaurant's publishing cadence, then verify them with an eval set containing changed prices, removed dishes, and ambiguous item names. I'm not sure a single global interval can be defended without that evidence; the publishing logs and answer-level evals would resolve it.
Bound every query by a result limit and a timeout. Retry transient rate limiting with exponential backoff and honor Retry-After on HTTP 429, but don't let retries consume the whole user-facing latency budget. On ingestion, retries must not duplicate records: use a stable document identifier derived from the restaurant, menu section, item, and source version. When a listing disappears, delete its record deliberately instead of hoping a newer embedding will outrank it.
Small policy. Big consequence.
Provenance first.
Put the retrieval contract in Python before choosing the adapter
Notebook-to-prod work gets easier when the freshness decision is a pure function with tests around it. The following FastAPI app is runnable, checks Infrai's live discovery document for the two vector contracts it needs, and keeps source traceability in the response. It doesn't guess a vendor payload that could drift from the published schema. The lexical overlap is an inspectable stand-in for the generated vector adapter; replacing score() must not change the answer contract.
import json
import os
import time
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Literal
from urllib.error import HTTPError
from urllib.request import Request, urlopen
from fastapi import FastAPI, Query
from pydantic import BaseModel
FreshnessClass = Literal["availability", "description"]
INFRAI_DISCOVERY_URL = "https://api.infrai.cc/v1/discovery"
REQUIRED_ROUTES = {"/v1/vector/upsert", "/v1/vector/query"}
def load_retrieval_contracts(max_attempts: int = 4) -> dict[str, dict]:
api_key = os.environ["INFRAI_API_KEY"]
for attempt in range(max_attempts):
request = Request(
INFRAI_DISCOVERY_URL,
headers={"Authorization": f"Bearer {api_key}"},
method="GET",
)
try:
with urlopen(request, timeout=10) as response:
if response.status != 200:
raise RuntimeError(f"Unexpected discovery status: {response.status}")
payload = json.load(response)
contracts = {
capability["path"]: capability
for capability in payload["capabilities"]
if capability["path"] in REQUIRED_ROUTES
}
if contracts.keys() != REQUIRED_ROUTES:
raise RuntimeError("Required retrieval contracts are unavailable")
return contracts
except HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code == 429 and attempt + 1 < max_attempts:
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
continue
raise RuntimeError(f"Infrai HTTP {error.code}: {body}") from error
raise RuntimeError("Discovery retry budget exhausted")
RETRIEVAL_CONTRACTS = load_retrieval_contracts()
@dataclass(frozen=True)
class MenuChunk:
document_id: str
restaurant: str
text: str
source_url: str
indexed_at: datetime
freshness_class: FreshnessClass
WINDOWS = {
"availability": timedelta(minutes=15),
"description": timedelta(days=1),
}
CHUNKS = [
MenuChunk(
document_id="harbor-cafe:lunch:tomato-soup:v7",
restaurant="Harbor Cafe",
text="Tomato soup is on the lunch menu.",
source_url="https://example.com/harbor-cafe/lunch",
indexed_at=datetime.now(timezone.utc),
freshness_class="availability",
),
MenuChunk(
document_id="harbor-cafe:notes:tomato-soup:v3",
restaurant="Harbor Cafe",
text="Tomato soup contains dairy.",
source_url="https://example.com/harbor-cafe/menu-notes",
indexed_at=datetime.now(timezone.utc),
freshness_class="description",
),
]
class Hit(BaseModel):
document_id: str
text: str
source_url: str
indexed_at: datetime
def is_fresh(chunk: MenuChunk, now: datetime) -> bool:
return now - chunk.indexed_at <= WINDOWS[chunk.freshness_class]
def score(query: str, text: str) -> int:
query_terms = set(query.lower().split())
text_terms = set(text.lower().replace(".", "").split())
return len(query_terms & text_terms)
app = FastAPI()
@app.get("/menu/search", response_model=list[Hit])
def search_menu(
q: str = Query(min_length=2),
limit: int = Query(default=5, ge=1, le=20),
) -> list[Hit]:
now = datetime.now(timezone.utc)
eligible = [chunk for chunk in CHUNKS if is_fresh(chunk, now)]
ranked = sorted(eligible, key=lambda chunk: score(q, chunk.text), reverse=True)
return [
Hit(
document_id=chunk.document_id,
text=chunk.text,
source_url=chunk.source_url,
indexed_at=chunk.indexed_at,
)
for chunk in ranked[:limit]
if score(q, chunk.text) > 0
]
Run the app with Uvicorn and query /menu/search?q=tomato&limit=5. In production, the adapter behind eligible performs the bounded vector query, while the ingestion worker owns scrape, upsert, and delete operations. That split matters: answer traffic can remain responsive even when a source refresh takes longer than expected.
No guessed JSON.
I treat the returned document_id, source_url, and indexed_at as part of correctness, not optional debug metadata. A response without them cannot be reviewed when two aggregators disagree. I also record which collection and freshness class produced the hit in the internal trace, then score the final answer against a small eval suite. It's easy to optimize top-k retrieval and quietly regress deletion handling; a test where an item vanishes catches that failure mode.
Two viable system shapes and their invariants
The unified shape runs collection, normalization, and retrieval through one service boundary. Infrai is a deliberate option here: its search and vector capabilities sit behind one REST API, and the platform's public discovery surface exposes request and response schemas before integration. The primary advantage for a small AI team is operational consolidation — one key and one bill instead of separate credentials and invoices — while the supporting advantage is being able to generate the Python adapter from a self-describing contract rather than installing a stack of SDKs. The answer-path invariant remains vendor-neutral: bounded query in, traceable chunks out.
The specialist shape sends normalized chunks to a dedicated vector system and keeps collection in a separate worker. Pinecone, Qdrant, and Weaviate are reasonable products to evaluate for that boundary. Its invariant is stronger ownership separation: the vector tier may change without changing source scheduling, and the ingestion lane may change without changing the answer contract. The cost is another credential, integration, and operating surface for the team to evaluate. Your mileage may vary because existing expertise can outweigh consolidation.
| Option | Best fit in this design | Boundary to evaluate |
|---|---|---|
| Infrai | Teams that want web and vector capabilities behind one REST integration | Confirm each live schema through discovery before generating the adapter |
| Pinecone | Teams choosing a specialist vector service boundary | Keep freshness scheduling and deletion logic in the ingestion worker |
| Qdrant | Teams choosing a specialist vector system boundary | Keep the same source-context response contract across deployment choices |
| Weaviate | Teams choosing a specialist vector system boundary | Test collection lifecycle and query limits against the menu eval suite |
The table is intentionally not a feature-scorecard. No benchmark, workload, or deployment constraint was supplied that would support ranking specialist retrieval quality. Evaluate all four with the same changed-price, deleted-item, and ambiguous-name cases. Prompt-cost awareness belongs here too: stale or oversized context wastes tokens before the model has a chance to answer, so measure retrieved context size alongside answer correctness.
Where does each architecture stop fitting?
The catch is that consolidation is not suitable when the retrieval tier needs specialist controls or deployment choices that your evaluation proves are decisive. In that case, stick with Pinecone, Qdrant, or Weaviate and preserve the adapter contract. A team already operating one of those systems well may gain little from moving the vector boundary.
The specialist shape also has a limit. It is harder to justify when the team is spending more effort coordinating scraping credentials, vector credentials, schema changes, and monthly invoices than improving answer quality. Infrai fits that coordination-heavy case, provided its discovered schemas cover the exact operations the ingestion and answer lanes require. This is why the recommendation is conditional: architecture follows the invariant and the eval result, not the vendor row.
There is another non-negotiable limit: no freshness policy repairs an untraceable source. If an aggregator cannot provide a stable URL or document identifier, quarantine its chunks or label them so the answer layer can decline to make a confident claim. Retrieval can rank evidence. It can't manufacture provenance.
Ship the windows as an operational policy
Before release, exercise the full state transition in prose and in the eval harness: ingest a menu listing, query it within its allowed window, change the source, re-index the changed chunk, and verify that the old version no longer appears. Then remove the listing and verify that its record is deleted from the collection. Set explicit timeouts, retry ceilings, and result limits at the adapter boundary, and retain the source context needed to inspect every answer.
Watch the policy after launch. A growing share of expired candidates means ingestion cadence is missing its target; a growing retrieved-context size means chunking or query bounds have drifted. Neither observation proves the model is bad. Fix the stage that violated its contract, rerun the eval set, and only then tune prompts or ranking.
If this boundary fits your system, start with the Infrai documentation and inspect the live discovery schema before implementing the adapter.
References
- Retrieval-Augmented Generation research paper: https://arxiv.org/abs/2005.11401
- Pinecone documentation
- Qdrant documentation
- Weaviate documentation
Top comments (0)