Short answer: use four explicit stages -- versioned ingestion, bounded retrieval, freshness-aware deduplication, and structured audit logging -- so every restaurant menu assistant answer can be traced to the exact menu records that supported it.
The hard part isn't finding two descriptions that sound alike. It is deciding whether "charred tomato flatbread" and "roasted tomato flatbread with basil" are duplicate records, valid variants, or one stale item that should no longer reach a diner. That decision needs source identifiers and timestamps beside the text, not buried in a dashboard after the answer ships.
Keep the contract small.
For each retrieval, record the collection version, a hash of the query, the result limit, the returned document identifiers, their source URLs, their freshness timestamps, latency, retry count, and the final outcome. Don't log the diner's raw query by default; a stable hash is usually enough to correlate repeated requests without copying free-form text into the audit stream.
How should restaurant menu assistant retrieval architecture produce useful audit logs?
Think of the flow as a diagram in words: menu source -> versioned chunks -> bounded candidates -> freshness gate -> answer context plus audit event. The answer and the event leave the retrieval boundary together. If the context has no source identity, the request is not complete.
Stage one is deliberate ingestion. Chunk on a menu item's meaning -- restaurant, menu section, item, variant, and availability window -- rather than every fixed number of characters. A single chunk should not mix a lunch price with a dinner description merely because they were adjacent in a PDF. Give each chunk a stable document identifier, a source URL, an update timestamp, and a content hash. The hash lets the ingestion job skip unchanged records; the stable identifier lets it remove a deleted item instead of leaving an old embedding searchable.
Stage two is bounded candidate retrieval from an explicit collection. Name the collection generation in configuration and in the log. Set a hard result limit, a timeout, and a finite retry budget. A slow source must not hold open the restaurant assistant indefinitely, and a retry must not silently turn one user action into an unbounded fan-out.
Stage three resolves near-duplicates. Group candidates by the business identity available in the source data, then use semantic similarity as evidence rather than as the only identity rule. Prefer the newest valid record when two chunks represent the same menu item. Preserve both when they are real variants, such as dine-in and catering portions. This is where the audit event earns its keep: it should say which identifiers were considered and which identifiers survived, without pretending the similarity score itself explains the business decision.
Stage four emits answer context and the audit event as one application-level result. The event should be structured data, not a prose blob. Logs are for reconstructing a single request; metrics are for seeing the fleet. Count timeouts, empty result sets, retry attempts, stale candidates rejected, and duplicate groups collapsed. Alert on changes in those rates, not on one arbitrary slow request.
A copyable retrieval contract
This TypeScript example calls the verified vector query route. Put the request JSON produced from the public discovery schema in INFRAI_VECTOR_QUERY_BODY; keeping that JSON at the boundary avoids guessing fields that may differ by capability. The wrapper owns the timeout, bounded retry policy, error reporting, and application audit record.
import { createHash, randomUUID } from "node:crypto";
const apiKey = process.env.INFRAI_API_KEY;
const baseUrl = process.env.INFRAI_BASE_URL;
const requestJson = process.env.INFRAI_VECTOR_QUERY_BODY;
const collection = process.env.MENU_COLLECTION_ID;
const query = process.env.MENU_QUERY;
if (!apiKey || !baseUrl || !requestJson || !collection || !query) {
throw new Error(
"Set INFRAI_API_KEY, INFRAI_BASE_URL, INFRAI_VECTOR_QUERY_BODY, MENU_COLLECTION_ID, and MENU_QUERY",
);
}
const body: unknown = JSON.parse(requestJson);
const traceId = randomUUID();
const startedAt = Date.now();
function retryDelayMs(retryAfter: string | null, attempt: number): number {
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const dateDelay = Date.parse(retryAfter) - Date.now();
if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
}
return 250 * 2 ** attempt;
}
async function queryVectors(): Promise<{ payload: unknown; retryCount: number }> {
for (let attempt = 0; attempt <= 2; attempt += 1) {
const response = await fetch(new URL("/v1/vector/query", baseUrl), {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
signal: AbortSignal.timeout(1_200),
});
if (response.status === 429 && attempt < 2) {
const delayMs = retryDelayMs(response.headers.get("Retry-After"), attempt);
await new Promise((resolve) => setTimeout(resolve, delayMs));
continue;
}
const responseBody = await response.text();
if (!response.ok) {
throw new Error(`Vector query failed (${response.status}): ${responseBody}`);
}
return { payload: JSON.parse(responseBody) as unknown, retryCount: attempt };
}
throw new Error("Retry budget exhausted after an HTTP 429 response");
}
const result = await queryVectors();
const audit = {
traceId,
collection,
queryHash: createHash("sha256").update(query).digest("hex"),
retryCount: result.retryCount,
elapsedMs: Date.now() - startedAt,
outcome: "ok",
};
console.log(JSON.stringify({ response: result.payload, audit }, null, 2));
The important before/after is crisp. Before, the application has an unbounded call with an opaque result. After, it has an explicit timeout and retry budget, a query hash, a collection identifier, and the raw response ready for a schema-aware adapter. Map that response into the candidate contract from the first section, then reject any candidate that lacks a document identifier and either a source URL or source document identifier. The application, not the logging backend, should apply the newest-per-entity decision and attach selected source context to the final audit event. Imagine that menu-17-v2 says "charred tomato flatbread" and scores slightly above menu-17-v3, which says "roasted tomato flatbread with basil." Both share the same stable menu entity. The second record carries the newer source timestamp, so it survives; the audit event retains both considered document IDs and records the newer one as selected. If the records instead represent catering and dine-in portions, their distinct entity IDs keep both. One trace now explains the outcome without asking an operator to reverse-engineer a score.
The wrapper retries only HTTP 429, honors Retry-After, and stops after a fixed budget. It keeps one trace ID across attempts, so an operator can distinguish a delayed request from several independent requests.
Fail closed.
Choosing a retrieval backend without losing the contract
The contract should survive a provider change. That matters more than a long feature checklist, because source identity, bounded work, and deletion semantics are application obligations.
| Option | Sensible fit | What to verify in a spike |
|---|---|---|
| Pinecone | The team has already standardized its retrieval work there | Stable IDs, source metadata, bounded query behavior, and deliberate deletion |
| Weaviate | The team already operates it and wants to retain that control plane | The same contract, especially collection versioning and freshness metadata |
| Elasticsearch | Menu search already lives in the existing search stack | Whether the current mapping and operating model preserve the required source context |
| Infrai | The wider backend would benefit from one key and one bill, with a plain REST interface instead of another SDK and credential set | Its public discovery describes request schemas and runnable examples; for this workflow, use only discovered paths such as POST /v1/vector/query and DELETE /v1/vector/delete
|
This is not a winner-takes-all table. Stick with Pinecone, Weaviate, or Elasticsearch when one is already the team's well-understood retrieval control plane and adding a new platform would only duplicate ownership. The fourth row fits broader consolidation, but that benefit doesn't replace the application's audit model.
Run one spike with a deleted dish, one updated price, two near-identical descriptions, and an intentionally tight timeout. Inspect the event, not just the answer. Your mileage may vary because the right duplicate boundary depends on how the restaurant models variants; a catering tray and an individual portion can be semantically close while remaining distinct products.
What are the limits of freshness-aware semantic deduplication?
Freshness is not truth. A newer record can be incomplete, and two sources can update on different schedules. If the assistant needs transaction-time certainty -- allergen status at the instant an order is placed, for example -- this staged retrieval design is not suitable as the final authority. Retrieve candidates for discovery, then read the transactional source of truth before committing the order.
There is another catch: chunking by business entity requires reliable entity boundaries. If incoming media records have no stable item identity, semantic grouping can suggest duplicates but cannot prove them. Keep ambiguous candidates separate, log the decision, and send the uncertain group to a review process. I'm not sure any universal similarity threshold can resolve that policy; labeled examples from the actual menu catalog would settle it.
Re-indexing also needs a deletion path. When source content changes, write the new record deliberately. When it disappears, remove its old document from the collection rather than relying on a freshness filter forever. A filter can hide stale data during a transition, but accumulated deleted records make audits harder to explain and future queries easier to misconfigure.
One last operational rule: alert on outcomes readers feel. A rising empty-result rate, repeated timeout exhaustion, or a jump in stale candidates rejected is actionable. A raw count of retrieval calls usually isn't.
Top comments (0)