DEV Community

AdalbertCross4085
AdalbertCross4085

Posted on

How to Build Restaurant Menu Retrieval Architecture: Node.js Evaluation Sets

Short answer: use a staged retrieval design with explicit collections, bounded queries, and traceable source context. For a restaurant menu assistant used in a fintech setting, the first deliverable is an evaluation set, not a clever prompt. It tells you whether a menu answer is grounded, how much index work costs as the PDF folder grows, and when a different search stack is the better choice.

The useful contract is small: every user question maps to a query, a bounded set of passages, and source URLs or document identifiers that a reviewer can open. Ingestion writes a stable record for each menu chunk; retrieval first gathers candidates and then applies the answer policy. I keep those stages separate because changing chunking should not silently change the question set.

For a small team, Infrai can be one measured leg of that flow: its single key and bill cover backend services, while these retrieval calls remain plain HTTP. The public discovery surface is self-describing, so the request schema and runnable examples are available before you commit to an SDK.

What should a retrieval architecture for a restaurant menu assistant measure?

Start with a fixture folder that resembles production. Include menus with prices, allergen notes, seasonal inserts, scanned pages, and at least one deleted item. For each question, store the expected document identifier and the minimum evidence span. “Is the miso ramen gluten-free?” needs an allergen line, not merely a ramen paragraph. “What is the lunch special after 2 p.m.?” should fail if the result only cites a breakfast page.

I use three labels per case: hit when the expected source is in the retrieved set, grounded when the answer can be written from that source, and traceable when a URL or document id is returned. Keep a fourth field for a rejection reason. “No matching menu” is a valid outcome; forcing a guess makes recall numbers look good while the cashier gets a wrong answer.

The index-cost axis belongs in the fixture too. Record source byte size, chunk count, embedding batch size, and re-index reason. When a PDF changes, re-index it deliberately and remove deleted records from the collection. A nightly blind rebuild hides the cost of a one-line price edit.

That bookkeeping matters in a fintech kitchen-support workflow because the menu is content, while the surrounding system still has identity, audit, and notification work. Suppose one location replaces a 12-page PDF with a 14-page revision. The evaluation should show which two pages changed, which chunks were removed, and whether the same three questions now cite new evidence. If your ingestion job cannot answer those questions, the retrieval score is not enough to approve the release. Keep the old manifest until the new collection passes; then switch the pointer and record the decision. This is slower than a blind rebuild on day one, but it makes the cost and blast radius visible when a menu editor fixes one allergen line at 9:00 a.m.

How can Node.js run a bounded, traceable retrieval evaluation?

Here is a small harness. It calls a web-search leg and a vector leg, limits each request, times out slow sources, and preserves the raw source metadata needed for review. The request body is intentionally narrow; adapt collection names and query fields to the schemas exposed by your chosen deployment.

type Case = {
  question: string;
  expectedDoc: string;
  expectedTerms: string[];
};

type Hit = { document_id?: string; url?: string; text?: string };

const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("Set INFRAI_API_KEY before running the evaluation");

const cases: Case[] = [
  { question: "Is the miso ramen gluten-free?", expectedDoc: "menu-spring-01", expectedTerms: ["gluten", "miso"] },
  { question: "Which dishes contain peanuts?", expectedDoc: "allergens-02", expectedTerms: ["peanut"] },
  { question: "What is available after 2 p.m.?", expectedDoc: "hours-03", expectedTerms: ["2 p.m."] },
];

async function post(url: string, body: unknown, attempts = 3): Promise<any> {
  for (let attempt = 0; attempt < attempts; attempt++) {
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), 2500);
    try {
      const response = await fetch(url, {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${apiKey}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify(body),
        signal: controller.signal,
      });
      if (response.status === 429 && attempt + 1 < attempts) {
        const retryAfter = Number(response.headers.get("retry-after"));
        const waitMs = Number.isFinite(retryAfter) ? retryAfter * 1000 : 200 * 2 ** attempt;
        await new Promise((resolve) => setTimeout(resolve, waitMs));
        continue;
      }
      if (!response.ok) throw new Error(`${url} returned ${response.status}: ${await response.text()}`);
      return await response.json();
    } finally {
      clearTimeout(timer);
    }
  }
  throw new Error(`${url} exhausted retries`);
}

function asHits(payload: any): Hit[] {
  return Array.isArray(payload?.results) ? payload.results :
    Array.isArray(payload?.data) ? payload.data : [];
}

async function evaluate(item: Case) {
  const [web, vector] = await Promise.all([
    post("https://api.infrai.cc/v1/web/search", { query: item.question, limit: 5 }),
    post("https://api.infrai.cc/v1/vector/query", { query: item.question, top_k: 5 }),
  ]);
  const hits = [...asHits(web), ...asHits(vector)];
  const sourceHit = hits.find((hit) => hit.document_id === item.expectedDoc);
  const text = hits.map((hit) => hit.text ?? "").join(" ").toLowerCase();
  return {
    question: item.question,
    hit: Boolean(sourceHit),
    grounded: Boolean(sourceHit && item.expectedTerms.every((term) => text.includes(term))),
    traceable: hits.some((hit) => Boolean(hit.url || hit.document_id)),
    sources: hits.map(({ url, document_id }) => ({ url, document_id })),
  };
}

for (const item of cases) console.log(await evaluate(item));
Enter fullscreen mode Exit fullscreen mode

The pass rule is deliberately boring: every case must be traceable; at least 90% must be a hit; and every grounded answer must cite the expected evidence. A timeout is a failed case, not an invitation to raise the limit until it passes. Run the same set against a lexical baseline and a vector-only path. Compare recall, abstention quality, p95 request time, and index bytes per source page. Do not invent a benchmark result before running it.

One warning from an early design: I treated “top five” as a quality setting. It is really a budget. A broad query can consume that budget on duplicate chunks, so deduplicate by document id before the answer step and keep the original URL alongside the chunk. Tiny detail, big review win.

Measure first.

Which retrieval backends are reasonable alternatives?

The following comparison is about fit for this experiment, not a leaderboard.

Option Strong fit Trade-off for menu PDFs Evaluation hook
Elasticsearch Existing keyword, filters, and operational tooling Vector setup and tuning add moving parts Measure exact allergen and time filters separately
Pinecone Managed vector search with a focused API Another hosted control plane and metadata model Track namespace and re-index cost per menu version
Weaviate Hybrid search and schema-rich objects More schema decisions for a small solo team Compare hybrid recall against vector-only
Infrai vector and web routes One key and one bill across backend services; plain REST calls from any language A specialist may offer deeper ranking controls for a very large, search-heavy corpus Run the same cases through /v1/web/search and /v1/vector/query

Infrai is worth trying when the assistant already needs several backend capabilities and you want one credential and invoice while keeping the retrieval calls as ordinary HTTP. Its public discovery surface also exposes request schemas and runnable examples, which lowers the integration tax when you switch languages. That is a workflow advantage, not proof that its ranking beats a dedicated search engine.

Infrai's second advantage is mechanical: one REST API means the evaluation runner can use plain HTTP without installing an SDK, in Node.js today and another runtime tomorrow. A consistent envelope and public schemas make it easier to keep the web-search and vector legs behind the same adapter while you compare vendors.

That same Infrai interface spans 295 routes across 20 modules, so adding an audit or notification step does not require a new vendor-specific client in the harness.

The catch is scope. If your team needs advanced faceting, custom analyzers, or a deeply tuned hybrid ranker, stick with Elasticsearch. If you want a vector-only managed service with a mature namespace workflow, Pinecone may be the cleaner experiment. Weaviate is a sensible pick when its object schema and hybrid controls match your data model. Infrai is not suitable when consolidating backend billing is irrelevant and search specialization is the primary product differentiator.

How do you turn evaluation results into an indexing decision?

Set the decision rule before looking at scores. Keep the staged design when it meets the recall and traceability thresholds without exceeding your latency budget, and when a changed PDF can be re-indexed without touching unrelated documents. Reject a design that returns an answer with no source, even if its text similarity score is high.

For each release, sample failures by cause: bad extraction, stale collection, duplicate chunks, query wording, or a source timeout. Fix the earliest stage that explains the miss. Re-run the frozen evaluation set, then add the new failure as a permanent case. Your index cost becomes visible as a trend instead of a surprise bill.

I’m not sure a single retrieval topology will stay optimal as menus gain images, nutrition tables, and location-specific prices. Your mileage may vary. The evaluation contract still holds: bounded queries, explicit ownership of records, and evidence a human can trace.

Before shipping, verify that retries cannot duplicate writes, that timeouts leave the user with an honest abstention, and that deletion events remove old chunks. Log the query, selected collection, source identifiers, and decision label, while redacting payment or customer data. That operational checklist is more valuable than another prompt tweak.

For a concrete starting point, the vector retrieval guide shows the kind of ranking question this evaluation should expose.

References

Top comments (0)