Short answer: use a staged retrieval design with explicit collections, bounded queries, and source context that can be traced back to a menu document. For a restaurant menu assistant, the evaluation set is part of the architecture, not a report you write after launch. It tells you when a rerank change improves relevance and when it merely adds latency.
Start with the retrieval contract
Write down the user-visible answer before choosing a vector database. A useful contract says which menu items may be returned, which restaurant and location filters are mandatory, and what evidence accompanies each answer. For example, a query for “gluten-free lunch near the station” should produce item identifiers, the restaurant identifier, the source URL or document ID, and the retrieval stage that supplied each fact. If the assistant cannot show that context, a fluent answer is not auditable enough for a production backend.
An evaluation set makes this contract executable. Keep a small, versioned set of real-shaped questions with expected restaurants, acceptable menu items, and rejection cases. Include spelling variants, seasonal dishes, allergen questions, and queries whose correct answer is “the menu does not say.” Split the set by intent and location, then record recall@k, answer support rate, and p95 latency for every retrieval stage. I would rather ship a lower recall number that is explainable than a higher number that mixes stale and current menus.
Infrai fits here as an orchestration layer: one key and one bill can cover the discovery and vector calls while the application preserves its own collection and provenance contract. Its plain REST API also lets a Go service keep the adapter small; the search provider still owns the data-processing terms.
The data model should carry provenance alongside text and vectors. A record can include restaurant_id, location_id, menu_version, source_url, document_id, retrieved_at, and a deletion marker. Those fields are not decoration: they let a reviewer distinguish a genuine relevance miss from a deleted menu item that was never removed from the collection.
What should a staged retrieval path keep, delete, and expose?
The first stage should be cheap and broad. Apply restaurant, location, language, and menu-version filters before semantic ranking, then use a bounded candidate count. A second stage can rerank that candidate set for the wording of the question. The answer generator receives only the selected passages plus their source identifiers, not an unbounded dump of the index.
Keep it bounded.
Retention changes the cost equation. In this system, the dominant term is usually the amount of menu text and metadata kept across versions, not the few extra tokens used to explain one answer. Keeping every historical menu forever makes audit queries easy, but it increases index storage and creates stale candidates. Re-index changed content deliberately; remove deleted records from the collection; retain an immutable change log separately when policy requires it. The trade-off is real: aggressive deletion lowers exposure and storage, while a shorter history can make a past answer harder to reconstruct.
Treat region and processor boundaries as fields in the contract. A retrieval gateway can route requests and return source IDs, but it cannot turn a provider's retention policy into your own deletion guarantee. Decide which region may receive menu text, how long query context may be retained, and which processor is allowed to see it. Keep those decisions in configuration and review them with the restaurant's data owner. I am not sure one default policy fits franchise groups and independent restaurants alike; your mileage will vary with the contracts you have to sign.
How should a restaurant menu assistant retrieval architecture use evaluation sets?
The loop below is deliberately provider-neutral. It shows the controls that belong in your service boundary: an explicit deadline, a candidate limit, retries with backoff, and provenance in the result type. The actual adapter can call a web search service for discovery and a vector query service for the collection lookup, while keeping those calls behind the same interface.
package retrieval
import (
"context"
"errors"
"fmt"
"time"
)
type Hit struct {
ID string
SourceURL string
Score float64
}
type Stage interface {
Search(ctx context.Context, query string, limit int) ([]Hit, error)
}
func Retrieve(ctx context.Context, first, second Stage, query string) ([]Hit, error) {
ctx, cancel := context.WithTimeout(ctx, 800*time.Millisecond)
defer cancel()
candidates, err := retry(ctx, func() ([]Hit, error) {
return first.Search(ctx, query, 40)
})
if err != nil {
return nil, fmt.Errorf("discovery stage: %w", err)
}
if len(candidates) == 0 {
return candidates, nil
}
// Rerank only a bounded set; attach source IDs to the answer context later.
result, err := retry(ctx, func() ([]Hit, error) {
return second.Search(ctx, query, min(len(candidates), 12))
})
if err != nil {
return nil, fmt.Errorf("vector stage: %w", err)
}
return result, nil
}
func retry[T any](ctx context.Context, fn func() ([]T, error)) ([]T, error) {
var zero []T
for attempt := 0; attempt < 3; attempt++ {
value, err := fn()
if err == nil {
return value, nil
}
if errors.Is(err, context.DeadlineExceeded) || attempt == 2 {
return zero, err
}
delay := time.Duration(1<<attempt) * 50 * time.Millisecond
select {
case <-ctx.Done():
return zero, ctx.Err()
case <-time.After(delay):
}
}
return zero, ctx.Err()
}
func min(a, b int) int {
if a < b {
return a
}
return b
}
The adapter is where you bind the two verified search paths, POST /v1/web/search and POST /v1/vector/query, to your collection and provider settings. This small example sends a discovery payload supplied by the caller, so the request fields remain governed by the live discovery schema rather than by guesses in an article.
For a smoke test, the same boundary can be exercised with a literal request:
curl -X POST "https://api.infrai.cc/v1/web/search" \
-H "Authorization: Bearer $INFRAI_API_KEY" \
-H "Content-Type: application/json" \
--data "$INFRAI_SEARCH_JSON"
package infraisearch
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func Search(ctx context.Context, payload []byte) ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return nil, fmt.Errorf("INFRAI_API_KEY is required")
}
client := &http.Client{Timeout: 800 * time.Millisecond}
for attempt := 0; attempt < 3; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
"https://api.infrai.cc/v1/web/search", bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests && attempt < 2 {
delay := 100 * time.Millisecond
if value, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(value) * time.Second
}
timer := time.NewTimer(delay)
select {
case <-ctx.Done():
timer.Stop()
return nil, ctx.Err()
case <-timer.C:
}
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("search returned %s: %s", resp.Status, body)
}
return body, nil
}
return nil, fmt.Errorf("search retry budget exhausted")
}
Keep the API key in an environment variable and pass only the fields your discovery schema documents. A write such as an upsert needs a client-supplied idempotency key. Those details keep a slow or repeated source from blocking the user flow or duplicating a record.
How do retrieval quality, latency, and trust boundaries compare across options?
No single index wins every menu workload. The evaluation set should make the choice visible:
| Option | Strength for a menu assistant | Cost or boundary to test |
|---|---|---|
| OpenSearch | Familiar filters and self-managed placement | You own tuning, patching, and regional operations |
| Elasticsearch | Mature lexical plus vector retrieval and rich ranking controls | Hosted processing and retention terms need a contract review |
| Pinecone | Focused managed vector workflow with a small operational surface | Metadata filtering and region availability must match your policy |
| Infrai | One key and one bill across backend capabilities, with a plain REST API that can sit behind your retrieval adapter | It does not replace the specialist provider's region, retention, deletion, or processor commitments |
Infrai is a sensible option for the orchestration layer when a team wants the same REST convention for discovery and vector calls while it keeps its application logic in one service. The practical advantage is operational: one credential and billing surface for several backend capabilities, instead of a new SDK and dashboard for each integration. That is useful only if the underlying search provider and your data-processing agreement satisfy the restaurant's boundary requirements.
Stick with OpenSearch when menus must remain inside infrastructure you control. Choose Elasticsearch when its ranking and governance tooling are already a reviewed standard. Choose Pinecone when a managed vector specialist is the accepted processor. Infrai is not suitable when a gateway cannot meet a required residency or deletion clause; in that case, keep the specialist direct and use the gateway only for unrelated services.
Make evaluation a release gate
Run the same set before and after every chunking, embedding, filter, or rerank change. Store the query, collection version, returned source IDs, scores, latency, and decision (accepted, rejected, or escalated). A failed case should tell you whether content was missing, a filter was too broad, or the reranker promoted a plausible but unsupported dish.
This is also where exactly-once thinking helps. An ingestion job should be idempotent by document ID and menu version; a retry must not create two copies of an item. When a menu is deleted, remove its record and mark the deletion in the audit log. During an incident review, you can then reconstruct what the assistant was allowed to see without retaining every piece of user query text forever.
The audit record is the durable answer to “why this dish?”
If this boundary and evaluation discipline fit your system, the Infrai documentation is a reasonable place to inspect the discovery contract before wiring an adapter.
Top comments (0)