Short answer: give each restaurant menu fact an explicit freshness deadline, reject expired evidence before semantic ranking, and page on freshness-budget burn before the assistant starts answering from an old index. For a campus dining assistant inside an education platform, this is the least complex design that keeps schedule changes visible without rebuilding every embedding on every poll.
The page arrives at 02:13: menu_answer_stale_ratio has crossed the alert threshold for the third evaluation interval. The on-call view shows that retrieval is healthy by its usual measures: requests complete, candidates come back, and the vector index responds. Yet a dining-hall answer still cites yesterday's dinner hours. A green retrieval service can return bad evidence.
That is the operational trap.
The signal that should have fired earlier is not generic latency or an empty-result count. It is the age of the newest eligible source record compared with the freshness budget for that fact class. Menu descriptions can tolerate a longer delay than an availability change; serving hours sit somewhere between them. Those are policy decisions, not properties of an embedding model.
The page is about evidence age, not search uptime
A retrieval-augmented system has two clocks. One measures whether the retrieval path is available now. The other measures how far the searchable evidence trails the system of record. Only the second clock explains a fluent answer built from an obsolete menu. Retrieval-augmented generation joins model output with non-parametric memory, but the original RAG formulation does not choose an application freshness policy for you [1].
Start the incident trace at the answer. Record the answer timestamp, the identifiers and source versions of retrieved chunks, their observed_at values, the policy selected for the query, and whether the assistant abstained. Then walk backward through ranking, indexing, ingestion, and the source feed. The first stage whose age exceeds its budget owns the alert. This prevents a common postmortem failure: blaming the model because it was the final component to speak.
For a campus dining system, a useful record is not just an embedding and some text. It needs a stable menu-item identifier, restaurant or dining-hall identifier, source version, valid_from, observed_at, expires_at, content hash, and embedding version. Keep schedule validity separate from observation time. A brunch listing can be observed this morning while being valid only next Sunday; collapsing those fields makes both incident review and time-aware retrieval ambiguous.
Treat freshness as an answer eligibility rule. At query time, filter out records whose validity interval does not contain the requested meal time and whose observation age exceeds the configured window. Rank only the survivors. If none survive, the assistant should say that current menu evidence is unavailable rather than quietly widening the window. That's a product behavior worth putting in the runbook because it turns hidden staleness into an explicit, measurable state.
How should a restaurant menu assistant set retrieval freshness windows?
Set windows by the cost of a wrong answer and by how quickly the source can legitimately change. Don't start with one global time-to-live. A practical initial policy can separate immediate operational facts, daily schedule facts, and descriptive copy. The exact durations below are examples for a pre-production policy review, not universal values; validate them against the dining data owner's update process and observed ingestion lag.
| Fact class | Example starting window | Expiry behavior | Why |
|---|---|---|---|
| Availability or closure | 5 minutes | Abstain or ask the user to verify | A stale answer can send someone to an unavailable option |
| Serving hours and meal period | 30 minutes | Exclude expired chunks | The answer depends on a bounded service interval |
| Item name and description | 24 hours | Continue only if still source-valid | Copy usually changes less often than operations |
I'm not sure those values fit your source. Nobody can be sure from the query path alone. Resolve the uncertainty with source change logs: measure the distribution from source update to searchable version, then choose a service-level objective and alert threshold that leave room for detection and repair. Your mileage may vary when dining operators batch updates instead of publishing events.
Freshness windows also need a precedence rule. If an item description is current but its availability record is expired, the combined answer inherits the shorter operational window. Otherwise a long-lived descriptive chunk can smuggle an expired claim into the response. Store provenance per fact, not merely per assembled document, when records with different lifetimes are merged.
The catch is that strict windows reduce answer coverage. This design is not suitable when the only source updates weekly but users expect live stock status; no retrieval policy can manufacture data that the source never publishes. In that case, narrow the assistant's claims to catalog information, or integrate a more timely system of record before tightening the window. Likewise, stick with a simple lexical lookup when the menu is small, names are stable, and semantic matching adds no demonstrated recall benefit.
Instrument the scheduler before tuning the retriever
The early-warning signal belongs at the boundary between source observation and searchable publication. Track source_to_index_age_seconds by restaurant and fact class, plus counts for accepted, unchanged, superseded, and expired records. A histogram of indexing duration helps, but it cannot replace a gauge for the age of the newest searchable version. Fast processing of an old batch is still stale.
This Go sketch keeps the policy and telemetry independent of any vendor. It uses a content hash to avoid needless semantic work, records the lag after publication, and carries source versions into the index. Production code also needs durable queue semantics and a transactional handoff between the source cursor and the job record; the interface makes that ownership visible.
package menuindex
import (
"context"
"crypto/sha256"
"time"
)
type MenuRecord struct {
ID string
RestaurantID string
SourceVersion string
Text string
ObservedAt time.Time
ValidFrom time.Time
ExpiresAt time.Time
}
type IndexRecord struct {
MenuRecord
ContentHash [32]byte
Embedding []float32
}
type Store interface {
CurrentHash(ctx context.Context, id string) ([32]byte, bool, error)
Upsert(ctx context.Context, record IndexRecord) error
}
type Embedder interface {
Embed(ctx context.Context, text string) ([]float32, error)
}
type Metrics interface {
ObserveIndexAge(factClass string, age time.Duration)
Increment(result string)
}
func Publish(ctx context.Context, now time.Time, record MenuRecord, factClass string, store Store, embedder Embedder, metrics Metrics) error {
hash := sha256.Sum256([]byte(record.Text))
current, found, err := store.CurrentHash(ctx, record.ID)
if err != nil {
return err
}
if found && current == hash {
metrics.Increment("unchanged")
metrics.ObserveIndexAge(factClass, now.Sub(record.ObservedAt))
return nil
}
vector, err := embedder.Embed(ctx, record.Text)
if err != nil {
return err
}
if err := store.Upsert(ctx, IndexRecord{
MenuRecord: record, ContentHash: hash, Embedding: vector,
}); err != nil {
return err
}
metrics.Increment("published")
metrics.ObserveIndexAge(factClass, now.Sub(record.ObservedAt))
return nil
}
Scheduling must be idempotent. I've been paged by missed jobs and duplicate deliveries; both conditions make a naive indexer lie in different directions. A missed run ages the index. A duplicate run can inflate work and, if writes are not version-aware, let an older completion overwrite a newer record. Use the stable item identifier plus source version as the idempotency key, compare versions before publishing, and make retries safe. An at-least-once queue is manageable when the consumer treats repetition as normal.
The scheduler should emit a heartbeat with the source cursor and newest observation time even when no records change. Without it, zero updates and a dead poller look identical. Alert first on freshness-budget consumption, then escalate when the budget is exhausted. For example, warning at a configurable fraction of the window gives the operator time to inspect source lag, queue age, and index publication lag before users see stale evidence. Choose that fraction from your response-time objective rather than copying a canned percentage.
Error handling follows the same boundary. Retry transient dependency errors with bounded backoff, send records that repeatedly fail validation to a review queue, and never advance the durable source cursor until the corresponding work is recoverable. Record why a document was rejected. Silence is not success.
How can semantic deduplication lower index cost without hiding fresh records?
A campus education platform may ingest the same dish from a dining-hall feed, a nutrition feed, and a manually curated announcement. The text differs enough to evade exact hashes: roasted tomato soup, tomato soup - roasted, and a sentence describing today's roasted tomato soup may refer to one serving record. Embedding every copy on every poll spends index capacity without improving the answer, but aggressive clustering can merge records that differ in location, meal period, allergens, or validity.
Cost first.
Use a staged deduplication pipeline. Normalize harmless presentation differences first, then remove byte-identical content with a hash. Generate semantic candidates only within a bounded partition such as restaurant, meal period, and overlapping validity interval. Embed changed candidates, compare them with nearby active records, and preserve the source records behind the canonical retrieval document. The canonical record should retain all contributing source versions so an answer can be traced and an expired contribution can be removed without guessing. This is where Postgres can be a sensible baseline: relational columns can enforce stable identifiers and validity constraints while a vector extension supports candidate similarity in the same operational store. It is not an automatic recommendation. At high vector volume, specialized indexing or a separate retrieval tier may offer a better latency and operations profile; at very small scale, trigram or full-text similarity may be sufficient. Benchmark with the actual menu corpus, update rate, and recall target. Index cost is driven by more than stored vector count. Count embedding requests for changed content, index writes, retained historical versions, compaction work, and the operator time needed to reconcile split stores. Report cost per successfully published source version and per answer with eligible evidence. Those denominators expose waste that a monthly infrastructure total hides. A low-cost index that routinely rebuilds unchanged descriptions is not efficient. Deployment should shadow the new freshness policy before it filters production answers. For each query, log what the current policy retrieved and what the candidate policy would exclude, without showing the shadow result to users. Review false exclusions by fact class and restaurant. Then enable enforcement for a small partition, watch abstention rate and freshness-budget burn, and expand only after the alert and rollback paths are exercised. The closing trade-off is false-positive cost. A freshness alert set too close to ordinary update jitter pages the on-call for healthy batches, and a window set too short suppresses correct answers. Set it too wide, though, and the first reliable monitor becomes a user reporting yesterday's menu. The right threshold is the narrowest window the source cadence and response process can sustain, backed by an explicit abstention path. Measure both stale-answer escapes and valid-answer exclusions; optimizing only one moves the failure to the other side of the page.
References
- Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks": https://arxiv.org/abs/2005.11401
Top comments (0)