How I'd Track Freshness Across Training, RAG, and Live Fetches

I keep freshness metadata outside the model and pass the relevant parts into answer generation. A model response alone cannot tell me when its supporting evidence was last observed.

The distinction I use is simple: training changes parameters, retrieval supplies selected external content, and a live fetch adds a new source observation. A single answer can involve all three.

I keep four timestamps separate

Timestamp	Meaning
source_updated_at	Source's stated update time, when available
fetched_at	When my collector observed the source
indexed_at	When that observation entered the index
answered_at	When the application generated the answer

I don't fill an unknown source update time with the fetch time. They describe different events. A page fetched today may contain an old statement.

Here is illustrative metadata for a retrieved document:

{
  "document_id": "example-policy",
  "document_version": "example-v3",
  "source_url": "https://example.com/policy",
  "source_updated_at": null,
  "fetched_at": "2026-10-01T08:00:00Z",
  "indexed_at": "2026-10-01T08:05:00Z",
  "access_scope": "example-team",
  "extraction_version": "example-v2"
}

I preserve the source version or content hash through chunking. Every retrieved chunk should still point to the observation that produced it.

My freshness rule belongs to the question

I classify requests by the kind of evidence they need. A conceptual explanation may use a stable reference. A policy question needs an approved document version. A current price or inventory question may require an authoritative live call and confirmation before action.

The age tolerance should be explicit and source-specific. I avoid one global “fresh for 24 hours” rule, because it treats all decisions as equally sensitive to change.

If evidence is too old, the application can refresh the source, use a clearly labeled older observation, or decline the current-state claim. The correct choice depends on the product requirement.

I test each boundary independently

My evaluation fixture would include a source document with a fact I can deliberately change. I would then inspect five stages: acquisition, indexing, retrieval, citation support, and final answer.

If acquisition sees the new text but the index keeps the old version, the repair belongs in ingestion. If the index is current but retrieval selects an older duplicate, the repair belongs in retrieval. If the right passage is present but the answer loses a qualification, the repair belongs in generation or prompting.

The RAG paper is useful background for the separation between model parameters and external memory. It does not establish that a particular implementation has correct refresh behavior; these boundary tests do.

Live failures get an explicit answer state

I would simulate a timeout and an unavailable source. The system should not silently use an old cache entry while presenting the result as newly verified.

My answer metadata would record evidence mode, source observation time, and whether a required refresh succeeded. The UI can turn that into a short useful statement instead of exposing all implementation details.

For important facts, I also check that the citation supports the exact claim. Topic relevance is weaker than claim support. A correct URL can still accompany an unsupported sentence.

Deletion is part of freshness

Removing a document should affect raw storage where required, the retrieval index, answer caches, and logs according to the retention policy. I test that a removed or newly restricted passage stops reaching unauthorized prompts.

Access filtering belongs before the model receives context. Filtering the final answer after retrieval is too late to protect the prompt from unauthorized content.

My first dashboard would show source age, indexing lag, retrieval version, and refresh failures alongside answer quality. That gives me a concrete way to explain a stale answer and verify that a fix reached the intended boundary.