Source-Aware LLM Cache Invalidation: Build a Freshness Contract

The support answer arrives in 80 milliseconds, sounds clear, and is wrong. The refund policy changed an hour ago, but a semantic cache is still returning an answer generated from the old document. The model did not hallucinate. The system served yesterday's work as if it were current.

That distinction matters. A better prompt, larger model, or reranker will not repair a stale cached response. The fix is a freshness contract: a small, explicit agreement that says what an answer depends on, which changes make it unsafe, and how the system proves it is still eligible to serve.

This guide shows how to build source-aware LLM cache invalidation for a production AI application. The goal is not to pick a cache product. It is to make exact, semantic, and retrieval-adjacent caches fast without turning old policies, revoked access, or prompt releases into confident answers.

Why LLM cache invalidation is different

Traditional response caching often has one obvious source of truth: a database record or an HTTP response. An AI answer can depend on several moving inputs at once:

Dependency	Example change	What becomes unsafe
Source evidence	A policy page is edited	Answers quoting the old policy
Access scope	A user loses a role	Answers containing restricted facts
Prompt or tool policy	A system rule changes	Cached behavior based on old instructions
Model or embedding version	A route is updated	Semantic matches and quality assumptions
Live business state	An order ships	Personalized status answers

A time-to-live (TTL) covers only one of these changes, and only eventually. A 24-hour TTL means a known-bad answer may remain eligible for almost a day. For low-risk, slow-changing help text that may be acceptable. For account status, pricing, permissions, or policy, it is not.

The useful mental model is simple: a cache entry is not merely question -> answer. It is an artifact with a provenance record. Serve it only when its provenance still matches the request and the current system.

Start with four cache layers, not one

Teams often call all reuse an “LLM cache,” even though the risk is different at each layer.

1. Prompt or prefix caching reuses model work for an identical stable prefix. It improves cost and latency, but still produces a new answer.
2. Exact response caching returns a prior answer for the same normalized request. It is deterministic, but can still be stale.
3. Semantic caching returns an answer for a similar request. It adds a correctness decision: similar wording is not necessarily the same authorization or intent.
4. Retrieval and tool-result caching reuses source material before generation. A stale result here can poison every new answer built on top of it.

Do not give these layers one global TTL. A prefix cache may be safe to retain while an answer cache must be purged. A cached retrieval result may need to disappear after a document deletion even if the document's replacement has not finished embedding.

Define the freshness contract

For each cacheable route, define five things before writing a key.

1. Scope: who is allowed to receive this?

Scope must be a hard lookup boundary, not a filter applied after semantic search. At minimum, include the tenant. Add user or role identifiers whenever an answer could include personalized facts or privileges.

type CacheScope = {
  tenantId: string;
  principalId?: string;
  roleSetHash: string;
  region: string;
};

A cache key such as support:refund-policy:<query> is risky in a multi-tenant application. A safer key namespaces the candidate set before lookup:

const namespace = [
  "support-answer",
  scope.tenantId,
  scope.principalId ?? "shared",
  scope.roleSetHash,
  scope.region,
].join(":");

If a response is safe for everyone, make that a deliberate route decision. Do not assume that because the prompt looked generic.

2. Dependencies: what facts made this answer true?

Store versions for every dependency that can change the answer. A compact entry might look like this:

type CachedAnswer = {
  value: string;
  namespace: string;
  createdAt: string;
  expiresAt: string;
  sourceVersions: Record<string, string>;
  promptVersion: string;
  modelRouteVersion: string;
  embeddingVersion?: string;
  policyVersion: string;
  provenance: Array<{ sourceId: string; chunkId?: string }>;
};

sourceVersions is the important field. It lets a policy update invalidate answers tied to policy/refunds, without flushing every support answer. Use a content hash, monotonically increasing revision, or change-feed sequence number. The value does not need to be human-friendly; it needs to change reliably.

3. Freshness budget: how old is acceptable?

Set a maximum age by route and cost of error, not by cache technology. A product-status answer might have a zero-minute budget and skip response caching. A general troubleshooting explanation may be safe for several days.

Route	Example freshness budget	Response-cache policy
Account balance or entitlement	0–60 seconds	Usually bypass
Pricing or policy explanation	Minutes to hours	Source-event purge + short TTL
Product documentation	Hours to days	Source-event purge + TTL
Stable writing transformation	Longer TTL	Exact cache, no external evidence

TTL remains useful as a backstop for missed events. It is not the primary correctness mechanism for information that changes through known workflows.

4. Invalidation triggers: which events revoke eligibility?

Write this down as a route-level table. It prevents a common failure where the indexing pipeline knows a document changed but the answer cache never hears about it.

Trigger	Action
Document updated or deleted	Mark entries citing that source version stale
Permission or role change	Purge the affected principal or role namespace
Prompt, tool schema, or policy release	Invalidate the route version
Embedding migration	Rebuild retrieval entries; retire old semantic indexes
Model-route change	Evaluate whether answer entries remain acceptable; version the route if not

5. Proof: what is recorded when an entry is served?

Log the decision, not just the hit. A trace should say served, bypassed, or invalidated, plus the reason. This turns “the assistant is wrong” into an inspectable path.

type CacheDecision = {
  requestId: string;
  route: string;
  action: "served" | "bypassed" | "invalidated";
  reason: string;
  cacheKeyHash: string;
  entryAgeMs?: number;
  sourceVersions?: Record<string, string>;
};

Never log the full prompt or answer just to debug a cache. Log identifiers and versions, then join to access-controlled traces when needed.

Make eligibility a function, not a convention

Put cache safety in one testable function. The order matters: reject scope failures before checking similarity, bypass stale material, and invalidate known version drift.

type LiveContext = {
  scope: CacheScope;
  now: Date;
  sourceVersions: Record<string, string>;
  promptVersion: string;
  modelRouteVersion: string;
  policyVersion: string;
};

function canServe(entry: CachedAnswer, live: LiveContext) {
  if (entry.namespace !== makeNamespace(live.scope)) {
    return { action: "bypassed", reason: "scope_mismatch" };
  }
  if (live.now >= new Date(entry.expiresAt)) {
    return { action: "bypassed", reason: "ttl_expired" };
  }
  if (entry.promptVersion !== live.promptVersion) {
    return { action: "invalidated", reason: "prompt_version_changed" };
  }
  if (entry.policyVersion !== live.policyVersion) {
    return { action: "invalidated", reason: "policy_version_changed" };
  }
  if (entry.modelRouteVersion !== live.modelRouteVersion) {
    return { action: "invalidated", reason: "model_route_changed" };
  }
  for (const [sourceId, version] of Object.entries(entry.sourceVersions)) {
    if (live.sourceVersions[sourceId] !== version) {
      return { action: "invalidated", reason: `source_changed:${sourceId}` };
    }
  }
  return { action: "served", reason: "fresh" };
}

For a semantic cache, run canServe after finding candidates but before returning text. Better still, partition the vector index by hard scope so an out-of-scope candidate never enters the candidate set.

Connect change events to targeted purges

A full flush is safe but expensive, and it hides whether the invalidation design works. Prefer an outbox or change-feed event emitted by the same transaction that changes the source.

{
  "event": "knowledge.source.changed",
  "source_id": "policy/refunds",
  "old_version": "rev-41",
  "new_version": "rev-42",
  "tenant_id": "acme",
  "occurred_at": "2026-10-31T10:15:00Z"
}

The consumer should do three things: update the live version registry, mark matching response and retrieval entries stale by source metadata, and enqueue re-indexing if embeddings changed. Marking stale is often better than immediate deletion: it preserves a debuggable record while guaranteeing the lookup path cannot serve it.

Make the consumer idempotent. Events can arrive twice or out of order. If rev-42 is already live, a duplicate must not reopen rev-41. If a delete arrives, revoke the source immediately; do not wait for a rebuild job.

Test the failures that a happy-path cache hides

Unit tests for hit rate are not enough. Add regression cases that prove a known-bad entry cannot escape.

it("does not serve an answer after its cited policy changes", () => {
  const entry = answerFrom({ sourceVersions: { "policy/refunds": "rev-41" } });
  const live = contextFrom({ sourceVersions: { "policy/refunds": "rev-42" } });
  expect(canServe(entry, live)).toMatchObject({
    action: "invalidated",
    reason: "source_changed:policy/refunds",
  });
});

Build a small test matrix for every high-impact route:

• a document update and deletion;
• a role downgrade or tenant switch;
• a prompt or tool-policy release;
• an embedding or model-route version change;
• a delayed, duplicate, and out-of-order invalidation event;
• a semantic near-match that must be rejected.

Run these tests in CI and in a staging environment with the real cache adapter. The bug is often not the eligibility function; it is a secondary path that reads an old key, a worker that lags, or an index that was never namespaced.

Measure freshness, not just hits

A higher hit rate can mean lower cost or more wrong answers. Pair performance metrics with correctness signals:

Metric	Why it matters
Hit rate by layer and route	Shows where reuse actually happens
Entry age at serve time	Reveals TTLs that are too generous
Invalidations by reason	Reveals noisy releases or unhealthy sources
Event-to-invalidation lag	Measures the window of stale risk
False-hit rate	Finds semantic matches that should have missed
Purge coverage	Confirms all entries tied to a changed source were blocked

Sample served semantic hits for review or run a route-specific verifier. Do not use an LLM judge as the only safety control for permissions or facts that have an authoritative source version.

Respond to a stale-answer incident

When someone reports a confident but old answer, resist the urge to immediately tune the prompt. Follow a short path:

1. Locate the request trace and cache decision.
2. Identify the source, scope, prompt, policy, and route versions recorded with the entry.
3. Block the affected entry and its metadata neighbors from serving.
4. Compare the source-change time with the invalidation-event and consumer times.
5. Add a regression case for the missing trigger or lookup path.
6. Re-enable caching only after the replay proves the current answer wins.

This keeps the incident focused on evidence. A cached answer can be fluent and internally consistent while being unacceptable for the present request.

A practical rollout order

You do not need a perfect dependency graph on day one. Start where stale answers have the highest user cost.

1. Namespace all existing answer caches by tenant and privilege boundary.
2. Add prompt, policy, and source-version metadata to new entries.
3. Implement canServe and reason-coded decision logs.
4. Wire one high-risk source update to a targeted invalidation event.
5. Add the five failure cases to CI, then expand route by route.

Fast answers are valuable only when they are still eligible to be true. A freshness contract makes that eligibility explicit: every cached answer has a scope, dependencies, budget, revocation path, and proof. That is how you keep the latency win without asking users to discover your stale data for you.

FAQ

What is source-aware LLM cache invalidation?

It is a cache design that records which documents, policies, permissions, prompts, and model settings an answer depended on. When one changes, the system blocks or removes only the affected entries instead of waiting for a broad TTL or flushing everything.

Is a TTL enough for an LLM response cache?

TTL is a useful fallback, but it is not enough when a known change must take effect quickly. Use TTL for eventual cleanup and an event-driven purge for document updates, access revocations, policy releases, and other known changes.

Should semantic caches use the same invalidation rules as exact caches?

They need the same scope and freshness checks, plus stricter evaluation of similarity. A semantically close query can have different permissions or intent. Partition candidates by scope before similarity search and use conservative thresholds for higher-risk routes.

What should be included in an LLM cache key?

Include the route, tenant, relevant principal or role boundary, normalized request, and versions that determine the answer when those versions are part of lookup semantics. Keep rich provenance metadata too, so an event can find entries to invalidate without rebuilding every key.

How do I invalidate cached RAG answers after a document changes?

Emit a source-change event containing the source ID and new revision. Update the live revision registry, mark cached retrieval results and answers that cite the old revision stale, then re-index or regenerate on the next request. Test document deletion separately because it must revoke access even before replacement content is ready.

Which metrics show whether an LLM cache is safe?

Track served-entry age, event-to-invalidation lag, invalidations by reason, false semantic hits, purge coverage, and cache decisions by route. Hit rate and cost savings are useful, but they cannot detect a fast wrong answer.