DEV Community

Mahan Tavakoli
Mahan Tavakoli

Posted on AI-assisted

Cache Invalidation Never Died. AI Just Made It Expensive Again.

Why a programmer joke from the 1990s suddenly explains prompt caching, KV cache, LLM inference costs, latency, and even a few scary security problems

By Mahan Tavakoli (MahanKenway)
Tehran, Iran
GitHub: MahanKenway


The joke was never really a joke

There is an old line in computer science:

“There are only two hard things in Computer Science: cache invalidation and naming things.”

The quote is commonly attributed to Phil Karlton, a Netscape engineer. The attribution is unusually interesting because there does not appear to be a contemporaneous written source proving exactly when or where he first said it. Karlton's son, David Karlton, has said his father did use the phrase, while Martin Fowler records that Tim Bray had heard it around 1996–97 and later helped popularize it online.

And then somebody improved the joke.

“There are only two hard problems in computer science: cache invalidation, naming things, and off-by-one errors.”

That third item was later associated with Leon Bambrick, not Karlton. Martin Fowler's history of the quote explicitly separates the original two-item joke from the later off-by-one variation.

The reason the line survived for decades is not because programmers love recycled jokes.

It survived because caching creates a very specific kind of lie.

A cache stores an answer that was correct.

Then the world changes.

The cache does not.

And suddenly your system is confidently returning yesterday's truth.

That sounds like a web-development problem.

It isn't anymore.

Because in modern AI systems, we are caching:

  • prompt prefixes
  • tokenized context
  • model attention state
  • key/value tensors
  • long conversation histories
  • documents
  • retrieval results
  • embeddings
  • tool definitions
  • generated artifacts
  • sometimes entire computational subgraphs

In other words:

AI did not solve cache invalidation.

It made the consequences much larger.


A cache is basically a machine for reusing the past

At the most basic level, caching is simple.

Suppose calculating something expensive costs 100 milliseconds.

You calculate it once.

You save the result.

Next time, instead of spending 100 milliseconds calculating it again, you return the saved value.

That is the entire idea.

                    WITHOUT CACHE

Request
   |
   v
Compute expensive thing
   |
   v
Result


                     WITH CACHE

Request
   |
   v
Is result already cached?
   |                \
  YES                NO
   |                  |
   v                  v
Return result      Compute it
                      |
                      v
                  Store result
                      |
                      v
                  Return result
Enter fullscreen mode Exit fullscreen mode

The performance benefit can be enormous.

That is why caches exist everywhere.

CPU caches.

Disk caches.

Browser caches.

CDNs.

Database caches.

Redis.

Memoization.

Operating-system page caches.

Build caches.

Package-manager caches.

Compiler caches.

And now:

LLM caches.

The problem is hidden inside one innocent question:

When is the cached result no longer valid?

That question is where the comedy ends.


The real problem is not storing a value

Imagine a website shows this:

User: Mahan
Plan: Pro
Tokens: 14,281
Enter fullscreen mode Exit fullscreen mode

The application reads that data from a database.

To make the website faster, it caches the result.

Five minutes later:

Mahan upgrades the account.
Enter fullscreen mode Exit fullscreen mode

The database says:

Plan: Enterprise
Enter fullscreen mode Exit fullscreen mode

The cache says:

Plan: Pro
Enter fullscreen mode Exit fullscreen mode

Now you have two realities.

The database represents the current state.

The cache represents an older state.

Neither machine is malfunctioning.

The CPU is fine.

The network is fine.

The database is fine.

The cache is doing exactly what you instructed it to do.

And that is the terrifying part.

The bug is the architecture.

Redis documentation describes the classic consistency problem the same way: when the source of truth changes but the cached copy does not, the cache becomes inconsistent. Different caching strategies exist precisely because there is no single universal solution.

This is why cache invalidation became famous.

Caching asks:

“Can I reuse this old answer?”

Invalidation asks:

“Can I prove that this old answer is still safe to reuse?”

Those are very different questions.

The first is easy.

The second is where systems engineering starts getting weird.


Why invalidation is fundamentally annoying

Suppose this function exists:

def get_user():
    return database.query("SELECT * FROM users")
Enter fullscreen mode Exit fullscreen mode

You cache it:

@cache
def get_user():
    return database.query("SELECT * FROM users")
Enter fullscreen mode Exit fullscreen mode

Great.

Now another function updates the user:

def update_user(name):
    database.execute(...)
Enter fullscreen mode Exit fullscreen mode

The database changed.

But your cached get_user() result did not.

So the system needs some relationship like:

update_user()
      |
      +---- invalidates ----> get_user() cache
Enter fullscreen mode Exit fullscreen mode

But real programs do not contain one neat dependency.

They contain graphs.

Database
   |
   +--> User API
   |      |
   |      +--> Dashboard
   |
   +--> Search index
   |
   +--> Recommendation engine
   |
   +--> Analytics
   |
   +--> Mobile API
   |
   +--> Background jobs
Enter fullscreen mode Exit fullscreen mode

Now change one piece of data.

How many caches are potentially affected?

That is the actual problem.

Invalidation is dependency tracking disguised as housekeeping.

And the larger your system gets, the less obvious those dependencies become.


Enter the LLM

Now replace our user database with a language model.

Suppose an application repeatedly sends:

You are a senior software engineer.

Project:
[large repository]

Architecture:
[long documentation]

Tool definitions:
[dozens of tools]

Rules:
[system instructions]

Conversation history:
[thousands of tokens]

Now answer this question:
...
Enter fullscreen mode Exit fullscreen mode

Without caching, the system may need to repeatedly process the same large prefix.

With caching, the system can reuse work that has already been done.

This is exactly the kind of workload modern AI providers are optimizing for.

OpenAI introduced automatic prompt caching in October 2024, explaining that repeated input tokens could be reused to reduce latency and lower input-token cost. The initial announcement described a 50% discount for cached input tokens on supported models.

Anthropic similarly describes prompt caching as a mechanism for reusing frequently repeated context, with potential reductions of more than 2× in latency and up to 90% in costs for repetitive long-context workloads.

Google's Gemini API now also supports context caching, including automatic caching on newer Gemini models. Google's current documentation explicitly tells developers to put large, common content near the beginning of prompts and send similar prefixes close together in time to increase cache-hit probability.

Notice what just happened.

A computer-science joke that existed before modern LLMs became mainstream has quietly become part of the economics of AI inference.


Prompt caching is not magic

Let's simplify an LLM request.

You have:

SYSTEM
+ giant documentation
+ tool definitions
+ conversation history
+ user question
Enter fullscreen mode Exit fullscreen mode

The model processes the context.

The beginning of the request is often called the prefill stage.

Then the model generates tokens one by one during decode.

A simplified mental model:

INPUT TOKENS
     |
     v
  PREFILL
     |
     v
 KV CACHE
     |
     v
  DECODE
     |
     +--> token
     +--> token
     +--> token
     +--> token
Enter fullscreen mode Exit fullscreen mode

The KV cache stores intermediate attention information so that the model does not need to recompute the entire history from scratch for every generated token.

This is one of the foundational optimizations behind efficient transformer inference.

And now our old friend appears again.

We have data that is expensive to compute.

We store it.

We reuse it.

Now we need to decide when that stored data can safely be reused.

That is caching.

And therefore:

cache invalidation.


KV cache is where the joke gets much more interesting

The phrase KV cache sounds almost harmless.

It is not.

During autoregressive inference, the model generates output token by token while maintaining key/value tensors representing prior context.

These tensors consume GPU memory.

And they are dynamic.

And they grow.

And they have to be scheduled across concurrent requests.

And they compete for extremely valuable memory bandwidth.

The original PagedAttention paper behind vLLM described the KV cache as a major memory-management challenge. The authors specifically identified fragmentation and redundant duplication as major reasons existing systems wasted GPU memory, then introduced PagedAttention to manage KV cache blocks similarly to pages in virtual memory. Their evaluation reported 2–4× higher throughput than the compared serving systems at similar latency.

The conceptual connection is beautiful.

Operating systems solved memory management using:

logical memory
      |
      v
pages
      |
      v
physical memory
Enter fullscreen mode Exit fullscreen mode

LLM inference engines can use:

logical sequence
      |
      v
KV blocks
      |
      v
GPU memory
Enter fullscreen mode Exit fullscreen mode

This is not just a metaphor.

vLLM's documentation describes automatic prefix caching as caching KV-cache blocks and reusing them when a new request shares the same prefix. Its current implementation uses hashed blocks and a cache-management layer to find reusable prefixes.

That means modern AI serving has independently rediscovered a lot of old systems ideas.

Virtual memory.

Paging.

Reference counting.

Hashing.

LRU-style eviction.

Memory pooling.

Cache sharing.

Dependency tracking.

We did not escape old computer science.

We brought it with us.


The first trap: cache hits are not guaranteed

This is where AI marketing language can become misleading.

People hear:

“Prompt caching reduces cost by 90%.”

And mentally convert that into:

“My AI application will now cost 90% less.”

Not necessarily.

A cache only helps when the workload produces useful cache hits.

Consider:

Request 1:
[system][docs][tools][question A]

Request 2:
[system][docs][tools][question B]
Enter fullscreen mode Exit fullscreen mode

Excellent.

Large common prefix.

Potentially very cacheable.

Now:

Request 1:
[system][user context A][question A]

Request 2:
[system][user context B][question B]
Enter fullscreen mode Exit fullscreen mode

Much less reusable.

The prefixes diverge.

The economics change.

The system might still cache the beginning, but the useful reusable portion depends heavily on request structure.

Google's documentation makes this operational reality explicit: developers can improve implicit-cache hit rates by placing common content early and keeping similar prefixes close together in time.

So the performance problem is no longer merely:

“Do we have a cache?”

It becomes:

“Did we architect the workload so that the cache can actually help?”

That is a much harder question.


The second trap: invalidation is different for every layer

Here is where modern AI systems become beautifully messy.

Imagine an AI coding agent.

It has:

System prompt
+
developer rules
+
tool definitions
+
repository files
+
retrieved documentation
+
conversation history
+
current task
Enter fullscreen mode Exit fullscreen mode

Now change one thing.

A single file in the repository is modified.

What becomes invalid?

Maybe the repository context.

Maybe one retrieval result.

Maybe the tool output.

Maybe a cached prefix.

Maybe a generated summary.

Maybe an embedding.

Maybe a derived index.

Maybe nothing.

Or maybe parts of all of them.

That is the nightmare.

The cache does not understand your business semantics.

It understands keys.

So engineers have to create a mapping:

meaning
   |
   v
cache key
   |
   v
cached artifact
Enter fullscreen mode Exit fullscreen mode

If the key is wrong, the cache can remain perfectly efficient while being completely incorrect.


A cache key is a tiny piece of architecture

Suppose you cache:

cache["user:123"] = ...
Enter fullscreen mode Exit fullscreen mode

Straightforward.

Now imagine caching a model context:

model
system prompt
tool definitions
conversation
documents
permissions
tenant
locale
version
configuration
Enter fullscreen mode Exit fullscreen mode

Suddenly the key could effectively need to represent:

K =
(model version)
+ (system instructions)
+ (tool configuration)
+ (document version)
+ (conversation prefix)
+ (tenant)
+ (permission state)
+ (runtime configuration)
Enter fullscreen mode Exit fullscreen mode

Forget one important input.

Congratulations.

You just created a stale-cache bug.

And now the bug may not crash the service.

It may simply produce a wrong answer.

That is much worse.


This is where AI turns caching into a semantic problem

Traditional caching usually deals with relatively clear objects.

For example:

URL -> HTTP response
Enter fullscreen mode Exit fullscreen mode

Or:

database query -> rows
Enter fullscreen mode Exit fullscreen mode

LLM systems are different.

The result can depend on a huge amount of context.

For example:

Prompt
  +
Model version
  +
System instruction
  +
Tools
  +
Retrieved data
  +
Conversation state
  +
Sampling/configuration
Enter fullscreen mode Exit fullscreen mode

Now ask:

What exactly determines whether the previous result is still valid?

There is no universal answer.

You have to define one.

That definition is your cache's semantic contract.

And this is why cache invalidation is often less about deletion and more about understanding dependencies.


There are two different “caches” people casually call caching

This distinction matters a lot.

1. Prompt / context caching

The system reuses processing associated with repeated input context.

Conceptually:

same context
      |
      v
reuse previous work
Enter fullscreen mode Exit fullscreen mode

This is what public APIs increasingly expose as prompt caching or context caching.

OpenAI, Anthropic, and Google all now provide caching mechanisms intended to reduce repeated-input processing cost and/or latency.

2. KV caching during generation

This is the model-runtime side.

Instead of recomputing attention state for previous tokens every time, the serving system retains the relevant K/V tensors.

vLLM, for example, treats KV data as blocks that can be allocated, shared, hashed, evicted, and reused.

They are related.

But they are not identical.

And lumping them together makes discussions about “AI caching” confusing.


The weird part: caching can become a security problem

This is where the story gets properly strange.

We usually think:

cache = performance optimization
Enter fullscreen mode Exit fullscreen mode

But shared caches can create side channels.

Suppose two customers use the same inference infrastructure.

Customer A's private prompt causes certain prefix blocks to be cached.

Customer B sends a request with a guessed prefix.

If B's request gets a noticeably different latency depending on whether the prefix is already cached, the cache itself may leak information.

vLLM's security documentation now explicitly discusses this threat and references CVE-2025-46570, describing how differences in Time To First Token can reveal whether a guessed prompt prefix matches cached data. The documentation describes cache salting as a mitigation so different security principals do not blindly share the same prefix-cache namespace.

That is a remarkable evolution:

1990s:
"Cache invalidation is hard."

2020s:
"Cache invalidation affects cost."

2026:
"Cache design can become a confidentiality boundary."
Enter fullscreen mode Exit fullscreen mode

The cache went from:

optimization

to:

systems problem

to:

security surface

without changing its basic purpose.


The third trap: bigger caches are not automatically better

There is a temptation in infrastructure engineering to think:

More cache = more performance.

Not necessarily.

A cache consumes resources.

Memory.

Metadata.

Bandwidth.

Eviction logic.

Synchronization.

Invalidation traffic.

Management overhead.

And in LLM inference, the KV cache can become large enough that memory capacity and bandwidth directly influence performance.

Recent research continues to describe LLM inference as strongly constrained by memory movement. One 2026 AAAI paper, for example, focuses specifically on asynchronous KV-cache prefetching to reduce HBM-memory bottlenecks.

Another recent line of work studies the KV cache as an increasingly important bottleneck as context lengths expand, because the cache footprint grows with sequence length and puts pressure on both memory capacity and bandwidth.

This creates an interesting inversion.

Caching exists to save computation.

But caching itself costs memory and data movement.

So eventually you ask:

Am I saving compute by spending bandwidth?

And sometimes:

Did the cache just become the bottleneck I created while trying to remove the original bottleneck?

That is peak computer science.


Prefix caching is basically memoization for giant models

One of the most intuitive ways to understand modern LLM prefix caching is to think about a function:

f(prefix)
Enter fullscreen mode Exit fullscreen mode

If you already calculated:

f("A very long shared prefix...")
Enter fullscreen mode Exit fullscreen mode

why calculate the same thing again?

You can memoize it.

Modern inference engines can do something structurally similar with prefix blocks.

vLLM's automatic prefix caching documentation gives examples such as repeatedly querying the same long document or continuing a multi-turn conversation. In those workloads, the system can reuse the previously processed prefix instead of recomputing that section for every request.

It is a very old trick wearing very new clothes.

The scale changed.

The underlying idea did not.


And then we discover the off-by-one joke was accidentally relevant

Remember the expanded version?

cache invalidation
naming things
off-by-one errors
Enter fullscreen mode Exit fullscreen mode

The off-by-one part looks like a joke stapled onto another joke.

But in systems that divide sequences into cache blocks, boundaries matter.

For example:

Tokens:
[0 1 2 3][4 5 6 7][8 9 10 11]
Enter fullscreen mode Exit fullscreen mode

If your system's indexing, block accounting, token offsets, or ownership logic is wrong by one unit, you may get:

wrong block
wrong lookup
wrong reuse
wrong eviction
wrong attention state
Enter fullscreen mode Exit fullscreen mode

Modern cache managers therefore have to care about details that feel hilariously low-level compared with the headline:

“We built a frontier AI system.”

Underneath that headline you might still find:

hash table
LRU queue
block index
reference count
memory allocator
offset
eviction policy
Enter fullscreen mode Exit fullscreen mode

Humanity invented trillion-parameter models and then immediately had to debug a linked list.

I love this industry.


Why this matters for the cost of inference

This is the part I think gets missed most often.

People talk about AI inference economics as if the only important variable is:

FLOPs
Enter fullscreen mode Exit fullscreen mode

But real inference is constrained by a much richer system:

Model size
+
GPU memory
+
HBM bandwidth
+
interconnect bandwidth
+
batching
+
sequence length
+
KV cache memory
+
scheduler efficiency
+
cache hit rate
+
request shape
+
latency requirements
Enter fullscreen mode Exit fullscreen mode

Caching can reduce repeated work dramatically.

But it cannot magically eliminate the underlying data.

It can also fail to help if requests do not share prefixes.

And it can introduce memory pressure.

And it can introduce security boundaries.

And it creates lifecycle questions.

So saying:

“Cache invalidation is preventing AI inference from getting cheap”

would be too simplistic.

The more accurate statement is:

Caching is one of the mechanisms making inference cheaper, but cache efficiency itself becomes a first-class systems constraint as AI workloads become more repetitive, longer-context, and more multi-tenant.

That distinction matters.

The problem is not that caching is failing.

The problem is that the cost structure of AI increasingly depends on getting caching right.


The hidden economics of a cache miss

Imagine an AI application processing 1 million requests.

Every request has:

8,000 common input tokens
+
500 unique input tokens
Enter fullscreen mode Exit fullscreen mode

Now imagine the common 8,000-token prefix can be reused.

Without useful caching:

8,500 tokens
x
1,000,000 requests
Enter fullscreen mode Exit fullscreen mode

With a strong prefix cache:

8,000 tokens processed once
+
500 unique tokens
x
1,000,000
Enter fullscreen mode Exit fullscreen mode

The exact economics depend on the model, API pricing, cache policy, workload shape, and serving architecture.

But the conceptual difference is enormous.

You are no longer paying to repeatedly understand the same thing.

You pay once, then reuse.

That is why companies expose explicit or automatic caching mechanisms.

OpenAI advertises reduced input-token prices for cached prompts. Anthropic advertises savings of up to 90% on suitable workloads. Google similarly documents cost savings for cache hits and offers both implicit and explicit context-caching mechanisms.

So the old joke has quietly moved into the P&L.

A cache miss can now have a direct inference-cost attached to it.


But here's the paradox

Caching is supposed to make systems cheaper.

Yet modern AI systems sometimes need increasingly sophisticated infrastructure to make the cache useful.

You might need:

hashing
+
block management
+
memory pools
+
eviction policies
+
prefix matching
+
scheduler integration
+
tenant isolation
+
cache salting
+
metrics
+
observability
+
versioning
Enter fullscreen mode Exit fullscreen mode

So the system saves compute by adding complexity.

That trade is often worth it.

But it leads to a broader rule:

The more valuable your cache becomes, the more dangerous it becomes to get the cache wrong.

A cache nobody cares about can be dumb.

A cache that saves millions of dollars becomes infrastructure.

And infrastructure needs invariants.


The old distributed-systems nightmare comes back

Traditional distributed systems have always wrestled with consistency.

You have:

Source of truth
      |
      +--> Cache A
      |
      +--> Cache B
      |
      +--> Replica
      |
      +--> Search index
Enter fullscreen mode Exit fullscreen mode

Now AI applications often add:

Source
  |
  +--> Retrieval index
  |
  +--> Embedding store
  |
  +--> Prompt cache
  |
  +--> KV cache
  |
  +--> Tool state
  |
  +--> Agent memory
Enter fullscreen mode Exit fullscreen mode

A modern agent may effectively operate over several layers of remembered state.

That is an important conceptual shift.

The AI system is no longer just:

prompt -> model -> answer
Enter fullscreen mode Exit fullscreen mode

It starts looking more like:

             +-------------------+
             |   Source Data     |
             +---------+---------+
                       |
          +------------+------------+
          |            |            |
          v            v            v
      Retrieval     Context       Agent
       Cache        Cache         Memory
          |            |            |
          +------------+------------+
                       |
                       v
                    Model
                       |
                       v
                  KV Cache
                       |
                       v
                    Output
Enter fullscreen mode Exit fullscreen mode

Now ask:

What happens when the source data changes?

That question is no longer academic.


A useful way to think about invalidation

Instead of thinking:

“Delete the cache.”

Think:

“Which facts changed, and which derived artifacts depend on those facts?”

For example:

Document v17
    |
    +--> embedding v17
    |
    +--> retrieval index v17
    |
    +--> prompt context v17
    |
    +--> generated summary v17
Enter fullscreen mode Exit fullscreen mode

When the document becomes v18, you do not really have a “cache deletion” problem.

You have a dependency graph problem.

document v17
    X
    |
    +---- invalid
    |
    +--> embedding v17
    +--> retrieval result
    +--> summary
Enter fullscreen mode Exit fullscreen mode

Then:

document v18
    |
    +--> new embedding
    +--> new retrieval result
    +--> new summary
Enter fullscreen mode Exit fullscreen mode

That is why sophisticated systems increasingly use versioning, content hashes, immutable artifacts, timestamps, or explicit cache namespaces instead of trying to manually chase every stale object.


Sometimes the best invalidation strategy is not invalidation

This is one of the strangest and most useful lessons from systems engineering.

You can try to determine exactly when a cache entry is stale.

Or you can design the system so that staleness is naturally bounded.

A common strategy is TTL:

cache entry
    |
    +--> expires after 5 minutes
Enter fullscreen mode Exit fullscreen mode

Another is versioned keys:

document:123:v17
document:123:v18
Enter fullscreen mode Exit fullscreen mode

Another is content addressing:

hash(contents) -> artifact
Enter fullscreen mode Exit fullscreen mode

Another is immutable data.

Another is write-through or write-behind.

Another is simply refusing to cache data that is too difficult to invalidate safely.

Caching is not a religion.

Sometimes:

Do not cache this

is the best design decision.


vLLM shows what happens when old systems ideas meet LLM workloads

The engineering behind modern inference frameworks is a perfect example of this entire article.

vLLM's PagedAttention treats KV cache more like memory pages than like one giant contiguous array. Its later prefix-caching system extends that idea by hashing KV-cache blocks so identical prefixes can be recognized and reused.

That is a beautiful evolution:

Virtual memory
      ↓
PagedAttention
      ↓
KV block management
      ↓
Prefix caching
      ↓
Cross-request reuse
Enter fullscreen mode Exit fullscreen mode

The AI revolution keeps producing systems that look suspiciously like classic operating-system research.

Because at some point, a GPU is still a machine with:

finite memory
finite bandwidth
finite time
finite buses
finite queues
Enter fullscreen mode Exit fullscreen mode

No amount of AI hype changes that.


The uncomfortable implication

There is a tendency in AI engineering to think of optimization as a race:

better model
better GPU
more FLOPs
longer context
Enter fullscreen mode Exit fullscreen mode

But there is another race happening underneath:

better memory management
better scheduling
better batching
better cache locality
better reuse
better data movement
Enter fullscreen mode Exit fullscreen mode

And some of the biggest wins are not coming from making the model mathematically smarter.

They come from not doing the same work twice.

That sounds boring.

It is not boring.

At scale, “don't calculate the same thing twice” is an economic strategy.


A cache is basically a memory of what the system believed yesterday

This is the philosophical part.

A cache is not truth.

A cache is a copy of truth.

And copies create divergence.

The interesting thing about AI is that modern models already work with representations instead of the raw world.

Then we add:

retrieved data
+
summaries
+
embeddings
+
prompt caches
+
KV caches
+
agent memory
Enter fullscreen mode Exit fullscreen mode

Now we are building layers of representations on top of representations.

Every layer can become stale.

Every layer needs provenance.

Every layer needs invalidation rules.

The further away you get from the source of truth, the harder it becomes to know whether the thing in your hand is still trustworthy.

That is exactly why cache invalidation has stayed difficult for so long.


And maybe the joke has a modern version

The old version was:

cache invalidation and naming things.

The 2026 version might be:

cache invalidation, naming things, and convincing a 400-line AI agent that the thing it cached yesterday is no longer true.

The joke is funny because it is only slightly exaggerated.


What developers building AI systems should actually care about

Not every application needs an exotic cache architecture.

But if you are building a serious LLM product, it is worth explicitly documenting:

What is being cached?

Be precise.

tokens?
embeddings?
KV blocks?
retrieval results?
model responses?
tool results?
documents?
Enter fullscreen mode Exit fullscreen mode

What makes two requests equivalent?

For example:

same model
+
same prefix
+
same tool configuration
+
same tenant
Enter fullscreen mode Exit fullscreen mode

What invalidates the cached object?

content change
model change
tool change
permission change
tenant change
time
Enter fullscreen mode Exit fullscreen mode

What happens when the cache is wrong?

This is the question many systems forget.

Some stale data is annoying.

Some stale data is catastrophic.

Can one tenant observe another tenant's cache?

In multi-tenant inference, that is now a security question, not just a performance question. vLLM's documentation explicitly addresses cross-request prefix-cache side channels and cache salting for isolation.

What is the economics of a hit vs a miss?

Measure it.

A cache that sounds impressive but produces a 12% hit rate may not matter much.

A cache that consistently eliminates enormous repeated prefixes may be one of the most important pieces of infrastructure in the application.


The metrics I would actually watch

For an LLM caching system, “cache enabled: true” is nearly useless.

I would want to know:

Cache hit rate
Cache miss rate
Reusable token count
Cached token count
Bytes stored
Bytes evicted
Eviction rate
TTFT cold vs warm
Prompt processing time
Decode time
GPU memory usage
HBM traffic
Cost per request
Cost per cached request
Cost per uncached request
Enter fullscreen mode Exit fullscreen mode

And ideally:

hit rate by tenant
hit rate by workload
hit rate by model
hit rate by prefix length
hit rate by request type
Enter fullscreen mode Exit fullscreen mode

Because aggregate statistics can lie.

You might have:

90% average hit rate
Enter fullscreen mode Exit fullscreen mode

and still have:

0% hit rate
for your most expensive requests.
Enter fullscreen mode Exit fullscreen mode

Numbers need context.


Why this matters even more as context windows grow

Longer contexts sound like a pure model-capability improvement.

But long context also means:

more tokens
more memory
more memory movement
larger KV cache
more data to manage
Enter fullscreen mode Exit fullscreen mode

Recent research continues to focus on KV cache compression, offloading, heterogeneous memory, and prefetching precisely because the cache becomes increasingly significant as context lengths increase. For example, RocketKV studies substantial KV-cache compression for long-context inference, while other recent work examines dynamic placement and memory-bandwidth bottlenecks.

So there is a weird feedback loop:

Longer context
      ↓
More useful information
      ↓
More reusable information
      ↓
More reason to cache
      ↓
Larger cache
      ↓
More memory pressure
      ↓
More cache engineering
Enter fullscreen mode Exit fullscreen mode

AI did not eliminate the old problem.

It amplified it.


The biggest misunderstanding about “cheap inference”

People sometimes imagine that AI inference becomes cheaper mainly because GPUs become faster.

Yes, hardware improvements matter enormously.

But software can also make a huge difference by increasing reuse.

The fastest operation is often:

the operation you do not perform.

That principle is ancient.

Compiler optimization uses it.

CPU caches use it.

Databases use it.

Operating systems use it.

Web browsers use it.

CDNs use it.

LLM inference uses it.

And every time you choose reuse, you inherit the problem of proving that the reused thing is still valid.

That is the price of memory.


So... did AI finally solve cache invalidation?

No.

And that is exactly why this is interesting.

Modern AI infrastructure has become sophisticated enough to expose many of the same old problems at a scale where they directly affect:

  • inference latency
  • GPU utilization
  • memory capacity
  • bandwidth
  • API pricing
  • cloud bills
  • multi-tenant isolation
  • system correctness

Prompt caching has become a product feature.

KV caching has become a fundamental inference mechanism.

Prefix caching has become an optimization strategy.

Cache isolation has become a security concern.

And cache hit rate can become an economic variable.

The joke survived because the underlying problem survived.


The deeper lesson

The AI industry loves to describe itself using words like:

scaling

intelligence

reasoning

agents

context

autonomy

But underneath all of that, the computers are still asking extremely old questions:

Where is the data?

How much memory does it take?

Can I reuse it?

Is it still valid?

Who owns it?

When can I delete it?

Who else can observe it?

What happens if I am wrong?
Enter fullscreen mode Exit fullscreen mode

Those questions are not fashionable.

They are foundational.

And that might be the most interesting thing about modern AI engineering.

We keep inventing systems that look revolutionary from the outside.

Then we open the hood.

And there is still a cache.

And it is still lying to us.


TL;DR

Cache invalidation did not become less important because of AI. It became more important.

Modern LLM systems increasingly depend on several forms of caching, including prompt/context caching, KV caching, and prefix caching. OpenAI, Anthropic, and Google now expose caching mechanisms aimed at reducing repeated-input cost and latency, while inference systems such as vLLM use sophisticated KV-cache management and prefix reuse.

The challenge is that cached AI state can depend on huge amounts of context.

That turns cache invalidation into a problem of:

dependency tracking + consistency + memory management + economics + security.

And the really weird part?

The same old systems ideas that engineers were fighting decades ago are now sitting underneath trillion-token, GPU-heavy AI infrastructure.

So yes:

there are only two hard things in computer science.

Apparently we just decided to make the cache larger.


Frequently Asked Questions

What is cache invalidation?

Cache invalidation is the process of determining when cached data is no longer valid and must be removed, refreshed, or replaced. The difficulty comes from keeping cached copies consistent with the underlying source of truth.

Who said “cache invalidation and naming things”?

The statement is commonly attributed to Phil Karlton. The historical record is based largely on recollections and later documentation rather than a definitive contemporaneous publication. Tim Bray reported hearing the quote in the 1990s, and Martin Fowler documented the phrase and its later variations.

Did Phil Karlton invent the “off-by-one” version?

Not according to the commonly cited history. Martin Fowler attributes the later three-part variation to Leon Bambrick, while preserving Phil Karlton as the source of the original two-part version.

What is prompt caching?

Prompt caching allows an AI system to reuse processing associated with repeated input context, reducing repeated computation and potentially lowering latency and input-token costs. OpenAI, Anthropic, and Google all provide versions of this capability.

What is a KV cache?

A KV cache stores attention key/value information from previous tokens so an autoregressive model can avoid recomputing the entire prior context during generation. Managing this data efficiently is a major part of high-throughput LLM serving.

Does caching always make LLM inference cheaper?

No. Caching only produces major benefits when requests share reusable context or state. Cache effectiveness depends on workload structure, cache-hit rates, memory capacity, eviction behavior, and serving architecture. Google's documentation explicitly notes that prompt structure affects implicit-cache hit rates, while vLLM notes that prefix caching mainly reduces prefilling work rather than decode time.

Can LLM caches create security vulnerabilities?

Yes. Shared prefix caches can create timing side channels in multi-tenant inference, allowing an attacker to infer information about another request's cached prefix. vLLM's security documentation describes this problem and cache-salting mitigations.


Keywords / Topics

cache invalidation, Phil Karlton, cache invalidation problem, prompt caching, LLM caching, KV cache, KV cache optimization, prefix caching, LLM inference cost, AI inference optimization, vLLM, PagedAttention, prompt cache, context cache, cache consistency, distributed systems, GPU memory, HBM bandwidth, inference latency, AI infrastructure, LLM serving, cache security


Suggested DEV / CoderLegion SEO metadata

SEO Title:
Cache Invalidation Never Died. AI Just Made It Expensive Again.

Meta Description:
The famous cache-invalidation joke is suddenly relevant to AI. Explore prompt caching, KV cache, prefix caching, LLM inference costs, memory bottlenecks, and cache security.

Suggested Slug:
cache-invalidation-llm-prompt-kv-prefix-caching

Suggested Tags:
AI, LLM, Programming, Performance, System Design

Primary Search Intent:
Why is cache invalidation hard, and how does it affect modern AI inference?

Semantic topics for search / GEO:
cache invalidation explained, prompt caching explained, KV cache explained, prefix caching, LLM inference optimization, vLLM cache, AI inference cost, cache consistency, multi-tenant LLM security


Sources

  1. Martin Fowler — “Two Hard Things”
    Historical discussion of the Phil Karlton quote and its later variants.

  2. Tim Bray — “On XML Language Design” / archival references
    Early public attribution of the quote to Phil Karlton and evidence of its circulation in the 1990s.

  3. David Karlton / Skeptics Stack Exchange discussion
    First-hand family recollection regarding Phil Karlton's use and likely origin of the quote.

  4. OpenAI — “Prompt Caching in the API”
    Official description of automatic prompt caching and cached-input pricing.

  5. Anthropic — “Contextual Retrieval in AI Systems”
    Discussion of prompt caching and its potential latency and cost reductions.

  6. Anthropic Claude Cookbook — “Prompt caching through the Claude API”
    Technical description of automatic and explicit caching approaches.

  7. Google AI for Developers — “Context caching”
    Current Gemini documentation covering implicit and explicit caching, cache-hit behavior, and prompt-structure guidance.

  8. Kwon et al. — “Efficient Memory Management for Large Language Model Serving with PagedAttention”
    Research introducing PagedAttention and the vLLM serving system.

  9. vLLM documentation — “Automatic Prefix Caching”
    Technical explanation of KV-cache block reuse and hash-based prefix caching.

  10. vLLM security documentation — Prefix Cache Timing Side-Channel Mitigation
    Discussion of multi-tenant prefix-cache leakage and cache salting.

  11. Dong et al. — “Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching”
    Research on memory-bandwidth bottlenecks during LLM inference.

  12. Xu, Khaira & Singh — “KV Cache Optimization Strategies for Scalable and Efficient LLM Inference”
    Recent review of KV-cache capacity and bandwidth challenges in long-context inference.

  13. Behnam et al. — “RocketKV”
    Research on KV-cache compression for long-context LLM inference.

Top comments (0)