A version-control service I worked on kept a small list in Redis: which version of the code bundle was sitting on the shared NFS volume. Every request checked the list first. Hit meant the file was local and the executor could load it. Miss meant a download from object storage.
One morning the cleanup job ran. It deletes old versions when NFS fills up. It deleted v1.42 from NFS. The list still said v1.42 existed. The cache lied, and a lying cache is worse than a missing one. A miss sends you to the source of truth. A false hit tells you the file is there, the loader opens it, and you get an error three steps later with no clear culprit.
Picking Redis or Caffeine or Memcached was not the question. The question was how to keep that list aligned with NFS. Every cache, on every system, faces the same question. Three patterns answer it, and each one hands you a different bill.
Three patterns, three bills
TTL: the default that lies until it stops
Most teams pick TTL because it costs nothing to set up. Each key gets an expiration time. Read the key. If the time has passed, treat it as a miss and reload from the source. Done.
TTL cache flow from write to read to expiry
The good part is the simplicity. The bad part is the gap between expiration and the actual re-read. Anything that writes to the source before that re-read lands will be missed. Every miss costs a trip to the source, which is fine until the source is slow, and then the cache that was supposed to take load starts generating it.
Tunables: expiration time, refresh-ahead. Neither removes the window. Shorter expiration shrinks it. Longer expiration makes the cache useful again. Pick one bill and own it.
Event-driven: the cache stays honest, the event bus stays busy
The second pattern answers the lying problem directly. Instead of letting entries time out, you tell the cache when the source changed. Every write to the source emits an event. A consumer in the cache layer picks it up and invalidates or updates the key.
Event-driven invalidation races against writes
Two failure modes show up in practice.
First, missed events. The producer crashed, the network dropped the message, the consumer was down. The cache says the old value is still right. The window is smaller than TTL because it depends on the outage, but the window is never zero.
Second, out-of-order events. Two writes happened in quick succession. The first invalidation lands, then the second one lands before the first write has been read back into the cache. A reader who arrives between them sees nothing, reloads the first write, then the second invalidation hits and evicts it. Net effect: a clean cache, one wasted reload. If the order reverses, the cache holds the older value while the source has the newer one. That is the same lying cache, just for shorter.
Messages can be lost
The bill is the event bus. A queue with delivery guarantees, idempotent consumers, retries with backoff, dead-letter handling, ordering where it matters. Each of these is real engineering, and the system that owns the source has to wire it in. Event-driven caches are not free. They are cheap to write the first time and expensive to operate.
Write-through: the cache never holds a lie
The third pattern removes the lying window by removing the path that creates it. The cache sits on the write path. Every write goes to the cache and the source. Reads go to the cache. There is no separate invalidation step because there is no separate write.
Write-through keeps cache and source in step
The tradeoff moves up the stack. Writes get slower because they pay two round-trips, and the cache becomes part of the critical path. If the cache layer fails, writes fail. The source has to be the system of record that recovers state on restart.
Pick one, or pay three bills
This works well when reads dominate writes by a wide margin and the cache layer can match the source's durability story. It works badly when the source and the cache disagree on failure modes, because now you have two systems that both believe they own the truth.
Pick one and own its failure mode
The three patterns answer the same question with three different bills.
- TTL gives you a cache that lies until the timer expires. You own the gap.
- Event-driven gives you a cache that stays honest when the events flow. You own the event bus.
- Write-through gives you a cache that never holds a lie. You own the write path.
There is no pattern that has none of these bills. The list of caches in your system is the list of failure modes you have agreed to operate.
When the lying cache shows up in an incident report, look at which pattern the cache is using. The failure mode is not a surprise. It is the bill.
Top comments (0)