📺 Prefer to watch? 90-second YouTube Short · 💬 Telegram
Originally published on software-engineer-blog.com.
A customer fixes their delivery address at 20:31. Green tick, "Saved." At 20:32 a routine deploy restarts the service. At 08:14 the next morning the parcel ships to the old address. Nothing crashed. The log tail is clean: zero errors, zero alerts, zero exceptions.
Every explanation of caching teaches the read path — you ask for something, the cache has it, you skip the database, everything is fast. Almost nobody tells you what happens on a write. That is where this class of bug lives.
- Mental model: A cache is not a shortcut. It is a second copy. The only thing a write strategy decides is which copy becomes true first — and how long the other one is allowed to be wrong.
The code you already have
Most services never chose a write strategy. They chose a read strategy — cache-aside — and left the write path empty:
def get_order(order_id):
hit = cache.get(order_id) # the read path everyone writes
if hit is not None:
return hit
row = db.fetch_one(order_id)
cache.set(order_id, row)
return row
def update_address(order_id, addr):
db.execute("UPDATE orders SET addr = ? WHERE id = ?", (addr, order_id))
db.commit()
# ...and nothing at all happens to the cache
That is a write strategy — an accidental one. The database is right, the cache is stale, and the staleness lasts until the key expires. Which is exactly how a "saved" address gets served back as the old address.
Write-through: the database first, then the cache
Write-through makes both copies move together. Commit the durable write, then update the cache — in that order, always:
def update_address(order_id, addr):
db.execute("UPDATE orders SET addr = ? WHERE id = ?", (addr, order_id))
db.commit() # durable first
cache.set(order_id, addr) # then the second copy
return "saved"
The two copies can never disagree for longer than the microsecond between those lines. If the process dies after commit(), the database is already correct and the cache just misses. Nothing is lost.
The bill comes on the hot path. Measured on a real database, not estimated:
| operation | cost per write |
|---|---|
| durable write (commit + fsync) | 5.948 ms |
| the same value into an in-memory cache | 0.0012 ms |
| durable writes per second, one connection | 171 |
Write-through means every single change — forever — pays that ~6 ms, and your write throughput per connection is capped somewhere near 171/s. For a settings page nobody notices. For a counter updated on every request, that ceiling is your outage.
Write-back: write the cache, say yes, flush later
Write-back (also called write-behind) inverts it. The cache takes the write, the user is told yes immediately, and a background flusher pushes the changes to the database in batches:
dirty = {} # key -> newest value not yet in the DB
def update_address(order_id, addr):
cache.set(order_id, addr)
dirty[order_id] = addr # remember it needs flushing
return "saved" # the user is told YES, right here
def flusher(): # background, every 5 seconds
while True:
time.sleep(5)
batch, dirty_now = list(dirty.items()), dict(dirty)
dirty.clear()
with db.transaction():
for key, value in batch:
db.execute("UPDATE orders SET addr = ? WHERE id = ?", (value, key))
Acknowledgements now cost about a microsecond instead of about six milliseconds, and the database absorbs a burst as a trickle.
The trap
Write-back does not make the write faster. It makes the write a promise.
For the window between the acknowledgement and the flush, the only copy of that fact is in RAM, on one machine. Here is the loss, reproduced:
>>> save_write_back("o-4471", "Bahnhofstrasse 12")
'saved' # the user watched this succeed
>>> cache["o-4471"]
'Bahnhofstrasse 12'
# ...the process restarts before the flusher runs. RAM is gone.
>>> db.execute("SELECT addr FROM orders WHERE id='o-4471'").fetchone()
None # not stale. NOT THERE AT ALL.
Not a stale row. No row. The same sequence under write-through returns the address, because the durable write happened before the user was ever told yes.
That is the whole failure at the top of this article: an ordinary deploy, inside the durability window, and a confirmed change that never existed.
Why anyone accepts that
Because of coalescing. A hot key updated 900 times does not need 900 durable writes — the dirty map keeps only the newest value, so the flush writes it once:
| one-at-a-time | batched behind a dirty map | |
|---|---|---|
| database writes for 900 updates to one key | 900 | 1 |
| wall clock | 5276.8 ms | 13.3 ms (396× faster) |
For a cart quantity, a session, a counter, a last_seen timestamp, that is an enormous win and the risk is genuinely acceptable — nobody is harmed if five seconds of "last seen" evaporate. For a confirmed delivery address or a payment record, the same trade is indefensible.
The durability window is a number you choose
Write-through's answer is "zero milliseconds." Write-back's answer is "however long until the next flush" — and if you cannot say that number out loud, you have not chosen it, you have inherited it.
Two guards make write-back safe to actually run:
MAX_DIRTY = 10_000
def update_address(order_id, addr):
if len(dirty) >= MAX_DIRTY: # 1. bound the queue
flush_now() # backpressure, never unbounded RAM
cache.set(order_id, addr)
dirty[order_id] = addr
return "saved"
def flush_lag_seconds(): # 2. measure the real window, live
return time.time() - last_successful_flush_at
The bound stops a flusher that has fallen behind from turning into an out-of-memory kill that takes every unflushed write with it. The metric is the one number to put on a dashboard and alert on: it is your data-loss exposure, in seconds, right now.
Write-around: the third strategy
Write-around writes the database and skips the cache entirely:
def import_row(order_id, addr):
db.execute("UPDATE orders SET addr = ? WHERE id = ?", (addr, order_id))
db.commit()
cache.delete(order_id) # invalidate, do not populate
It exists for writes that nobody is going to read soon — a nightly import, a bulk backfill. Populating the cache with a million rows that no user will request just evicts the keys people are reading, and your hit rate falls off a cliff right after the import finishes.
The same decision in LLM serving
This is not a legacy-backend concern. Every inference stack in production runs the same three strategies under different names:
- A semantic cache in front of a model is cache-aside on the read. If you write a corrected answer or a moderation verdict into it and defer the durable write, that is write-back — and a pod restart loses the corrections you believe you shipped.
- KV-cache and session state for a multi-turn conversation are the textbook write-back case: enormous update rates on one key, coalescing wins, and nobody is harmed by losing the last few seconds of an ephemeral attention state.
- Usage and token accounting, on the other hand, is money. Batch it for throughput if you must, but the flush lag on that queue is the number of billable tokens you are willing to lose. That belongs on a dashboard, not in a comment.
The rule transfers exactly: batch what is cheap to lose, commit what is expensive to lose, and know your window.
| Strategy | What happens on a write | Data-loss window | When to use |
|---|---|---|---|
| Write-through | Durable commit first (~6 ms), then the cache | Zero — the answer is true when you say it | Anything a user is told succeeded: addresses, orders, payments, permissions |
| Write-back | Cache + dirty map, acknowledge in ~1 µs, flush in batches | Up to the flush interval — seconds of confirmed data | High-rate, low-value state: counters, sessions, carts, last-seen |
| Write-around | Durable commit, cache invalidated and not populated | Zero (durability); costs a read miss | Bulk imports and backfills nobody will read back soon |
| Nothing (accidental) | Database updated, cache untouched | Zero loss, but unbounded staleness until TTL | Never on purpose — this is the bug, not a strategy |
The whole thing in one breath
A cache is a second copy, so every write has to decide which copy becomes true first. Write-through pays a durable ~6 ms on every change and can never lose a confirmed write. Write-back acknowledges in microseconds and coalesces 900 updates into one database write, but for the length of its flush interval the only copy of a confirmed fact lives in RAM on one machine — and an ordinary restart in that window deletes a change your user watched succeed. Write-around keeps bulk writes from evicting your hot keys. Bound the dirty queue, export the flush lag, and you have turned an invisible risk into a number.
Verdict
Default to write-through. It is boring, it costs about six milliseconds, and "saved" means saved.
Reach for write-back deliberately, for a specific hot key whose last few seconds are genuinely disposable — and only with a bounded dirty map and a flush-lag metric on a dashboard.
The question is never "is my cache fast." It is: how many milliseconds of confirmed data am I willing to lose?
Watch the full 11-minute walkthrough for all of the code and the loss reproduced live, or the 2-minute version for the short of it.
Top comments (0)