D1 read replicas returned stale data for 6 seconds — here's what I measured and how I designed around it

At p99, a D1 read replica in Tokyo was 6.1 seconds behind a write that landed on the Eastern NA primary. I found this out not from the docs, but from a Korean ad-ops campaign tracker silently throttling the wrong impressions.

D1's architecture is straightforward: one SQLite primary, read replicas pushed to edge POPs, writes always hit the primary, reads go to the nearest replica. The docs acknowledge eventual consistency but give you no number to plan around. So I built a staleness probe — write a row with a UUID and epoch, poll the replica until it appears, record the delta. Across 200 probes from ap-northeast-1:

• p50: ~800ms
• p95: ~3,400ms
• p99: ~6,100ms

That's the shape of the problem if your primary lands in enam and your users are in Asia.

The failure that forced my hand wasn't slow throttling — it was this:

Error: D1_ERROR: no such table: sessions

The table existed. What happened: a migration ran on the primary, the Worker restarted, and the first few requests hit a replica that hadn't caught the schema change yet. The error was misleading. The root cause was assuming replica reads would reflect a write that had just committed.

The fix I landed on doesn't fight the replication lag — it routes around it for the cases that matter. The writer stamps a written_at epoch on the row and sets an X-D1-Written-At response header. The reader compares that epoch against the written_at column it gets back from the replica. If the replica's value is older than the write signal, it falls back to KV — which runs sub-500ms in the same region and is free up to 10M reads/day. For a binary throttle flag that changes maybe a few times per minute, KV is a cheap freshness backstop.

The important part is that you only pay the KV fallback cost on the narrow window where the replica is genuinely behind. Most reads — once the replica catches up — hit D1 normally.

I wrote up the full breakdown — including the complete staleness probe script, the KV write-pair pattern, and how to handle the schema migration race — over on dailymanuallab.com.

Full post →