Tailscale, the company behind the popular mesh VPN built on WireGuard, just published a forensic deep dive into a database corruption bug that haunted them for months. The culprit? A 16-year-old bug in SQLite's WAL (Write-Ahead Log) reset logic — one of the most widely used database engines in the world.
The Problem: Months of Unexplained Outages
At the end of last year, Tailscale's uptime started getting shaky. Their status page told the story: a series of outages that continued into the new year. Many of these were caused by a single bug, buried deep inside SQLite.
For a company whose entire value proposition is reliability and seamless connectivity, this was a serious problem. Customers expect Tailscale to just work. For several months, it didn't always deliver on that promise.
Tailscale's Architecture: Why SQLite Matters
Here's the interesting part: Tailscale's control plane isn't built on PostgreSQL or MySQL like you might expect from a major infrastructure company. It's built on SQLite.
Their control plane is split into a series of coordination servers (or "shards"). Each tailnet lives on one internal shard at a time but can migrate seamlessly between them. Each shard has an SQLite database holding all the information about the tailnets on that shard.
This architecture choice is deliberate. SQLite is embedded, fast, and requires no separate server process. But it also means that a bug in SQLite's internals is a bug in your critical path.
The 16-Year-Old Bug
The bug was in SQLite's WAL reset logic. The Write-Ahead Log is a core SQLite feature that allows concurrent reads while writes are happening — crucial for performance. When the WAL is reset (typically when it reaches a certain size or during a checkpoint), a subtle error could occur that corrupted the database.
This bug had been present in SQLite for 16 years. It survived countless releases, code reviews, and tests. SQLite is one of the most tested pieces of software on the planet — it's used in practically every mobile device, web browser, and embedded system. And yet, this bug lurked in the WAL reset code for over a decade and a half.
The Forensic Investigation
What makes this story remarkable isn't just the bug — it's the investigation. Tailscale's team spent months tracking it down. They had to:
- Correlate outages across shards — not all shards were affected equally, which made pattern recognition difficult
- Reproduce the corruption — the bug was intermittent and depended on specific timing conditions
- Trace through SQLite's internals — the WAL reset code path is complex and involves interaction between the WAL file, the database file, and the journal
- Work with the SQLite team — once they identified the root cause, they collaborated with SQLite's maintainers to fix it
Why This Matters for Every Developer
This story has implications far beyond Tailscale:
1. Even the Most Battle-Tested Software Has Bugs
SQLite is used by billions of devices. It's been audited, fuzzed, and tested more than almost any other codebase. If a 16-year-old bug can hide in SQLite, bugs can hide anywhere. Humility about our tools is essential.
2. Architecture Choices Have Long-Term Consequences
Tailscale's choice of SQLite was unconventional for a large-scale infrastructure service, but it gave them simplicity and performance. The tradeoff: they were now dependent on SQLite's reliability for their core service.
3. Forensics Are Underrated
In an era of AI-assisted debugging and automated testing, this story is a reminder that some bugs still require patient, methodical forensics. There's no AI prompt that would have found this bug. It required human investigators who understood the system deeply.
4. Open Source Works
Tailscale found the bug, reported it to the SQLite project, and the fix is now available to everyone. Every SQLite user benefits. This is the open source flywheel in action.
Lessons for AI Agent Builders
As someone building AI agents on a Raspberry Pi, this story resonates. When your agent's database corrupts, you need to be able to trace through the stack — from your application code down to the storage layer. Understanding what's happening beneath your abstractions is what separates reliable systems from fragile ones.
The next time your database does something unexpected, remember: it might not be your code. It might be a bug that's been hiding for 16 years.
Based on Tailscale's blog post by Alex Chan, published August 12, 2026.
Top comments (0)