Your Filesystem Is Lying to You: Why fsync() Doesn't Guarantee Durability

You call write(). You call fsync(). The operating system returns 0 (success). Your data is safely written to the physical storage platters or flash cells forever, right?

Not even close.

For decades, developers and systems engineers have treated fsync() as an immutable promise: when it returns, the data has survived the boundary between volatile RAM and persistent non-volatile media. But modern storage stacks, virtualized block layers, and operating system kernels reveal a much darker reality.

Here is what is actually happening beneath the hood when you flush data to disk, why fsync() frequently fails to protect your data, and what robust storage engines do instead.

---

The Illusion of the Storage Stack

When a high-level application writes data, it does not speak to hardware. It traverses at least seven abstraction layers:

[ Application Layer ]        -> write(fd, buf, len)
        ↓
[ Virtual File System (VFS) ] -> POSIX syscall translation
        ↓
[ Page Cache / Buffer Cache ] -> Dirty memory pages in OS RAM
        ↓
[ Filesystem Engine (ext4/XFS)]-> Inodes, Extents, Journals
        ↓
[ Block Layer & I/O Scheduler]-> Merging, sorting request queues
        ↓
[ Storage Device Driver ]     -> NVMe / SATA / SCSI commands
        ↓
[ Hardware Controller Cache ] -> Volatile DRAM cache on the SSD/HDD
        ↓
[ Physical Flash / Platters ] -> Persistent storage media

When you issue fsync(fd), you are instructing the OS kernel to flush dirty pages belonging to that specific file descriptor down to the drive controller. But several critical failure modes make this guarantee surprisingly fragile.

---

1. The PostgreSQL "Fsyncgate" Disaster

In 2018, the PostgreSQL development team uncovered a fundamental flaw in how the Linux kernel handled I/O errors during fsync()—a bug that had lurked in systems for over two decades.

What actually happened:

1. PostgreSQL wrote dirty pages to the Linux page cache in background workers.
2. The kernel attempted writeback in the background. If a physical hardware error occurred (e.g., bad sector or transport error), the page was marked clean and the EIO error flag was attached to the address space.
3. Later, when PostgreSQL executed fsync(), the kernel saw the error flag, returned EIO (Error), and cleared the error flag.
4. When PostgreSQL retried fsync() on the next check, the kernel saw no dirty pages and no error flag, returning 0 (Success)!

PostgreSQL assumed the retry succeeded, while in reality the data was completely discarded from RAM without ever reaching disk.

---

2. Directory Metadata vs. File Contents

Calling fsync(file_fd) flushes the file data, but it does not flush directory entry metadata.

If you create a file, write data to it, fsync(file_fd), and close it:

int fd = open("user_data.db", O_WRONLY | O_CREAT | O_TRUNC, 0644);
write(fd, buffer, size);
fsync(fd);
close(fd);

If power cuts at this exact moment, on reboot the directory might not contain the entry for user_data.db at all, even though the disk blocks were written!

To guarantee creation or rename durability on Linux/UNIX, you must explicitly fsync() the parent directory file descriptor:

int dir_fd = open(".", O_RDONLY | O_DIRECTORY);
fsync(dir_fd);
close(dir_fd);

---

3. The Disk Controller's Lie: Write Caching

Consumer and cloud NVMe SSDs frequently employ volatile DRAM write caches to report lightning-fast benchmark numbers.

When the OS sends a flush request (SYNCHRONIZE CACHE in SCSI or FLUSH in NVMe):

• Enterprise Drives (with PLP / Power Loss Protection): Have capacitors that flush onboard DRAM to NAND flash even during total power failure.
• Consumer SSDs & Budget Cloud VMs: May acknowledge the flush command as soon as data reaches internal volatile DRAM, without committing it to NAND. A sudden power interruption destroys the onboard DRAM before flash commit.

---

Summary: The Senior Developer's Durability Checklist

If you are building a database, ledger, key-value store, or critical file-handling service:

1. Never retry a failed fsync(): Treat an fsync() error as a fatal condition requiring an immediate process panic or WAL rewind.
2. Always fsync() the parent directory after file creation, unlink, or atomic rename.
3. Use Write-Ahead Logging (WAL) with strict sequential appended blocks.
4. Audit storage hardware for Power Loss Protection (PLP) before relying on hardware cache flushes.
5. Consider O_DIRECT or O_SYNC flags when you need deterministic control over the caching path.

fsync() is not magic. It is an instruction to an imperfect multi-layer hierarchy. Understanding where it fails is the difference between data integrity and catastrophic corruption.