DEV Community

Cover image for Design Dropbox Step by Step: 4 MB Blocks, Content Hashes and the Conflicted Copy
Gabriel Anhaia
Gabriel Anhaia

Posted on Edited on AI-assisted

Design Dropbox Step by Step: 4 MB Blocks, Content Hashes and the Conflicted Copy


You open a shared folder on Monday and find two files where there used to be one: budget.xlsx and budget (<username>'s conflicted copy 2026-10-24).xlsx. Nobody merged anything. Dropbox looked at two edits it could not order, kept both, and put a name and a date on the loser. Its help page says the most recently saved version is always the one labelled as the conflicted copy, and tells you to compare the two and merge them by hand.

That file is the last step of the "design Dropbox" interview question, and many candidates never reach it. They spend forty minutes on "we store files in S3" and run out of time. You do not have to guess how a real file-sync system works: Dropbox wrote it down on its engineering blog in a series of posts from 2014 to 2020. Follow those posts and name the trade-off at each step out loud, before the interviewer asks.

Step 1: split every file into 4 MB blocks and name them by hash

The founding decision is in Nipunn Koorapati's 2014 post Streaming File Synchronization: "Every file in Dropbox is partitioned into 4MB blocks, with the final block potentially being smaller." Each block is hashed with SHA-256, and the ordered list of those hashes is what Dropbox calls the blocklist. The post says a file's contents can be uniquely identified by that list.

Here is the client-side half of that in a few lines of Python:

import hashlib

BLOCK = 4 * 1024 * 1024  # 4 MB, as in Dropbox's 2014 post

def blocklist(path: str) -> list[str]:
    hashes = []
    with open(path, "rb") as f:
        while chunk := f.read(BLOCK):
            hashes.append(hashlib.sha256(chunk).hexdigest())
    return hashes
Enter fullscreen mode Exit fullscreen mode

A file is now a list of names. A block is now a value addressed by its content. Everything that follows comes from those two facts.

The trade-off to name: fixed-size blocks are cheap to compute and easy to reason about, but they are fragile under insertion. Add one byte at the start of a 40 MB file and every block boundary shifts, so every hash changes and the client re-uploads the whole thing. Content-defined chunking (boundaries cut where a rolling hash hits a pattern, as in restic or Borg) survives insertions at the cost of variable block sizes and more CPU. Say which one you picked and why. For large binary files that are mostly appended or rewritten wholesale, fixed blocks are a defensible choice.

Step 2: the metadata server stores lists, not bytes

The same post splits the backend into two services. The blockserver "maintains a key-value store of hash to encrypted contents." The metaserver "maintains database of users, namespaces, and of course SFJ," the server file journal, where each row records a file version at a (namespace, path) along with its blocklist.

Uploading is a negotiation. The client calls commit with the new blocklist. The metaserver checks whether it already knows those hashes and whether the user has access to them. If anything is missing, the commit fails and returns "need blocks," naming exactly which ones. The client uploads only those to the blockserver with store_batch (the post's example caps a request at 8 MB), then retries the commit, which writes the new SFJ row.

The decision on the server boils down to this:

def need_blocks(blocklist: list[str],
                known: set[str]) -> list[str]:
    return [h for h in blocklist if h not in known]
Enter fullscreen mode Exit fullscreen mode

If that list is empty, the upload costs one metadata write and zero bytes of file data. Overwrite a few bytes in place inside a 400 MB file and, with fixed blocks, you send the one or two 4 MB blocks that changed.

The trade-off to name: content addressing gives you dedup and resumable uploads for free, but it puts a security question on the table. If the server says "I already have that block" to anyone who presents a hash, the hash becomes a credential and an oracle for whether a file exists. Dropbox's post makes the metaserver check access as well as existence. In an interview, bring this up yourself.

Step 3: tell idle clients something changed, then let them pull

Another device needs to find out about the new SFJ row. Per the 2014 post, "Idle clients maintain a longpoll connection to a notification server." The notification carries no file data. It wakes the client, which calls list with a per-namespace cursor (a journal ID) and gets back the rows newer than that cursor. It then fetches whatever blocks it does not already have with retrieve_batch.

That split, a thin "something changed" signal plus a cursor-based pull, is the part worth drawing on the whiteboard. The notification path can drop messages and nothing breaks, because the cursor is the source of truth: a client that reconnects asks "what is after journal ID N" and catches up.

The 2014 post is about making this faster. Streaming sync lets a receiving client start fetching blocks before the uploader's commit lands, instead of waiting for it. Dropbox reported "an up-to-2x improvement on multi-client sync time", a theoretical ceiling the post ties to large files and equal upload and download bandwidth, and roughly a 25% improvement on its own test setup.

The trade-off to name: longpoll is simple and works through proxies, but every idle client holds an open connection, so you size the notification tier by connected devices. Write traffic barely matters. WebSockets or server-sent events change the wire protocol and leave that sizing problem where it was. Push a tiny signal and let clients pull, and the fan-out stays cheap.

Step 4: store the blocks somewhere immutable

Dropbox was an early adopter of Amazon S3. In March 2016, Akhil Gupta wrote that the company was "storing and serving over 90% of our users' data on our custom-built infrastructure", over 500 petabytes at the time. That system is Magic Pocket.

James Cowling's Inside the Magic Pocket describes it as an immutable block storage system: blocks are "encrypted chunks of files up to 4 megabytes in size," keyed for most use cases by the SHA-256 hash of the block, and once a block is written "it never changes." The 4 MB unit is too small to move around efficiently, so blocks are grouped into 1 GB buckets. New blocks are replicated directly onto multiple machines first, then buckets are erasure coded later for storage efficiency. The index that maps a hash to its cell and bucket is a sharded MySQL cluster.

The trade-off to name: immutability is what makes this tractable. A block that never changes can be cached anywhere, replicated without coordination and verified against its own name. The price is that deletes become garbage collection: a block can be dropped only once no blocklist references it, and you have to design that reference counting or sweep. Replicate-then-erasure-code trades a short window of expensive storage for fast, durable writes.

Step 5: why two offline edits become a conflicted copy

To know whether a change came from this device or from the server, the client needs a common ancestor. Dropbox's 2020 rewrite of the desktop sync engine, Nucleus, written in Rust, keeps three trees. Isaac Goldberg's Testing sync at Dropbox defines them: the Remote Tree is the latest state in the cloud, the Local Tree is the last observed state on disk, and the Synced Tree is the last known fully synced state between them. The post compares the Synced Tree to a version-control merge base: "A merge base allows us to derive the direction of a change."

Compare each path against that base. Local equals base and remote differs: the server changed it, so download. Remote equals base and local differs: you changed it, so upload. Both differ from the base: two people edited the same file since they last agreed, and there is no direction to derive.

At that point a merge would need to understand the file. Dropbox syncs spreadsheets, Photoshop files and SQLite databases, and a byte-level merge of two of those produces a corrupt file, not a merged one. The same testing post states the governing rule plainly: data loss "must be avoided at all costs." Picking a winner silently loses one person's work. Keeping both, and naming the newer save as the conflicted copy per the help page, loses nothing and pushes the merge to the only party that understands the format: the user.

The trade-off to name: a conflicted copy is ugly, and it confuses people. The alternatives are worse for a general-purpose file store. Last-writer-wins is silent data loss. Locking breaks offline editing. Real merging (operational transforms, CRDTs) needs a document model, which is why collaborative editors build one and a file-sync product does not. A strong answer says: "for opaque files, preserve both and surface the conflict; for formats we own, we could merge."

What the whole answer looks like

If you only get to draw one box first, draw the metadata service. It owns the journal of blocklists and is the single point of consistency, so every other piece (the block store, the notification tier, conflict detection) is defined by what it reads from or writes to that journal. Capacity estimates and the exact API shape are the parts you add live, once the interviewer pushes on scale.

Anyone can say "chunk the file." The senior answer says what happens when someone inserts one byte at the start of it.


If this was useful

This post is one design walked end to end, the way I'd want to hear it in an interview: decision, real-world source, then the trade-off you'd name before being asked. The System Design Pocket Guide: Interviews does that for 15 real systems, step by step, so you have the shape of each answer before the clock starts.

System Design Pocket Guide: Interviews — 15 Real System Designs, Step by Step

Top comments (0)