DEV Community: RONI DAS

How Reddit Stores Comment Trees and Ranks Hot Posts

RONI DAS — Fri, 10 Jul 2026 09:36:53 +0000

Reddit looks simple and hides two genuinely hard problems. Comments nest arbitrarily deep, and a naive tree structure makes loading a busy thread slow. The front page reorders itself constantly, so ranking cannot just count votes or old posts would never leave. Both problems have well-known answers, and both are good lessons in choosing the right model.

The core problem

A comment thread is a tree. Each comment can reply to any other, so depth is unbounded. If you store only "this comment's parent id" and then try to load a whole thread, you walk the tree one level at a time, one query per level, which gets slow for deep or wide threads. Loading a popular post with thousands of nested comments should not take thousands of queries.

Ranking is the second problem. If the front page sorted by raw vote count, the highest-voted post of all time would sit at the top forever. If it sorted by newest, quality would drown in noise. You need a score that blends how good a post is with how fresh it is, so good new posts can climb and old ones fade even if they were once popular.

Key design decisions

Store the parent pointer, but do not traverse at read time. The simple model is a parent_id per comment, which is easy to write but expensive to read as a tree. To load a thread cheaply, fetch all comments for the post in one query, then assemble the tree in application memory. One read, in-memory tree building. This works because a single post's comments, while numerous, fit in memory to assemble.

Consider a path or closure model for deep trees. For very deep threads, some systems store a materialized path on each comment, an encoded ancestor chain, so you can fetch an entire subtree with a single prefix query and sort by the path to get correct display order. Another option is a closure table that records every ancestor-descendant pair, which makes subtree queries direct at the cost of extra write work. The right choice depends on how deep threads get and how often you read subtrees versus write comments.

Cache the assembled thread. A hot thread is read far more than it is written, so caching the assembled comment tree and updating it as new comments arrive avoids rebuilding it on every view.

Rank with a time-decaying score. The hot ranking blends a post's vote balance with its age. The core idea is that the logarithm of the score contributes a lot for the first votes and less for each additional vote, while a steady time term pushes older posts down. A post submitted later starts with a built-in advantage, so a strong new post can overtake an older one without needing an impossible number of votes. Because the score depends on submission time and votes, it can be computed and stored per post and only recomputed when votes change.

The trade-offs

The parent-pointer plus in-memory assembly model is simple and cheap to write, and it is fine for most threads, but pathological threads that are extremely deep or wide stress the single fetch. Materialized paths make subtree reads fast but add cost and complexity on every insert and move, and moving a comment means rewriting paths. Closure tables make ancestor queries trivial but multiply write volume, since one comment at depth n writes n ancestry rows. You are trading read speed against write cost, and the correct point depends on your read-to-write ratio.

Hot ranking trades precision for stability and cheapness. A decay-based score can be precomputed and rarely needs recalculation, which keeps the front page cheap to serve, but it is a heuristic, not a truth, and it can be gamed by early vote bursts. Tightening it against manipulation means adding signals beyond votes and time, which complicates the clean formula.

How the real system does it

The real design stores comments with parent references, loads a post's comments in bulk and builds the tree in memory or uses a path model for deep subtrees, and caches hot threads heavily. Front-page ranking uses a score that combines vote balance with a time term so freshness and quality both matter, computed per post and refreshed on vote changes rather than on every page view.

The lesson that carries over: model the read you actually perform. Comments are written once but read constantly, so pay at write time or cache aggressively to make reads cheap, and let ranking be a stored score, not a query you run on every request.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-reddit

How Elasticsearch Searches Fast: The Inverted Index and Shard Routing

RONI DAS — Fri, 10 Jul 2026 09:36:49 +0000

Searching billions of documents for a phrase and getting ranked results in tens of milliseconds looks like magic. It is not. It comes down to two ideas working together: an index that maps words to documents instead of scanning documents for words, and a way to spread that index across machines so each holds only a slice. Understand both and full-text search stops being mysterious.

The core problem

A database scans rows. If you ask a plain database to find every document containing a word, it reads documents and checks them, which is linear in the amount of data. That is fine for exact key lookups and hopeless for free-text search across huge corpora. You need the opposite mapping. Instead of "given a document, what words does it have", you want "given a word, which documents have it". That inversion is the whole trick.

The second problem is size. One machine cannot hold the index for billions of documents, and one machine cannot serve the query load. So the index has to be split across nodes, and a query has to find the right nodes and combine their answers.

Key design decisions

Build an inverted index. At index time, each document is broken into tokens by an analyzer that lowercases, splits on word boundaries, and often strips or stems words. For every token, the engine keeps a posting list: the set of document ids that contain it, often with positions for phrase matching. A query for a word becomes a direct lookup of its posting list, not a scan. A multi-word query intersects or unions posting lists, which is fast because the lists are sorted.

Store the index in immutable segments. New documents go into small new segments rather than editing existing ones. Segments are immutable, which makes them cache-friendly and safe to read without locks. A background process merges small segments into larger ones over time. A delete is just a marker; the document is removed for real during a later merge.

Split an index into shards. An index is divided into shards, each a self-contained inverted index over a subset of documents. Shards spread across nodes, so both storage and query work are distributed. The number of shards is chosen up front because it determines how documents map to shards.

Route by hashing the document id. To decide which shard a document belongs to, the engine hashes its routing key, usually the id, modulo the shard count. The same hash decides where to look on read, so a lookup by id goes straight to one shard. A full-text query, which could match anywhere, scatters to all shards, each returns its top ranked hits, and a coordinating node gathers and merges them into the final ranking. This scatter-gather is why fixing the shard count early matters: change it and the hash no longer points at the right place.

The trade-offs

The inverted index makes reads fast and writes heavier. Indexing a document means analyzing it and updating many posting lists, and results are not visible until a refresh exposes the new segment. That is why search is near-real-time, not instant: there is a small, tunable delay between writing a document and being able to find it. You trade write latency and immediacy for very fast reads.

Immutable segments trade update efficiency for read speed and simplicity. Updates and deletes accumulate as new segments and delete markers, and merging them costs I/O and CPU in the background. Skip the merges and query speed degrades as segment count grows.

Sharding trades operational simplicity for scale. Too few shards and you cannot spread load; too many and every query pays coordination overhead across all of them, and each tiny shard wastes overhead. Because the shard count is baked into routing, getting it wrong means reindexing to change it.

How the real system does it

The real design analyzes text into tokens, stores an inverted index in immutable, periodically merged segments, splits each index into a fixed number of shards, and routes by hashing the routing key so id lookups hit one shard while full-text queries scatter to all shards and gather ranked results. Replicas of each shard add read capacity and failover.

The general lesson: pick the data structure that matches the question. Databases scan because they answer "what is in this row"; search inverts the index because it answers "which rows contain this term". The right structure, spread the right way, is what makes the impossible query fast.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-elasticsearch

Why Redis Is Fast Even Though It Is Single-Threaded

RONI DAS — Fri, 10 Jul 2026 09:31:34 +0000

Redis handles enormous request rates on one thread for command execution. That sounds backwards. Everyone is taught that more cores mean more throughput, so how does a single-threaded server outrun multithreaded databases? The answer is that Redis removed the two costs that usually dominate, disk and lock contention, and that changes what "fast" even means.

The core problem

Most databases spend their time on two things: waiting for disk, and coordinating threads that share data. Disk waits are slow by orders of magnitude compared to memory. Thread coordination means locks, and locks mean contention, context switches, and cache lines bouncing between cores. A multithreaded design fights these costs constantly, and a lot of engineering goes into fighting them.

Redis sidesteps both. Data lives in memory, so there is no disk in the hot path. And command execution runs on a single thread, so there are no locks around the data structures at all. If only one thread ever touches the data, you do not need to protect it. The problem "how do I coordinate many threads safely" disappears by never having many threads on the data.

Key design decisions

Keep everything in memory. Redis is an in-memory store. Reads and writes hit RAM, not disk. Persistence exists, through snapshots and an append-only log, but it happens in the background or on a schedule and is not on the critical path of a normal command. This is the single biggest reason for the speed: memory access is a different universe from disk seeks.

Run commands on one thread with an event loop. A single thread pulls ready connections from the operating system using an efficient event notification mechanism, executes each command to completion, and moves on. Because commands are short and there is no disk wait inside them, one thread churns through a very large number of them. No locks, no thread scheduling overhead, no cache-line ping-pong on shared data. Each command is effectively atomic because nothing else runs during it.

Use non-blocking I/O. The thread never sits idle waiting on one slow socket. The event loop asks the OS which sockets are ready and only touches those. Idle or slow clients do not stall the others.

Purpose-built data structures. Redis is not just key-value. It ships hashes, sorted sets, lists, sets, bitmaps, and more, each implemented for speed. A sorted set gives you ranked lookups in logarithmic time in memory, which is why leaderboards and rate limiters are one command instead of a query plan.

The trade-offs

Single-threaded execution means one slow command blocks everything. There is no other thread to make progress while a command runs, so a command that scans a huge collection freezes the whole server for its duration. The discipline this forces is to avoid expensive operations on large keys and to use cursor-based iteration instead of blocking scans.

In-memory storage trades durability and capacity for speed. Your dataset must fit in RAM, which is more expensive and more limited than disk, and a crash can lose whatever has not been persisted yet, depending on your persistence settings. You tune this by choosing between snapshotting and the append-only log, trading data safety against write overhead.

Single-threaded command execution also does not use extra cores for the data path. To use more cores you run more Redis instances and shard across them, pushing the parallelism up to the deployment level instead of inside one process. Newer versions do use extra threads for network I/O, but the command execution model stays single-threaded on purpose.

How the real system does it

The real design is memory-resident data, a single-threaded event loop over non-blocking sockets for command execution, atomic commands because nothing interleaves, and specialized data structures that turn common problems into single operations. Persistence and replication run around that hot path rather than inside it, and horizontal scale comes from sharding across instances.

The lesson worth keeping: threads are not the only path to speed. Removing the slow parts, disk and lock contention, can beat adding cores. Sometimes the fastest design is the one that does less coordination, not more.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-redis

How Telegram Delivers Messages to Million-Subscriber Channels

RONI DAS — Fri, 10 Jul 2026 09:31:30 +0000

Telegram has two jobs that pull in opposite directions. A private chat needs to sync perfectly across your phone, laptop, and tablet, with every message in order and nothing lost. A public channel needs to take one post and deliver it to millions of subscribers without melting the servers. The same product, two very different delivery problems.

The core problem

Cloud messaging means the server is the source of truth, not the device. Unlike a model where the device holds all state, here messages live on the server so any device can pull the full history and stay in sync. That makes multi-device easy but puts the load on the backend, and it means the server needs a way to tell each device exactly what it has and has not seen.

Channels break the usual chat assumption. A normal group has tens or hundreds of members, so you can afford to write a copy of each message to every member. A channel can have millions of subscribers. Writing a copy per subscriber for every post is a firehose of writes that does not scale.

Key design decisions

Sequence numbers, not timestamps, define order. Each chat has a monotonically increasing message sequence. A device remembers the last sequence it has, and on reconnect it asks for everything after that number. This makes sync a simple range fetch and sidesteps clock skew between devices. Ordering is defined by the server's counter, so all devices converge on the same order.

A per-device update stream. Rather than each device polling every chat, the server keeps a stream of updates per account and each device catches up from its last known point. Miss a few minutes of connectivity and you replay the gap, not the whole history.

Fanout on read for large channels. For a channel, do not copy the post into millions of inboxes. Store the post once in the channel's own message history. When a subscriber opens the channel, they read from that shared history at their last-read position. One write, many reads. This is fanout on read, and it is what makes million-subscriber channels affordable.

Fanout on write for small chats. For private and small group chats, the opposite is cheaper: push the message to each member's update stream immediately so their devices get it in real time without asking. With few members the write amplification is small and the latency win is real.

Push notifications are a separate concern. Waking a sleeping phone goes through platform push services and carries a hint, not the full message. The device then pulls the actual content over the sync channel, which keeps notification payloads small and lets the server stay the source of truth.

The trade-offs

Fanout on write gives instant delivery but its cost grows with membership. It is right for small chats and wrong for huge channels. Fanout on read scales to any number of subscribers but shifts work to read time, so a channel with a sudden spike of readers opening it at once creates a read burst instead of a write burst. Many systems, including this one, use a hybrid: write-fanout for small conversations, read-fanout for large broadcast channels, and pick per chat based on size.

Server-as-source-of-truth makes multi-device sync clean but concentrates load and trust on the backend. It is the reason history is available on a new device instantly, and also the reason the sync and storage layer has to be very robust. Sequence-based sync is simple and correct but assumes a reliable per-chat counter, which the storage layer has to guarantee even under failover.

How the real system does it

The real design keeps messages on the server as the authority, syncs each device with per-chat sequence numbers and a per-account update stream so reconnecting is a cheap range fetch, and splits delivery by scale: small chats fan out on write for low latency, massive channels store the post once and fan out on read. Push services only wake the device, which then pulls the real content.

The general lesson: there is no single correct fanout strategy. Choose write or read fanout based on the ratio of writers to readers, and do not be afraid to run both in the same product.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-telegram

How Delta Lake Brings ACID to a Data Lake

RONI DAS — Fri, 10 Jul 2026 09:26:13 +0000

A data lake is cheap object storage full of files. It is great for volume and terrible for correctness. Two jobs writing at once can corrupt a table, a reader can see a half-finished write, and there is no clean way to update or delete rows. The lakehouse idea is to keep the cheap storage but add a transaction layer on top, so a pile of files starts behaving like a real table. Delta Lake is one way to do that.

The core problem

Object storage gives you durable, cheap, effectively infinite space, but it gives you almost no guarantees about a collection of files as a unit. There is no atomic "replace these ten files with these twelve". A writer that fails halfway leaves the table in a broken state. A reader scanning the directory during a write sees an inconsistent mix. Appends are easy, but updates and deletes mean rewriting files, and doing that safely while others read is the hard part.

Classic warehouses solved this decades ago with a transaction manager, but they store data in proprietary formats on managed storage and cost more. The question is whether you can get the same ACID guarantees while keeping open file formats on plain object storage.

Key design decisions

A transaction log is the source of truth, not the file listing. Delta keeps an ordered log of commits alongside the data files. Each commit records which files were added and which were removed. The current state of the table is not "whatever files are in the directory" but "the result of replaying the log". A reader consults the log to learn exactly which files make up the table right now, so it never sees a partial write.

Writes are atomic through the log. A writer stages new data files, then makes one atomic commit that appends an entry to the log. Until that entry lands, readers see the old version. The moment it lands, they see the new one. There is no in-between, which is what turns a set of file operations into a single transaction.

Optimistic concurrency for multiple writers. Delta assumes conflicts are rare. Each writer reads the current log version, does its work, and tries to commit at the next version number. If someone else committed first, the loser detects the collision, checks whether the changes actually conflict, and retries against the new version. This avoids heavy locking while still keeping writers from clobbering each other.

Updates and deletes without breaking readers. Because the log tracks add and remove actions, an update rewrites only the affected files and commits a swap: remove the old files, add the new ones, atomically. Old files stick around for a retention window, which gives you time travel and rollback, since you can replay the log to an earlier version.

The trade-offs

The log is powerful but it is also a bottleneck you have to manage. Every commit appends to it, and over time it grows, so the system periodically checkpoints the log into a compact snapshot to keep reads fast. Small frequent writes create many small files and many log entries, which hurts read performance until a compaction job merges them. You trade write simplicity for a background maintenance burden.

Optimistic concurrency is cheap when writers rarely touch the same data and expensive when they do. Under heavy contention on the same files, writers keep retrying and throughput drops. If your workload is many writers hammering the same partition, a lock-based store may serve you better.

You also inherit object storage's latency and its consistency behavior. Metadata operations that a warehouse does in memory become log reads and file operations, so single-row lookups are not the point. The lakehouse wins on large scans over open formats, not on transactional point queries.

How the real system does it

The real pattern stores data as open columnar files on object storage, with an ordered transaction log that defines table state, atomic log commits for isolation, optimistic concurrency with conflict detection for multiple writers, and periodic checkpointing and compaction to keep the log and files healthy. Time travel falls out naturally from retaining old file versions the log still references.

The broad lesson: you do not need a new storage engine to get transactions. A well-designed log over immutable files can give you atomicity, isolation, and versioning on top of the cheapest storage you have.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-databricks

Why Snowflake Separates Storage From Compute

RONI DAS — Fri, 10 Jul 2026 09:26:09 +0000

Classic data warehouses tie your data to the machines that query it. Add more compute and you have to reshuffle the data. Run a heavy report and it starves your loading jobs. Snowflake's central idea is to break that tie: keep the data in one shared place and let any number of independent compute clusters read it. That one decision reshapes cost, scaling, and concurrency.

The core problem

In a traditional shared-nothing warehouse, each node owns a slice of the data on its local disk. This is fast, because compute sits next to its data. But it couples two things that have nothing to do with each other. Storage grows with how much data you keep. Compute grows with how many queries you run and how fast you want them. Coupling them means every time one dimension changes, the other has to be re-provisioned, and rebalancing data across nodes is slow and disruptive.

Worse, workloads fight. Your BI dashboards, your data science team, and your nightly load all hit the same cluster. One expensive query drags everyone down. You end up over-provisioning for the peak and paying for idle capacity the rest of the time.

Key design decisions

Three layers instead of one. Snowflake splits into storage, compute, and a cloud services layer. Storage is object storage in the cloud, holding table data as compressed files. Compute is made of clusters called virtual warehouses, each a group of nodes that read from that shared storage. The services layer handles metadata, query planning, transactions, and security.

Data lives in object storage, not on compute nodes. Tables are stored as immutable micro-partitions, columnar files of a bounded size. Because the files never change in place, a write produces new files and the metadata layer tracks which files make up the current table. This is what makes time travel and cheap cloning possible: you keep old file sets around and point at them.

Compute is elastic and isolated per workload. Each team spins up its own virtual warehouse. They all see the same data because they read the same storage, but they do not share CPU or memory, so one team's heavy query does not slow another. A warehouse can resize on demand and can be suspended when idle so you stop paying for it. Billing follows compute time, not a fixed cluster.

Metadata makes queries skip data. The services layer keeps min and max values and other statistics per micro-partition. A query with a filter can prune whole files without reading them, which cuts scan cost sharply on selective queries.

The trade-offs

Separating storage and compute adds network distance. Compute no longer sits on top of local disk, so reads cross the network to object storage. Snowflake hides this with aggressive local caching on the warehouse nodes, so hot data stays close, but a cold query pays the fetch cost. You trade the guaranteed locality of shared-nothing for elasticity and isolation.

Immutable storage trades write-in-place efficiency for versioning and concurrency. Updating a few rows means rewriting the affected files, which is heavier than an in-place update. In return you get snapshot isolation, easy rollback, and zero-copy clones, because everything is just pointers to file sets.

Elastic per-team compute can also surprise you on cost. It is easy to leave warehouses running or to scale up more than a query needs. The system bills by compute time, so the flexibility that saves money when managed can waste it when ignored.

How the real system does it

The real architecture is exactly this separation: durable columnar micro-partitions in cloud object storage, independent virtual warehouses that scale and suspend on their own, and a shared services layer that owns metadata, planning, and transactions. Caching on compute nodes recovers most of the locality lost by moving data off local disk, and partition pruning keeps scans cheap.

The general lesson: when two resources scale for different reasons, do not force them onto the same hardware. Decoupling storage from compute is why the warehouse can grow reads and writes independently and bill only for what runs.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-snowflake

How Spotify Streams Audio and Builds Discover Weekly

RONI DAS — Fri, 10 Jul 2026 09:20:53 +0000

Press play and music starts in under a second, anywhere in the world, on a phone that keeps switching between wifi and cellular. Once a week, a playlist appears that somehow knows you like a band you have never heard of. These are two very different systems wearing the same app, and both are worth understanding.

The core problem

Streaming has two enemies: startup latency and rebuffering. A user will forgive a slightly lower bitrate, but not silence. So the goal is to start fast and never stop, even when the network gets worse mid-song. Recommendation has a different enemy: the cold catalog. You have tens of millions of tracks and most of them almost never get played, so you cannot rank by popularity alone or you would recommend the same hits to everyone.

Key design decisions for playback

Serve audio from a CDN, not from origin. Tracks are static files. Encode each one into several bitrates ahead of time, push them to edge servers close to users, and let the client pick a bitrate based on measured throughput. The origin is only touched on a cache miss.

Chunk the audio and adapt. The client requests small segments rather than one long file. If bandwidth drops, the next segment is fetched at a lower bitrate. This is the same adaptive idea video streaming uses, applied to audio, where segments are small and switching is nearly inaudible.

Prefetch the next track. The player usually knows what comes next in a queue or playlist, so it fetches the first segments early. That is how the gap between songs feels like zero.

Cache aggressively on device. Recently played and downloaded tracks live in local storage, which cuts both latency and bandwidth cost, and makes offline playback possible.

Key design decisions for Discover Weekly

The recommendation side is a batch pipeline, not a live request. It runs on a schedule and writes a playlist per user that the app simply reads.

Collaborative filtering as the backbone. Build a large matrix of users and the tracks they play. Factor it into user vectors and track vectors so that similar listening habits sit near each other in a shared space. Two users who overlap heavily will surface each other's less common tracks. This is what makes the playlist feel personal without anyone hand-picking songs.

Blend in content and context. Pure collaborative filtering cannot rank a brand new track that nobody has played yet, the classic cold-start problem. So the pipeline adds signals from the audio itself and from text about the track, which lets fresh songs enter someone's recommendations before they have play history.

Filter what the user already knows. The point is discovery, so tracks the user plays often are removed, and the final list is capped at a fixed size and frozen for the week.

The trade-offs

Adaptive bitrate trades audio quality for continuity. On a bad connection you get a lower bitrate rather than a pause, which is almost always the right call for music. Prefetching trades bandwidth for smoothness, and wastes a little data when the user skips, which is a cheap price.

On recommendations, batch generation trades freshness for cost and stability. A weekly job over the full listening history is far cheaper than scoring in real time, and a playlist that does not change under your feet feels more trustworthy. The cost is that it cannot react to what you played an hour ago. Popularity-weighted ranking risks a feedback loop where hits get bigger, so the pipeline has to deliberately inject less popular tracks or discovery dies.

How the real system does it

The playback path leans on precomputed encodings, edge caching, and adaptive client-side bitrate selection, with device caching for the tracks you return to. The recommendation path is an offline pipeline built on matrix factorization over listening logs, mixed with content-based signals to handle new music, run on a weekly cadence and served as a static, per-user playlist.

The takeaway that generalizes: split the fast synchronous path from the heavy asynchronous one. Playback must answer in milliseconds, so keep it simple and cached. Personalization can take hours, so let it run as batch and just hand the app a finished answer.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-spotify

How Airbnb Search and Booking Avoid Double-Booking

RONI DAS — Fri, 10 Jul 2026 09:20:50 +0000

Two guests open the same listing at the same second. Both see the same free week in June. Both tap Reserve. Only one of them can win. Get this wrong and you have an angry guest, a refund, and a host who trusts you less. This is the single hardest correctness problem in a lodging marketplace, and it hides behind a search box that looks simple.

The core problem

Booking is a distributed race condition. The read path (search) and the write path (reserve) want opposite things. Search wants to be fast, cached, and eventually consistent across millions of listings. Reserve wants to be strongly consistent for one listing on specific dates. You cannot serve both with the same storage strategy, so you split them.

A naive design stores availability as a boolean per listing and checks it in application code: read "is it free", then write "now it is booked". Those two steps are not atomic. Between the read and the write, another request can slip in and read the same "free" value. Both proceed. You just double-booked.

Key design decisions

Model availability as date ranges, not a single flag. A reservation is a half-open interval [check_in, check_out). Two reservations conflict when their intervals overlap. Postgres can enforce this directly with an exclusion constraint using a GiST index over a daterange plus the listing id. The database rejects the second overlapping insert. No application-level check, no race.

Push the conflict check into the database, not the app. Application-level "check then set" is where double-booking lives. Whether you use an exclusion constraint, a unique constraint on (listing_id, date) with one row per night, or SELECT FOR UPDATE on the listing row, the winner is decided by the database inside a single transaction. Pick one and let it be the source of truth.

Separate the search index from the booking store. Search runs against a denormalized index (Elasticsearch or a read replica) filtered by location, dates, price, and guest count. That index can lag by seconds and nobody cares, because search results are a hint, not a promise. The promise is made only when the transaction commits against the primary store.

The trade-offs

Row-per-night is simple to reason about and makes "which nights are free" a trivial query, but a long stay writes many rows and hot listings create lock contention on popular dates. The daterange exclusion constraint is compact and elegant, but it ties you to Postgres and its GiST support, and a rejected insert is less obvious to a new engineer debugging it.

SELECT FOR UPDATE serializes writers on a listing, which is correct but limits throughput per listing. For a marketplace that is usually fine, because contention is per property, not global. You are not serializing the whole site, only the handful of people fighting over one cabin.

There is also the pending-hold question. Payment takes a few seconds and can fail. If you commit the booking before payment clears, a failed charge leaves a phantom reservation. If you wait for payment, a slow processor lets a competitor grab the dates. The common answer is a short-lived hold: reserve the dates in a pending state with an expiry, confirm on payment success, and let a background job release expired holds. The hold itself must go through the same conflict check, or you have just moved the race.

How the real system does it

Large lodging platforms keep booking on a strongly consistent relational store and treat search as a separate, replicated, denormalized read path. Availability is intervals, conflicts are enforced by the store rather than trusted to application code, and money moves through a saga: reserve, authorize payment, confirm, or compensate by releasing the hold. Idempotency keys on the reserve endpoint stop a double-tapped button or a client retry from creating two bookings.

The lesson that carries to other systems: when a read is allowed to be stale but a write must be exact, do not force them to share a consistency model. Give search speed and give booking correctness, and let a transaction be the only place the truth is decided.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-airbnb

Designing a payment system that never loses a cent

RONI DAS — Fri, 10 Jul 2026 09:14:38 +0000

Most systems can tolerate being a little bit wrong. A like count that is off by one, a feed that is a few seconds stale, a search result slightly out of order. Nobody gets hurt. A payment system does not have that luxury, because being wrong means charging someone twice or moving money into a void, and that is real harm to a real person. So the design goal is not just fast or scalable. It is correct under failure, and failure is guaranteed, because networks time out and servers crash in the middle of things.

The first problem to solve is duplicates. Imagine a user taps pay, the request reaches your server, you charge their card, and then the response back to their phone is lost to a flaky network. Their app, having heard nothing, retries. Now you are about to charge them a second time for one purchase. This is not a rare edge case, it is the normal behavior of unreliable networks, and any payment design that ignores it is broken. The answer is idempotency. The client attaches a unique key to the payment request, and the server records that key the first time it processes the request. If a request arrives with a key the server has already seen, it does not charge again. It returns the result of the original attempt. So a retry is safe, because the retry is recognized as the same operation, not a new one. Getting idempotency right is arguably the single most important decision in the whole system.

The second problem is knowing the truth about money, and this is where the ledger comes in. The wrong model is to keep a single balance number and mutate it up and down, because a mutable number tells you the current value but not how you got there, and when something looks wrong you have no way to reconstruct what happened. The right model is a ledger: an append only record where every movement of money is a new immutable entry. You never edit or delete an entry. To reverse a mistake you add a compensating entry. A balance is then not a stored number you trust, it is something you can derive by summing the entries. This is double entry accounting, the same idea banks have used for centuries, and it gives you an audit trail where every cent is traceable to the events that produced it. The trade-off is that you write more data and compute balances rather than reading a single field, but for money that is exactly the trade you want.

The third problem is that a payment is not one instant, it is a process with stages, and it can fail at any stage. Money gets authorized, then captured, then settled, and a payment can also be pending, failed, or refunded. Modeling this as a state machine, with explicit states and only certain allowed transitions, keeps the system honest. A payment cannot jump from failed to settled. A refund only applies to something that was actually captured. When a step fails partway, the state machine tells you exactly where you are and what recovery is valid, instead of leaving you guessing. This matters most when a payment spans more than one system, for example your service and an external card processor, because now you have two parties that can disagree about what happened.

That cross system disagreement is the fourth problem, and the honest answer is that you cannot make two independent systems update in one perfect atomic step. So you do not pretend to. You record your intent first, then call the external processor, then reconcile. If you sent a charge to the processor but never got a confirmation, you do not assume success or failure. You have a reconciliation process that later checks with the processor for the true outcome and settles your records to match. This is eventual consistency applied to money, done carefully, with the ledger and idempotency making it safe to retry and safe to verify. Accepting that you will sometimes be temporarily unsure, and building a reliable process to resolve that uncertainty, is more honest and more correct than pretending distributed atomic writes exist.

How does the real company do it? Payment platforms like PayPal and Stripe lean hard on idempotency keys so retries never double charge, an append only double entry ledger as the source of truth for balances, an explicit state machine for the lifecycle of each payment, and asynchronous reconciliation against banks and card networks to settle the truth after the fact. They favor being auditable and eventually correct over being fast and occasionally wrong.

The mindset shift for an interview is this: stop trying to make the operation instant and atomic, and instead make every step safe to retry, every movement recorded immutably, and every uncertain outcome resolvable by reconciliation. That is how you build something that never loses a cent.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-paypal

How Discord stores an enormous number of messages without falling over

RONI DAS — Fri, 10 Jul 2026 09:14:34 +0000

Chat feels like a beginner project. A messages table with a channel id, an author, some text, and a timestamp. Insert on send, select on read. It genuinely works, right up until the message table gets very large, and then the database that served you so well becomes the thing that wakes you up at night. Discord lived this, and how they got out of it is one of the clearer real world storage lessons around.

The core problem is volume and access pattern together. Messages accumulate forever and are never really deleted in bulk, so the table only grows. Meanwhile the read pattern is specific and relentless: give me the most recent messages in this channel, and let me scroll back from there. A traditional relational database can do this, but as the table grows into the range where it no longer fits comfortably in memory and the indexes get huge, the pain shows up as unpredictable latency. Some reads are fast, some hit cold data on disk and stall, and maintenance operations start to hurt. The system was not wrong, it was just the wrong shape for data that grows without bound and is always queried by channel and time.

The key decision was to move messages to a wide column store built for exactly this pattern, in Discord's case Cassandra and later a compatible successor. The important part is not the brand, it is the data model, because a store like this rewards you for modeling around your query and punishes you for fighting it. The query is "recent messages in a channel," so the design makes channel the thing that groups data together and time the thing that orders it within that group. Concretely, the partition key decides which physical bucket a row lives in, and the clustering key decides the order inside that bucket. You want messages for one channel to sit together and to be stored already sorted by time, so a "give me the latest messages here" read is a single ordered scan of one bucket rather than a hunt across the whole dataset.

There is a subtle trap in that model and it is worth calling out, because it is the kind of thing that separates a textbook answer from a real one. If you make the partition simply the channel id, then a channel that has been active for years accumulates an unbounded number of messages in a single bucket, and that bucket becomes a hot, oversized partition that is slow to read and hard to balance. The fix is to bound the partition by time as well, for example by bucketing on channel plus a time window such as a range of days. Now each partition holds a manageable slice, active channels spread their history across many buckets, and no single partition grows forever. The trade-off is that reading a long scrollback may span more than one bucket, but that is a cheap, predictable cost compared to a single partition that never stops growing.

This kind of store buys throughput and smooth scaling by giving up things a relational database hands you for free. You largely give up rich cross row queries and joins, and you accept eventual consistency and the need to design around your access pattern up front. For chat, that is a good trade. You almost never need to join messages against something else in the hot path, and you always know you are reading by channel and time. When your queries are that predictable, modeling the storage tightly around them is a strength, not a limitation.

One more practical piece: a store like this handles deletes and updates through a mechanism that marks rows rather than removing them immediately, and those markers can pile up and slow reads if a workload does a lot of deletion. That is the sort of operational detail that does not show up in a whiteboard design but absolutely shows up in production, and being aware of it is part of choosing the tool honestly.

How does the real company do it? Discord moved messages off a general purpose relational database onto a wide column store, partitioning by channel and a time bucket and clustering by message id so recent history is a fast ordered read. They later migrated to a Cassandra compatible database for better performance and lower operational pain, but the data model stayed the same because the model was the real insight. In front of the store sits caching for hot channels so the most active conversations do not hammer the database on every read.

The lesson that generalizes is this: when data grows without bound and is always read the same way, pick a store that lets you model directly around that access pattern, and bound your partitions so nothing grows forever.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-discord

Designing Google Maps: geospatial indexing and routing without melting the server

RONI DAS — Fri, 10 Jul 2026 09:09:18 +0000

A mapping app hides two genuinely hard computer science problems behind a clean interface. The first is spatial search: given my location, find the coffee shops, gas stations, or friends near me. The second is routing: find the fastest way from here to there across a road network with millions of edges. They need different tools, and understanding why is most of the design.

Start with spatial search, because it exposes a nice trap. Say you store every place with a latitude and longitude. To find things near me, the naive query is "give me every place whose distance to my point is less than one kilometer." That works, and it is also a disaster, because it forces the database to compute distance to every single place before filtering. There is no index that a plain distance formula can use. You need a way to turn two dimensional position into something a normal index can handle.

That is what geospatial indexing does, and there are two common approaches. One is a tree structure such as a quadtree, which recursively divides the map into four cells, then divides busy cells again, so dense areas like city centers get fine grained cells and empty areas stay coarse. To find nearby places you walk down to the cell containing your point and look at that cell and its neighbors, instead of scanning the whole world. The other approach is a grid or geohash system, which chops the surface into cells and gives each a short string id, where nearby places tend to share a prefix. Then "find things near me" becomes "find things whose cell id matches this prefix," which an ordinary index handles well. The trade-off between them is roughly flexibility versus simplicity. Trees adapt beautifully to uneven density but are more work to maintain. Grid and geohash schemes are simple and index friendly but need care at cell boundaries, because two points can be physically close while sitting in different cells. Either way, the win is the same: you replace a scan of everything with a lookup of a small local region.

Now routing, which is a different beast entirely. The road network is a graph. Intersections are nodes, roads are edges, and each edge has a cost such as time or distance. Finding the fastest route is a shortest path problem. The textbook answer is Dijkstra's algorithm, and it is correct. It is also far too slow to run across a continent scale graph on every request, because it explores outward in all directions until it reaches the destination, touching an enormous number of nodes for a long trip.

So the real decisions in routing are about not running the full search every time. The first improvement is to guide the search toward the destination instead of expanding blindly, which is the idea behind A star: use an estimate of remaining distance so the search leans in the right direction. That helps, but for very long routes it is still not enough. The bigger idea is precomputation. You process the graph offline to build shortcuts, so that at query time you can skip across large stretches of the map in a few hops instead of crawling road by road. Techniques in this family precompute a hierarchy of the network, importantly ranking, so that long distance routing hops along a small set of major connections much like a human plans a trip by highways first and local streets last. The trade-off is heavy offline computation and extra storage in exchange for query responses fast enough to feel instant. For a service answering an enormous number of route requests, moving cost from query time to precompute time is exactly right.

Two more layers make it usable. Real routing needs live traffic, so edge costs are not fixed. They are updated from current conditions, which means the precomputed structures have to tolerate changing weights without being rebuilt from scratch. And the map itself is served as tiles, small pre rendered images or vector chunks per zoom level and region, delivered through a CDN so panning and zooming pull cached tiles instead of rendering the world on demand. That tiling is the same pre compute and cache idea showing up again in a different place.

How does the real company do it? Google Maps uses hierarchical spatial indexing to answer nearby queries and heavy offline preprocessing of the road graph so that routing at query time is closer to a lookup than a full search. Traffic data continuously adjusts edge costs, and the visible map is served as cached tiles through a global CDN. The recurring theme is precomputation: do the expensive graph and rendering work ahead of time so the request path stays cheap.

The interview takeaway is to separate the two problems clearly. Spatial search wants a geospatial index. Routing wants a graph plus precomputed shortcuts. Answering both with one tool is the mistake.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-google-maps

How Twitter's timeline actually works: the fanout problem

RONI DAS — Fri, 10 Jul 2026 09:09:14 +0000

When people first design a Twitter timeline, they write one query: give me the most recent posts from everyone this person follows, sorted by time. It works perfectly in a demo with a hundred users. It falls apart in production, and understanding why is the whole lesson.

The problem is the shape of the graph. Follower counts are wildly uneven. Most accounts follow and are followed by a modest number of people. A small number of accounts have tens of millions of followers. Any design that treats all accounts the same will be wrong for one end of that distribution. This is the fanout problem: when someone posts, how do you get that post into the timelines of everyone who should see it.

There are two base strategies and it is worth being precise about both.

Fanout on read means you build the timeline when the user asks for it. They open the app, and the system gathers recent posts from everyone they follow, merges them by time, and returns the list. Posting is cheap, just one write. Reading is expensive and gets worse the more accounts you follow, because every timeline load is a live gather across many sources. Do this for every user on every refresh and your database spends all its time assembling timelines.

Fanout on write means you build the timeline in advance. The moment someone posts, you push a reference to that post into a precomputed timeline for each of their followers. These timelines usually live in an in memory store, one list per user, so reading is close to instant because the answer is already sitting there. The catch is the write. If an account with forty million followers posts, that one action tries to write into forty million lists. That is a huge, spiky amount of work for a single event, and if a popular account posts often, the system is constantly doing these enormous fanouts.

So the honest answer to "which one" is: both, split by account size. Ordinary accounts use fanout on write. When they post, the system pushes to their followers' precomputed timelines, and those followers get fast reads. High follower accounts are the exception and are deliberately not fanned out. Their posts stay in their own store, and when you load your timeline, the system merges in recent posts from the few big accounts you follow at read time. So a normal timeline load is mostly a fast read of a precomputed list, plus a small live merge for the handful of celebrities you follow. You avoid the impossible write amplification of fanning a celebrity post to tens of millions of lists, and you keep reads fast for everyone else.

A couple of details make this practical. The precomputed timelines do not need to hold every post forever. They hold a bounded window of recent references, often a few hundred, because almost nobody scrolls past that in a session, and older content can be fetched on demand. The timelines store post ids, not full post content, so the list stays small and the actual post text and media are hydrated from a separate store when rendering. That separation keeps the hot timeline data compact and cheap to keep in memory.

There is also the question of ordering. Strict reverse chronological order is simple to reason about, but the modern timeline is ranked, which means posts are scored and reordered rather than shown purely by time. Ranking adds a scoring step on top of the retrieval you just built. The important point for a system design is that retrieval and ranking are separate stages. First you gather the candidate posts using the fanout machinery above, then a ranking layer decides the order. Keeping those two stages distinct is what lets you change ranking without touching how posts are delivered.

How does the real company do it? Twitter's timeline has long used a hybrid fanout model with precomputed timelines held in an in memory store, plus special handling that excludes very high follower accounts from write time fanout and merges them in at read time. On top of retrieval sits a separate ranking layer. The exact internals have changed over the years, but the core shape, precompute for the many and pull live for the few giants, has been remarkably stable.

If you remember one thing, remember that the whole design exists because follower counts are not uniform. A single rule cannot serve both a normal user and an account with tens of millions of followers, so you use two rules and split on account size.

I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-twitter