DEV Community

Cover image for Design Twitter Without the Fanout-on-Write Cliché
Gabriel Anhaia
Gabriel Anhaia

Posted on

Design Twitter Without the Fanout-on-Write Cliché

#architecture #database #tutorial #programming

The panel says: design Twitter. You have 45 minutes. Half the candidates draw the same diagram. They write to a fanout service, push tweet IDs into Redis lists keyed by feed:<user_id>, read the home timeline by LRANGE. Done in 12 minutes. The interviewer nods politely, then asks the question that ends the loop: "what happens when one of those users has 200 million followers?"

You watch the candidate try to bolt on an answer. Cache the celebrity timeline. Add a queue. Shard harder. None of it lands, because the hot key was baked in at minute 4.

Don't be them. The cliché answer is fine at small scale. The interview is asking whether you can name the moment it breaks and switch strategies. This is the version of "design Twitter" that surfaces the trade-off the panel is actually grading.

The cliché baseline, fast

Just enough to move past it. Fanout-on-write means: when user A tweets, the write path looks up A's followers and pushes the tweet ID into each follower's home timeline cache. Reads are cheap, one Redis call per user.

def post_tweet(author_id: int, tweet: dict) -> int:
    tweet_id = tweets.insert(tweet)
    followers = social.followers_of(author_id)
    for fid in followers:
        redis.lpush(f"feed:{fid}", tweet_id)
        redis.ltrim(f"feed:{fid}", 0, 799)
    return tweet_id


def get_home_timeline(user_id: int, limit: int = 50):
    ids = redis.lrange(f"feed:{user_id}", 0, limit - 1)
    return tweets.mget(ids)
Enter fullscreen mode Exit fullscreen mode

Why teams pick it. The home timeline is read 100x more than it's written for an average user, so paying once at write time is a sensible trade. LRANGE is one round trip. Tweet hydration is a multi-get against the tweet store.

Where it breaks. The for-loop. When followers_of(author_id) returns 50 follower IDs you finish the write in milliseconds. When it returns 50 million IDs, you have a write amplification problem that no amount of Redis can absorb. The kind of trade-off the panel wants you to surface (not the literal numbers) is that fanout cost grows linearly with follower count, and a small number of accounts have follower counts that are orders of magnitude above the median.

That asymmetry is the whole interview.

The celebrity problem, named clearly

Call this the hot-key problem. One author, one write, millions of downstream insertions. Three things go wrong at once.

Write latency explodes. A single tweet from a top-1000 account stalls the fanout queue. Other authors' tweets queue behind it. Latency p99 blows up across the system, not just for the celebrity.

Memory is wasted. Most followers of a celebrity will not open the app in the next hour. You wrote the tweet ID into 50 million Redis lists; maybe 2 million are read before the tweet falls off the timeline. The other 48 million writes were free RAM cost.

The hot key follows you. If you try to cache "celebrity tweets" centrally instead, every read from every follower hits the same cache key. One node, all the traffic. Redis Cluster won't save you because the slot is the slot.

Switch strategies for that author here. Scaling the existing one harder doesn't help.

Fanout-on-read for the long tail of celebrities

For high-follower accounts, flip the model. Don't push their tweets to followers at write time. Store celebrity tweets in a separate per-author timeline, and merge them in at read time when a follower asks for their home feed.

CELEBRITY_FOLLOWER_THRESHOLD = 1_000_000


def post_tweet(author_id: int, tweet: dict) -> int:
    tweet_id = tweets.insert(tweet)
    if is_celebrity(author_id):
        # No fanout. Store on author timeline only.
        redis.lpush(f"author:{author_id}", tweet_id)
        redis.ltrim(f"author:{author_id}", 0, 799)
    else:
        for fid in social.followers_of(author_id):
            redis.lpush(f"feed:{fid}", tweet_id)
            redis.ltrim(f"feed:{fid}", 0, 799)
    return tweet_id
Enter fullscreen mode Exit fullscreen mode

The read path now has work to do. It pulls the precomputed home timeline (cheap), pulls the recent tweets from each celebrity the user follows (a handful of small reads), merges them in chronological order, and trims.

def get_home_timeline(user_id: int, limit: int = 50):
    base_ids = redis.lrange(f"feed:{user_id}", 0, 199)
    celeb_authors = social.celebs_followed_by(user_id)

    celeb_ids: list[int] = []
    for author_id in celeb_authors:
        celeb_ids += redis.lrange(f"author:{author_id}", 0, 49)

    merged = merge_by_timestamp(base_ids + celeb_ids)
    return tweets.mget(merged[:limit])
Enter fullscreen mode Exit fullscreen mode

This is the move the panel is waiting for. You traded a write-time cost (linear in followers) for a read-time cost (linear in celebrities-followed). The numbers behind that trade are favorable: the median user follows a small number of high-follower accounts. Even if you follow 200 celebrities, that's 200 small Redis reads per timeline load. Pipelined, that's one round trip.

The threshold, and how to detect it

The question every interviewer asks next: where do you put the line? Pick a number that's easy to defend, then explain how you would tune it.

def is_celebrity(author_id: int) -> bool:
    count = follower_count_cache.get(author_id)
    if count is None:
        count = social.follower_count(author_id)
        follower_count_cache.set(author_id, count, ttl=3600)
    return count >= CELEBRITY_FOLLOWER_THRESHOLD
Enter fullscreen mode Exit fullscreen mode

A static threshold (e.g. one million followers) is the answer to give first. It's auditable, easy to reason about, and you can ship it. Then layer the nuance.

Posting rate matters. A million-follower account that tweets once a month is not the same load profile as a hundred-thousand-follower account that tweets every 90 seconds during a livestream. A better signal is followers * recent_post_rate.

Promotion has to be smooth. When an account crosses the threshold, you don't want to atomically flip them. Existing followers have stale entries on their fanout timeline; new tweets need to come in through the read path. Run a backfill window: for the next N hours, the account's writes go to both paths. Followers' merge code dedupes by tweet ID. Then drop the fanout side.

Demotion is the underrated direction. Accounts lose followers, go inactive, get suspended. A celebrity that no longer tweets shouldn't keep paying the read-side merge cost forever. Periodically re-evaluate, and demote stale celebrities back into normal fanout.

def reclassify(author_id: int) -> str:
    f = social.follower_count(author_id)
    rate = activity.posts_per_day(author_id, window_days=14)
    score = f * max(rate, 0.1)
    if score >= 5_000_000:
        return "celebrity"
    if score >= 500_000:
        return "borderline"
    return "normal"
Enter fullscreen mode Exit fullscreen mode

borderline is a useful third bucket. Borderline accounts fanout to active followers only, skipping followers who haven't opened the app in 30 days. That's the per-follower budget knob.

The per-follower budget

The hidden lever in any fanout system. You don't owe every follower the same delivery treatment.

The naive read of "fanout-on-write" is that every follower's timeline gets every tweet. In production, that's wasteful. A follower who hasn't opened the app in three weeks doesn't need their Redis list updated in real time. They'll get a fresh feed assembled on demand when they come back.

def fanout_to_followers(author_id: int, tweet_id: int):
    followers = social.followers_of(author_id)
    active = activity.filter_active(followers, days=7)

    pipe = redis.pipeline()
    for fid in active:
        pipe.lpush(f"feed:{fid}", tweet_id)
        pipe.ltrim(f"feed:{fid}", 0, 799)
    pipe.execute()

    # Inactive followers: rebuilt on next visit from
    # author timelines + a sliding window query.
Enter fullscreen mode Exit fullscreen mode

Two effects. Fanout cost shrinks to the active follower set, which is a fraction of total followers. Inactive users still get a coherent timeline when they return, because you assemble it on read from author timelines anyway. That's the same code path the celebrity case already needed. You wrote it once; reuse it.

This is also where you tell the interviewer about the bound you'd put on it. Cap fanout at, say, the most recently active 500k followers per author. Above that cap, the account is treated as a celebrity for delivery, even if its raw follower count is below the threshold. The system has a worst-case-write SLO and you can defend it.

What the hybrid looks like end-to-end

Three paths, picked at write time.

Author class Write path Read path
Normal Fanout to active followers Read precomputed feed:<uid>
Borderline Fanout to active followers, capped Read precomputed feed
Celebrity Append to author:<aid> only Merge feed + per-celeb tail

The read code is the same in all three cases. It always tries the precomputed feed, then merges in celebrity tails for the celebrities you follow. Normal users follow zero celebrities and the merge is a no-op. Heavy users of celebrity content pay the merge once per timeline load, on a path that pipelines well.

Two things to call out before the panel does.

Ordering on merge. Tweet timestamps from different sources are not enough; clock skew between fanout writers and author-timeline writers will give you out-of-order feeds at the boundary. Sort by (tweet_id, snowflake_time) where snowflake_time is monotonically assigned at the tweet store. The author-timeline path and the fanout path both write the same tweet ID, so the merge is deterministic.

Backpressure on celebrity reads. If a celebrity tweets and 50 million followers open the app at once, the celebrity's author:<aid> Redis key becomes the new hot key. Mitigate with a small in-memory cache at the read service (per-pod, 1-second TTL) so the per-celeb tail is fetched once per pod per second, not once per request. The math: a thousand pods, one second TTL, one Redis read per pod per second per hot author. That's 1000 reads/sec on the hottest key, which a Redis replica handles without breathing.

For more depth on these trade-offs and the interview frame around them, the System Design Pocket Guide: Interviews walks through the Twitter design alongside 14 other system designs the same way: the cliché answer, the failure mode, and the version that earns the offer.

What to draw first

Start with the threshold box on the whiteboard. One arrow into it labeled post_tweet, two arrows out: one to fanout, one to author timeline. Then sketch the read service pulling from both and merging. The panel will ask one of three follow-ups next: how you'd promote an account across the threshold without a thundering-herd refresh, how you'd pick the active-follower window, or how the merge handles a celebrity who deletes a tweet. Have a one-paragraph answer for each ready, and the loop turns into a conversation about which dial to turn.

If this was useful

The Twitter design is one of fifteen worked through end-to-end in the System Design Pocket Guide: Interviews. Each chapter follows the same shape: the obvious answer, the failure case the interviewer is hunting for, and the design that handles it. If you're prepping for a senior-or-above loop and want the version of these answers that doesn't sound like everyone else's, it's written for you.

System Design Pocket Guide: Interviews

Top comments (0)