How Twitter Scales Its Home Timeline: Fan-Out on Read vs Fan-Out on Write

Imagine opening Twitter (now X). Your home timeline shows tweets from hundreds or thousands of accounts you follow, sorted by time.

Seems simple, right?

But think about the scale:

• Millions of users accessing their timelines.
• Millions of tweets being published.
• Every user follows a different set of accounts.
• Some accounts have millions of followers.

How do you generate everyone's timeline efficiently?

Martin Kleppmann discusses this interesting problem in Chapter 1 of Designing Data-Intensive Applications (DDIA).

Let's understand the approaches.

The Problem

Suppose Alice follows 1,000 accounts.

Whenever Alice opens her home timeline, the system needs to fetch recent tweets from those accounts, combine them, sort them, and display them.

Now imagine millions of users doing this simultaneously.

How should we design this?

Solution A: Fan-Out on Read (Pull Model)

Instead of preparing timelines in advance, generate them whenever a user requests them.

How it works:

1. User opens their home timeline.
2. Fetch the list of accounts they follow.
3. Fetch recent tweets from those accounts.
4. Merge and sort the tweets by timestamp.
5. Return the timeline.

       User opens timeline
                |
                v
       Fetch following list
                |
       +--------+--------+
       |        |        |
       v        v        v
    User A   User B   User C
       |        |        |
       v        v        v
    Tweets   Tweets   Tweets
       |        |        |
       +--------+--------+
                |
                v
          Merge & Sort
                |
                v
          Home Timeline

Advantages:

• Publishing a tweet is relatively cheap.
• No need to update millions of timelines when someone publishes a tweet.

Disadvantages:

• Every timeline request requires multiple reads.
• Merging and sorting tweets can be expensive.
• High read traffic can put significant pressure on the system.

We optimize writes, but reads become expensive.

Solution B: Fan-Out on Write (Push Model)

What if we prepare timelines when tweets are published instead?

Instead of generating the timeline on every request, maintain a precomputed timeline for each user.

How it works:

1. Alice publishes a tweet.
2. Find all her followers.
3. Push the tweet ID into each follower's home timeline cache.
4. When a follower opens Twitter, simply fetch their precomputed timeline.

          Alice publishes tweet
                    |
                    v
             Fan-Out Worker
                    |
          Find all followers
                    |
          +---------+---------+
          |         |         |
          v         v         v
       Bob's     Carol's    David's
      Timeline  Timeline   Timeline
          |         |         |
          +---------+---------+
                    |
                    v
             Precomputed Feed

Advantages:

• Reading timelines becomes extremely fast.
• Most of the computation happens during publishing.
• Great for read-heavy applications.

Disadvantages:

• Publishing becomes expensive for accounts with millions of followers.
• A single tweet can trigger millions of writes.
• High-follower accounts can create enormous write amplification.

We optimize reads, but writes become expensive.

The Real Challenge: What About Celebrities?

Here's where things get interesting.

Consider an account with 30 million followers.

Every time this account publishes a tweet, Fan-Out on Write could trigger 30 million timeline updates.

That's a lot of unnecessary work!

So, which approach should we choose?

Solution C: The Hybrid Approach

Why not combine both approaches?

That's the idea behind Twitter's hybrid solution.

• Normal accounts: Use Fan-Out on Write. Precompute followers' timelines when a tweet is published.
• Celebrity accounts: Use Fan-Out on Read. Don't push tweets into millions of followers' timelines. Fetch their tweets when followers request their timelines.

       Tweet Published
              |
       +------+------+
       |             |
       v             v
   Normal User    Celebrity
       |             |
       v             v
   Fan-Out       Store Tweet
   on Write          |
       |             |
       v             | (Fetch on read)
  Update follower    |
   timelines         |
       |             |
       +------+------+
              |
              v
       User opens timeline
              |
              v
       Merge & Sort
              |
              v
        Home Timeline

When a user opens their timeline:

1. Fetch their precomputed timeline.
2. Fetch recent tweets from celebrity accounts they follow.
3. Merge both sets of tweets.
4. Sort them by timestamp.
5. Return the final timeline.

This balances the cost of reading and writing.

What Should We Learn From This?

There is no universally perfect approach. Every architectural decision comes with trade-offs.

Approach	Optimizes	Main trade-off
Fan-Out on Read	Writes	Expensive reads
Fan-Out on Write	Reads	Expensive writes
Hybrid	Both	More architectural complexity

The right choice depends on your workload, traffic patterns, and data distribution.

If you want to understand the underlying Fan-Out Pattern, I've covered it in a separate article.

TL;DR

Fan-Out on Read generates timelines when users request them. Fan-Out on Write prepares timelines when tweets are published. A hybrid approach combines both to handle different user workloads efficiently.

Reference: Designing Data-Intensive Applications (DDIA) by Martin Kleppmann — Chapter 1, Twitter's Home Timeline.