Fanout Push vs. Pull: Solving the Feed Distribution Problem at Scale
Modern systems don’t fail because they can’t store data — they fail because they can’t deliver the right data to the right consumers at the right time.
The fanout problem is one of the most fundamental challenges in distributed systems, especially visible in social media feeds, notification systems, messaging platforms, and event-driven architectures.
At its core, fanout answers one question:
When a single event occurs, how do we efficiently deliver it to millions of consumers?
There are two dominant strategies:
- Fanout-on-Write (Push)
- Fanout-on-Read (Pull)
Both are valid. Both are widely used. And both come with serious trade-offs.
Let’s break it down.
1. The Actual Fanout Problem
Imagine a social media post:
- A user posts one update
- That user has 10 followers… or 100 million followers
-
Each follower expects:
- Low latency
- Personalized feeds
- High availability
The naive solution — “just send the post to everyone” — collapses under:
- Write amplification
- Storage explosion
- Hot partitions
- Latency spikes
This is fanout:
One write → many reads
The challenge is deciding when and where that fanout happens.
2. Fanout-on-Write (Push Model)
How It Works
When a user creates content:
- The system immediately distributes (pushes) the content to followers
- Each follower gets a precomputed feed entry
- Reading the feed becomes a simple lookup
User posts → System pushes post to N follower feeds
What Problem It Solves
- Ultra-fast feed reads
- Predictable latency
- Minimal computation during reads
Why It’s Powerful
Reads massively outnumber writes in social systems.
Fanout-on-write moves complexity to write time, making reads cheap.
Core Trade-Offs
| Advantage | Cost |
|---|---|
| Extremely fast reads | Massive write amplification |
| Simple read queries | Heavy storage usage |
| Predictable latency | Hot users can overload the system |
A celebrity with 100M followers can generate:
100M feed writes for one post
This is not theoretical — it’s an operational nightmare.
3. Fanout-on-Read (Pull Model)
How It Works
- Content is written once, stored centrally
- When a user opens their feed:
- The system fetches recent posts from accounts they follow
- Merges, ranks, and filters them at read time
User opens feed → System pulls content from many sources
What Problem It Solves
- Eliminates write amplification
- Handles massive fanout safely
- Simplifies writes
Trade-Offs
| Advantage | Cost |
|---|---|
| Minimal write cost | Complex, expensive reads |
| Scales well for celebrities | Higher latency |
| Storage-efficient | Harder to cache |
This model pushes complexity to query time.
4. Hybrid Fanout (The Real-World Solution)
Almost no large platform uses pure push or pull.
They use hybrids.
Why?
Because:
- Not all users are equal
- Not all content deserves precomputation
- Not all reads need the same latency
5. How Major Social Media Platforms Solve Fanout
Twitter (X)
Hybrid Fanout Strategy
- Push for normal users
- Pull for celebrities
Why?
- The top ~0.01% of users generate extreme fanout
- Pushing their tweets would melt storage and queues
So:
- Tweets from celebrities are not pushed
- They’re fetched dynamically at read time
This is called Selective Fanout
Primarily Fanout-on-Write
- Posts are pushed into followers’ feeds
-
Heavy use of:
- Precomputation
- Ranking models
- Feed materialization
Why it works:
- Facebook prioritizes low-latency scrolling
- Heavy investment in storage and background processing
They mitigate costs with:
- Asynchronous workers
- Backpressure controls
- Partial fanout (not all posts go to all feeds)
Push + Pull Hybrid
- Push for normal users
-
Pull for:
- Celebrities
- Suggested content
- Ads
- Reels
Feed is composed from:
- Precomputed feed entries
- Dynamically fetched content
Mostly Fanout-on-Write
- Connections are limited
- Professional graph is smaller and denser
- Easier to precompute feeds
TikTok / YouTube Shorts
Primarily Fanout-on-Read
- Feed is interest-based, not follower-based
-
Content is pulled from:
- Recommendation pools
- ML-ranked candidate sets
This model would be impossible with push.
6. Fanout Beyond Social Media (Critical Non-Obvious Use Cases)
Fanout strategies solve problems far beyond feeds.
1. Notifications Systems
Push Fanout
- Send alerts to millions of devices
-
Use:
- Message queues
- Topic-based pub/sub
- Rate limiting
Examples:
- Mobile push notifications
- Emergency alerts
- Trading alerts
2. Event-Driven Architectures
Pull Fanout
- Consumers pull from Kafka partitions
-
Enables:
- Backpressure
- Replayability
- Fault isolation
Kafka is fanout-on-read by design.
3. CDN Cache Invalidation
Push
- Purge or update cache globally
-
Used for:
- Security patches
- Breaking content changes
Pull
- Lazy fetching of rarely accessed assets
4. Observability & Metrics
- Prometheus: Pull fanout (scraping)
- Datadog: Push fanout (agents)
Choice depends on:
- Control vs latency
- System stability during failures
5. Multiplayer Gaming
-
Push state updates for:
- Nearby players
- Active sessions
Pull world state on reconnect
Hybrid fanout prevents bandwidth explosion.
6. Financial Market Data
-
Push for:
- Price ticks
- Trade executions
-
Pull for:
- Historical data
- Analytics queries
Latency requirements dictate fanout model.
7. Configuration & Feature Flags
-
Push updates to services for:
- Kill switches
- Emergency rollbacks
Pull periodically for consistency
7. How to Choose the Right Fanout Strategy
Use Fanout-on-Write (Push) When:
- Read latency must be minimal
- Fanout size is bounded
- Storage is cheap
- Predictable performance is critical
Use Fanout-on-Read (Pull) When:
- Fanout size is unbounded
- Writes must be cheap
- Consumers vary wildly in demand
- Backpressure is required
Use Hybrid When:
You want to survive real-world traffic 😄
8. Final Mental Model
Think of fanout like logistics:
- Push = Home delivery (fast, expensive, predictable)
- Pull = Warehouse pickup (cheap, flexible, slower)
- Hybrid = Amazon Prime 😉
Top comments (0)