We Built a Social Feed for 100K Users — Here's What Broke First

Fan-Out on Write vs Fan-Out on Read: Choosing a Social Feed Architecture

If you're building a social feed, there's one decision that determines most of your scaling story, and it's easy to make by accident.

Almost every feed starts the same way — a join against the follow graph, ordered by recency. It's the correct starting point. The trouble is that it stops being correct at a specific, predictable point, and teams usually discover this in production rather than in a design review.

This is a walkthrough of the two architectures, where each one breaks, and the hybrid that most large feeds converge on.

The starting point

SELECT p.*
FROM posts p
JOIN follows f ON f.followee_id = p.author_id
WHERE f.follower_id = $1
ORDER BY p.created_at DESC
LIMIT 20;

This is fan-out on read: the timeline is computed at request time. Writes are trivial — insert one row. Reads do all the work. Content is always fresh, and there's no denormalised state to maintain.

The cost of this query scales with two things: how many accounts the user follows, and how much those accounts post. That matters more than it first appears, because social graphs are heavily skewed. Your median user follows a modest number of accounts and this query serves them fine. Your power users follow hundreds or thousands — and they're usually your most engaged cohort.

The practical consequence is a metrics trap: p50 latency stays healthy while p99 degrades badly. If your dashboards show averages, the feed looks fine right up until your most valuable users start complaining.

Indexing helps. An index on (author_id, created_at DESC) is essential, and a covering index buys more. But indexes don't change the shape of the problem — they move the threshold, they don't remove it.

Fan-out on write

The inversion: when someone publishes, push the post ID into a precomputed timeline for every follower. Reads become a single lookup with no join.

Redis sorted sets are the natural fit — one key per user, score by timestamp:

async function fanOutPost(post, followerIds) {
  const pipeline = redis.pipeline();
  for (const followerId of followerIds) {
    pipeline.zadd(`timeline:${followerId}`, post.createdAt, post.id);
    pipeline.zremrangebyrank(`timeline:${followerId}`, 0, -801); // keep newest 800
  }
  await pipeline.exec();
}

Reads collapse to a ZREVRANGE plus a batch fetch of post bodies. Sub-10ms timeline retrieval is achievable and stays flat as the graph grows.

Three details matter here:

Trim aggressively. Nobody scrolls to entry 5,000. Capping each timeline at a few hundred entries keeps memory bounded and predictable. Users who scroll past the cap fall back to a database query — rare enough that it doesn't matter.

Score by the post's own timestamp, not processing time. Background jobs don't complete in enqueue order. Scoring by when the worker happened to run produces visibly out-of-order feeds.

Sorted sets are idempotent by member. ZADD with the post ID as member means a job processed twice is harmless. This is a genuine advantage over lists, where retry-after-failure produces visible duplicates — and workers do get restarted mid-job.

The celebrity problem

Fan-out on write means one publish triggers N writes, where N is the follower count. An account with a million followers generates a million writes from a single action. That queue depth delays fan-out for every other post in the system, so one popular account posting degrades the experience for users who don't even follow them.

This is why pure fan-out on write doesn't survive contact with a real social graph.

The hybrid

Most feeds at scale converge on the same answer: fan-out on write for ordinary accounts, fan-out on read for high-follower accounts, merged at read time.

async function getTimeline(userId, limit = 20, cursor = null) {
  const maxScore = cursor ?? '+inf';

  const [precomputed, highFollowerPosts] = await Promise.all([
    redis.zrevrangebyscore(
      `timeline:${userId}`, maxScore, '-inf',
      'WITHSCORES', 'LIMIT', 0, limit
    ),
    fetchRecentPostsFromLargeAccounts(userId, limit, cursor),
  ]);

  return mergeByScore(precomputed, highFollowerPosts).slice(0, limit);
}

The threshold is empirical, not derived. Set it too low and you lose the benefit of precomputation because too many accounts are read-path. Set it too high and queue backpressure returns. Somewhere in the low tens of thousands of followers is a common starting point, tuned against your own write throughput.

It's an inelegant architecture. It's also the one that works.

Pagination: use cursors

Offset pagination breaks on any feed receiving new items at the top. By the time a user requests page 2, new posts have shifted the window, and OFFSET 20 returns items they already saw.

Cursor pagination — pass the score of the last item seen, fetch strictly older entries — fixes it. This bug is easy to miss in testing because it only appears when content arrives during a session, which staging environments rarely simulate.

The adjacent failure: media uploads

Feed architecture discussions usually skip this, but for image- and video-heavy feeds the upload path fails before the feed does.

If uploads route through your API server, every upload occupies a request handler for the duration of the transfer. Mobile uploads on poor connections hold those handlers for a long time. Enough concurrent uploads and your API starts refusing requests that have nothing to do with media.

Presigned URLs move the transfer off your infrastructure entirely — the client requests a URL, uploads directly to object storage, then notifies your API. Processing goes to a separate worker. This is a small change that removes an entire class of outage.

Perceived latency is a real metric

One technique worth knowing: write the author's own post to their own timeline synchronously, and let the async fan-out handle everyone else.

This changes nothing about system throughput. It eliminates most "my post didn't work" reports, because the author immediately sees their post. Users judge the system by what they can observe, and the author observing their own post is the observation that matters most.

A practical checklist

If you're at the design stage:

• Choose the architecture deliberately. Migrating later is expensive; deciding now costs a conversation.
• Alert on p99, not averages. Tail latency is where feed problems appear first, often weeks before averages move.
• Load test with skewed data. Seed data with uniform follower counts hides every failure mode described here, because all of them live in the tail of the distribution.
• Plan the celebrity case before you have celebrities. The threshold logic is much easier to add before it's urgent.
• Cursor-paginate from day one. Retrofitting is more painful than starting there.

The starting point is still correct

None of this argues against beginning with the naive join. For an early product, it's the right call — it's simple, it's fresh, and premature optimisation here costs you time you should spend on whether anyone wants the product at all.

The argument is just that you should know which failure you'll hit first, and roughly what you'll do about it, before you hit it.

---

Written by the engineering team at Dev Technosys.