Ask a candidate to design Instagram and most of them start with the wrong thing. They draw a phone, an upload button, a database, and a screen full of photos, and they think the problem is photos. The photos are the easy part. Storing a two megabyte image and serving it back through a CDN is a solved problem that a competent team ships in a week. The problem that makes Instagram a real system design question, the problem that decides whether the app opens in eighty milliseconds or in three seconds, is the feed. Deciding which posts appear when a user pulls to refresh, doing it for hundreds of millions of people at once, and doing it without asking a database to scan the posts of everyone they follow on every single open, is where the entire design lives.
This piece works the problem the way I would want a staff engineer to work it in a design review. We start from the requirements and the arithmetic, because the numbers pick the architecture before any component does. Then we build the pieces in the order they actually matter: the high level shape, the two ways to build a feed and why neither one alone survives contact with a celebrity, the hybrid that real systems run, the data model, where the media actually lives, how the social graph is stored, why caching is not an optimization but the product, ranking, consistency, counting, and the failure mode that takes the site down. We finish with the handful of patterns that transfer to every other system you will build. One note on numbers before we start. Every latency, throughput, and storage figure here is an industry-typical estimate chosen to drive the sizing, not a measured metric from any real deployment.
Start with the requirements, not the screens
The visible feature list for Instagram is large, but the part that shapes the architecture is small. A user posts a photo or a short video with a caption. A user follows other users. A user opens the app and sees a feed assembled from the people they follow. A user likes and comments. Everything else, stories, direct messages, explore, reels, search, is a variation or an extension built on those four verbs. If you can serve the home feed at scale, the rest is incremental.
The visible product is broad, but only four verbs shape the architecture, and the non-functional targets underneath them are where the design actually happens.
The design does not live in that feature list. It lives in the non-functional requirements, and they are worth stating out loud before you draw a box. The feed read has to feel instant, because a social app that stutters on open loses the session, so the target is tens of milliseconds at the high percentiles. The system is overwhelmingly read heavy, because people scroll far more than they post, which means the read path gets the engineering budget. Availability matters more than strict consistency for most of the surface, because a feed that is a few seconds stale is fine while a feed that fails to load is not. Media is effectively write-once and read-many and must be durable forever, since users never expect an old photo to vanish. And the whole thing has to hold a social graph and a posting rate that both grow without the feed getting slower. Write those down first and the rest of the design reads as a set of answers to them.
Do the arithmetic first
Before choosing databases or cache tiers, size the system, because the read-to-write ratio and the media volume decide almost everything downstream. Take a service with 500 million daily active users. Suppose 100 million photos and short videos are posted per day. That averages to roughly 1,160 writes per second, and with daily peaks it lands somewhere around 5,000 per second. That is a modest write rate. A handful of well configured database shards absorb it without drama.
Posts per second are modest, feed reads are two orders of magnitude larger, and media dominates storage. Each number points the engineering somewhere specific.
The read side is a different universe. If each active user opens the app and loads their feed around twenty times a day, that is 10 billion feed reads per day, or about 115,000 per second on average and several times that at peak. Every one of those reads wants roughly the fifty most relevant recent posts, immediately. That is the number that forces a precomputed, cached feed rather than a query. The storage math tells the third part of the story. At 100 million media items a day, and storing several resized renditions of each at roughly 1.5 megabytes total per item, media grows by about 150 terabytes every day, which is on the order of 50 petabytes a year and only ever climbs. Post metadata, by contrast, is about a kilobyte per post, so 100 million posts a day is around 100 gigabytes a day, a rounding error next to the media. The follow graph is maybe 100 billion edges at a couple hundred follows per user, which at a handful of bytes each is a few terabytes. Three numbers, three different destinations. Writes are cheap, reads are enormous, and media storage is the cost center.
Bandwidth is the fourth number and it is the one that quietly dominates the bill. If a feed open pulls a screen of images and each is served at even a few hundred kilobytes, 10 billion feed opens a day moves petabytes of image and video bytes out to clients daily. That traffic cannot come from your origin, and it is the reason a CDN is not optional but structural. It also explains why the peak multiplier matters. Average rates are comforting and misleading, because traffic clusters around waking hours and around events, so you size the read tier and the CDN for the peak, several times the average, not for the daily mean. The write path can be sized closer to average because a posting spike is small in absolute terms, but the read tier has to survive the evening when a whole time zone opens the app at once.
The high-level architecture
With the constraints and the numbers named, the shape follows. Clients hit a CDN and a load balancer, then an API gateway that authenticates the request and routes it to the service that owns it. The services are split by concern rather than by screen, so a small number of them handle everything.
Requests fan out from the gateway to services split by concern, each owning its own store, with the heavy work pushed onto an async pipeline.
A user service owns profiles and the follow graph. A post service owns the creation and metadata of posts. A media service owns uploads and the blob store where the actual bytes live. A feed service owns the precomputed timelines and the logic that assembles a feed on read. Behind them sit stores chosen per workload: an object store for media, a sharded key-value or wide-column store for post metadata, a sharded store for the graph, and an in-memory tier that holds the feeds. The two expensive operations, spreading a new post to millions of followers and computing engagement counts, run on an asynchronous pipeline off to the side, driven by a durable log, so neither one ever sits on a user-facing request. The organizing principle is the one that recurs in every system that scales cleanly. Keep the read path short and served from memory, keep services stateless so they scale by adding machines, and push anything slow or fan-heavy onto a queue where a delay of a few seconds costs nothing.
The feed is the whole problem
Here is the question the entire design turns on. When a user opens the app, how do you produce the fifty posts they should see, fast, without reading the recent posts of everyone they follow at request time? There are two classic answers, and understanding why each one fails on its own is the core insight.
Instagram is not a photo app with a feed bolted on. It is a fanout system that happens to show photos.
Fanout on read is cheap to write and slow to read; fanout on write is the reverse. The choice is really about who pays and when.
The first answer is fanout on read, sometimes called the pull model. You store posts once, and when a user opens their feed you query for the recent posts of everyone they follow, merge them, sort them, and return the top slice. Writes are trivial, one insert per post. The problem is the read. A user following a thousand accounts triggers a thousand-way gather-and-merge on every single open, at 115,000 opens a second, and it gets slower as people follow more accounts. The second answer is fanout on write, the push model. When a user posts, you immediately insert the post's id into a precomputed timeline for every one of their followers, held in memory. Reads become trivial, just read your own ready-made list. The cost moves to write time, and for a normal user that cost is fine, a couple hundred inserts per post. The tension is clean. Pull is cheap to write and brutal to read, push is cheap to read and expensive to write, and a read-heavy system wants the reads to be cheap.
The celebrity problem forces a hybrid
Fanout on write looks like the obvious winner for a read-heavy feed, and for most users it is. Then someone with 100 million followers posts a photo, and the push model asks you to perform 100 million timeline inserts for one action. Do that for every celebrity post and the fanout pipeline melts, timelines lag by minutes, and a single popular account can degrade the feed for everyone. This is the moment a rote answer and a considered one diverge.
Normal authors are pushed to follower timelines at write time; high-fanout accounts are pulled at read time and merged, so no single post triggers millions of inserts.
The fix is a hybrid, and it is what large social feeds actually run. Ordinary accounts use fanout on write. When they post, the post id is pushed into the in-memory timelines of their followers, and those followers read a ready list. High-fanout accounts, the celebrities and brands above some follower threshold, are handled with fanout on read. Their posts are not pushed to anyone. Instead, when a user opens their feed, the service reads their precomputed timeline from the push side, then separately fetches the recent posts of the small number of celebrities that user follows, and merges the two sets before ranking. The threshold is a tuning knob, not a constant. The result is that no single post ever triggers tens of millions of writes, while the common case, following mostly normal accounts, stays a cheap read of a precomputed list with a small merge on top.
The feed generation pipeline
Putting the hybrid into motion, a post travels through a specific sequence from creation to appearing on a screen, and each stage exists to keep the previous decision honest.
A post is written once, fanned out asynchronously to normal followers, and merged with a live pull of high-fanout accounts at read time, then ranked.
When a user posts, the post service writes the post and its metadata durably first, so the post exists before anything else happens. It then emits an event onto a durable log. A fanout worker consumes that event, looks up the author's followers, and for a normal author inserts the post id into each follower's timeline list in the in-memory store, capped at a few hundred entries so the lists never grow without bound. For a high-fanout author, the worker does nothing, because those posts are pulled later. On the read side, when a user opens the app, the feed service reads their timeline list, fetches the recent posts of the handful of high-fanout accounts they follow, merges the two, applies ranking, hydrates the post ids into full posts and media URLs from cache, and returns the slice. The write path is a single durable write plus an enqueue and returns in milliseconds. The fanout happens behind the log where a few seconds of lag is invisible. The read path touches memory, not the source-of-truth database, which is what lets it answer in tens of milliseconds under load.
The data model
The data model falls directly out of the access patterns, and it is deliberately denormalized because the read path cannot afford joins. There are a few core entities, each stored in the shape its queries need.
Each entity is stored in the shape its queries want. The timeline is not a table you query, it is a precomputed list you read.
Users are keyed by user id. Posts are keyed by post id and hold the author id, caption, timestamp, and references to the media renditions rather than the bytes themselves. Follows are the interesting one. The relationship is stored as two denormalized adjacency lists per user, a following list and a followers list, both sharded by user id, so you can answer both who does this user follow and who follows this user without a scan or a join. The timeline, or feed, is not a queried table at all. It is a per-user list of post ids held in the in-memory store, populated by fanout and read directly. Post metadata lives in a sharded key-value or wide-column store, partitioned by post id or author id, because the queries are point reads and small ranges, never ad hoc joins across tables. The lesson repeats across the model. Store the data in the shape you read it, accept the duplication that requires, and let the write path pay the cost of keeping the duplicates in sync so the read path never does.
Where the media actually lives
The photos and videos never touch the metadata database, and keeping them out of it is one of the most important decisions in the design. Media is large, immutable once uploaded, and read far more often than written, which is the exact profile an object store is built for.
The client uploads straight to the object store through a presigned URL, processing generates renditions asynchronously, and the CDN serves the bytes. The app servers never carry the payload.
The upload path avoids sending bytes through the application tier. The client asks the media service for a presigned upload URL, then uploads the file directly to the object store, an S3-style system that gives effectively unlimited capacity and eleven-nines durability by replicating each object across machines and facilities. That direct upload keeps hundreds of terabytes a day of payload off the app servers entirely. Once the original lands, an asynchronous processing job generates the renditions the clients need, several resized images or transcoded video bitrates, and writes those back to the object store. Only then does the post become servable. On read, the post metadata carries CDN URLs, and the bytes are served from the CDN edge close to the user, so the object store itself sees only cache-fill traffic and the origin is shielded from the bulk of reads. The store choice is justified by the workload directly. Write-once, read-many, huge, immutable blobs with a durability requirement belong in an object store fronted by a CDN, not in a database and never streamed through your application servers.
Storing the social graph
The follow graph is small in bytes and awkward in shape, and how you store it decides whether fanout and feed reads stay cheap. It is a directed graph of hundreds of billions of edges, and the two questions you ask of it constantly are who follows this user, needed by fanout, and who does this user follow, needed by feed reads and merges.
The graph is stored as denormalized adjacency lists sharded by user id, so both directions are a single-shard lookup, and hot accounts get special handling.
You could reach for a dedicated graph database, but at this scale the access pattern is simpler than a general graph. It is two directional lookups, not arbitrary traversal, so the pragmatic choice is denormalized adjacency lists in a sharded key-value or wide-column store. Each user's followers and followees are stored as lists partitioned by that user's id, so both questions are answered by a single-shard read. The cost is that a follow action writes to two places, the follower's following list and the followee's followers list, which is the same denormalization trade the rest of the design makes. The awkward case is again the celebrity, whose followers list holds tens of millions of entries and must be paged and processed in chunks by fanout rather than loaded whole. Sharding by user id spreads the graph evenly, keeps the common lookups on one partition, and lets the few enormous nodes be handled as a special path instead of forcing the whole store to accommodate them.
Caching is the product, not a layer
Given a read rate two orders of magnitude above the write rate, caching is not something you add at the end. It is the load-bearing structure, and it works because both media popularity and feed access are heavily skewed toward a hot minority.
The CDN absorbs most media bytes and the in-memory caches absorb most post hydrations, so the durable stores only ever see the remainder. The database is the exception path.
There are several cache tiers, each catching what the one before it missed. The CDN sits in front of media and serves the overwhelming majority of image and video bytes from edges near the user, so the object store origin sees a small fraction of media reads. The in-memory timeline store holds the precomputed feeds themselves, so a feed open is a memory read rather than a database query. A separate object cache holds hydrated posts and user profiles, keyed by id, so assembling a feed page pulls fifty posts from memory instead of the metadata store. The pattern throughout is cache-aside with least-recently-used eviction, which suits skewed traffic because it keeps the currently hot posts and accounts resident and lets cold ones fall out on their own. The practical rule is to size each cache to its working set, not its whole dataset. You never cache 50 petabytes of media or every timeline. You cache the hot slice, and because the traffic is skewed, that small slice absorbs most of the reads, leaving the durable stores handling only misses and writes.
The numbers make the point concrete. If the CDN serves 95 percent of media bytes and the object cache serves 90 percent of post hydrations, then the metadata store sees roughly a tenth of the post reads and the object store origin sees a twentieth of the media reads. Stack those hit rates and a request that started as one of 115,000 feed opens a second has a very small chance of ever reaching a source-of-truth database. That is the whole game. The durable stores are sized for writes and cache misses, not for the raw read rate, which is what makes the fleet affordable. It also means cache behavior is a first-class reliability concern, not a performance nicety, because a cold cache after a flush or a failover briefly exposes the databases to traffic they were never provisioned for, which is why warming and gradual rollout matter as much as the steady-state hit rate.
Ranking the feed
A modern feed is not reverse-chronological, and the switch from time-ordered to ranked changes the read path in a way worth being explicit about. Chronological is simple, you sort the merged post ids by timestamp and return them. Ranked means scoring each candidate post for how likely this user is to engage with it, and showing the best first.
Ranking is candidate generation, then feature lookup, then scoring, then ordering. It sits on the read path, so its budget is a few milliseconds.
The ranked path has four stages. Candidate generation gathers the pool of posts eligible for this user, which is exactly the merge of the pushed timeline and the pulled high-fanout posts already described. Feature lookup fetches signals about the user, the author, and each candidate, things like recency, past engagement between the two accounts, and post type, mostly from precomputed feature stores so the lookup is fast. Scoring runs a model over those features to produce a predicted engagement score per candidate. Ordering sorts by score, applies rules like not showing too many posts from one author in a row, and returns the slice. The constraint that shapes all of this is that ranking runs on the read path, inside the tens-of-milliseconds budget, so the heavy model training and most feature computation happen offline, and the online step is a fast scoring of a small candidate set. Ranking improves engagement, but it buys that with complexity and a hard latency budget, which is why the candidate set is kept small before the model ever sees it.
Consistency, chosen per surface
Instagram does not pick one consistency model, it picks one per surface, and being able to say which parts tolerate staleness and which do not is what separates a real answer from a slogan about eventual consistency.
Most of the surface tolerates a few seconds of staleness; a small set of operations needs read-your-writes. The model is chosen per operation, not globally.
The feed is eventually consistent, and that is a deliberate, correct choice. If a post you follow appears a few seconds after it was created because fanout is still in flight, no user notices or cares, and demanding otherwise would force the feed onto a synchronous path it cannot afford. Like and view counts are eventually consistent and often approximate, for the same reason. But a few operations do need read-your-writes consistency, because violating it looks like a bug to the user. When you post, your own profile grid must show the post immediately. When you follow someone, your following list must reflect it right away. When you edit a caption, you must see the edit. The design handles those by serving a user's own writes from the primary or from a session-pinned path so their own actions are always visible to them, while everyone else sees the change propagate at eventual-consistency speed. Match the guarantee to the operation. Spend strong consistency only where its absence is visible as a defect, and let everything else be fast and eventually correct.
Counting at scale
Like counts and view counts look trivial and are one of the sharpest scaling problems in the system, because a single viral post can take tens of thousands of likes per second, and a naive increment of one row turns that row into a global lock that everyone contends on.
A hot counter is sharded into many sub-counters that are incremented independently and summed on read, turning one contended row into many uncontended ones.
You cannot serve a hot count by having every like do an atomic increment against one database row, because that row becomes a single point of contention and the writes serialize. The standard fix is sharded counters. The count for a hot post is split across many sub-counters, each an independent row or key, and each like increments a randomly chosen shard. Reading the total sums the shards, which is cheap and can itself be cached. Writes spread across shards instead of piling onto one, so the contention disappears. For the very hottest posts, likes are often buffered and aggregated asynchronously, incrementing an in-memory counter that is flushed to durable storage periodically, trading a little precision and a little delay for the ability to absorb the spike at all. Displayed counts are eventually consistent by design, which is exactly the property the consistency discussion said the feed could tolerate. The lesson generalizes to any high-frequency counter. Never let a popular item funnel all its writes into one row, shard the count, sum on read, and aggregate the hottest ones out of band.
The failure mode: when a celebrity posts
Every system has a scenario that concentrates its stresses, and for Instagram it is a single extremely popular account posting something that everyone reacts to at once. It is worth walking through because it exercises three separate weak points at the same time.
One celebrity post triggers a fanout surge, a read stampede on a hot key, and a counter hotspot at once. The mitigations were built into the design, not bolted on after.
The moment the post lands, three things happen together. First, if that account were on the push model, fanout would try tens of millions of timeline inserts at once, a write surge that would back up the pipeline and delay everyone's feed, which is exactly why high-fanout accounts are pulled on read instead. Second, the post itself becomes a hot key. Hundreds of thousands of feeds want to hydrate that same post id in the same few seconds, and if it is not cached, they all stampede the metadata store at once, a classic cache stampede. The mitigation is to serve it from the object cache with request coalescing, so a cache miss triggers one fetch that the other waiting requests share rather than one fetch each. Third, the like counter on that post goes hot, which the sharded-counter design already absorbs. The reason this scenario is survivable is that the mitigations are not emergency patches, they are the architecture. Pulling celebrities, caching hot posts with coalescing, and sharding hot counters were each chosen for exactly this moment. A design that handles the average case and not this one is a design that has not been finished.
The patterns that transfer
Every choice above bought something and cost something, and naming both sides is what separates understanding the system from reciting it. Fanout on write bought instant feed reads and cost write amplification and stale-until-propagated timelines. The hybrid bought protection from celebrities and cost a merge step and a threshold to tune. Denormalized adjacency lists bought single-shard lookups and cost double writes on every follow. The object store and CDN bought cheap durable media at scale and cost a multi-step upload and processing path. In each case the design traded generality and strict freshness for predictable speed at volume, which is the right trade for a system whose job is to do a few things billions of times a day.
Strip away the photos and these four decisions are what remain, and they show up in almost every large read-heavy system you will build.
The reason Instagram is worth this much attention is that its lessons are not about photos. Four of them transfer to nearly any large read-heavy system. First, decide who pays for the work and when. Fanout on write moves cost from read time to write time, and in a read-heavy system that is almost always the right direction, but only until a high-fanout node forces a hybrid. Second, denormalize for the read path and let writes keep the copies in sync, because a system that reads far more than it writes should make reads cheap even if writes get more expensive. Third, store each kind of data in the system built for its shape, blobs in an object store behind a CDN, hot lists in memory, point-lookup metadata in a sharded key-value store, rather than forcing one database to do all of it. Fourth, choose consistency per operation, spending strong guarantees only where their absence is a visible bug and letting the rest be fast and eventually correct. Learn to see which of these levers your own bottleneck sits on, and the design stops being a memorized answer and becomes a method.
I teach system design this way, from first principles with real diagrams and the trade-offs that only surface in production, as an interactive course at systemdesign.academy. The foundation lessons are free and need no signup.
Read the free lessons: https://systemdesign.academy
Top comments (0)