Design a recommendation system: the ML system design interview walkthrough

Design a recommendation system: the ML system design interview walkthrough

The request to design a recommendation system sounds like one line — show users videos they will like — and that one line hides the whole interview. Do it for a tiny catalogue and you can score every item for every user; do it for a hundred million items with a 100-millisecond budget, learning from every click and surviving a brand-new user with no history, and it becomes a retrieval-plus-ranking-plus-data-pipeline problem with a dozen sharp edges. This is one of the most common recommendation system system design prompts because it is really a machine learning system design question in disguise: it forces every muscle at once — retrieval, embeddings, feature serving, training data, and the feedback loop.

The round turns on one structural insight. "Show users videos they'll like" sounds like a single model, but you cannot score millions of candidate items per request inside a tight latency budget, so a real recommender is a two-stage pipeline — cheap candidate generation retrieves a few hundred items from the full catalogue, then an expensive ranking model scores just those few. Get that one reframe — it is retrieval then ranking, glued to an offline training pipeline by a feedback loop, not "pick an algorithm" — and the rest of the ML system design interview is deciding how to retrieve cheaply, how to rank well, and how logged interactions become tomorrow's training data. This walkthrough follows the path a strong candidate draws on the whiteboard — requirements, estimate, API and data model, high-level design, then the deep dives — and at each step it names the follow-up the interviewer uses to tell a memorised answer from an engineer who understands the tradeoff.

How the recommendation system round is actually run

A recommendation system round runs about 60 minutes on a shared canvas — you draw the boxes and arrows while you talk, and the interviewer interrupts. It is a favourite because the one-line problem opens onto deep water fast, and because it blends two things most candidates practise separately: the architecture reasoning of a high-level design round and the modelling reasoning of a machine learning round. The moment you draw "user asks, system returns a ranked list," the question becomes how do you avoid touching every item, and every answer after that — embeddings, features, training labels — compounds from the two-stage split.

The interviewer rarely lets "score the items and sort them" stand. "The catalogue is a hundred million items and you have about a hundred milliseconds per request — how do you avoid scoring all of them?" That question — the latency-versus-coverage squeeze — is the one a copied answer can never rehearse, and the one this round is really testing.

The rounds here are modelled on how the loop runs, not scraped from any candidate's recall. What matters for prep is the shape: a blank canvas, a clock, and a follow-up after every decision you make.

The five-step framework for any HLD round

Every high-level design answer fits the same spine. Say it out loud at the start so the interviewer knows you have a plan, then walk it:

1. Clarify requirements — functional and non-functional, and what you are explicitly not building.
2. Estimate — catalogue size, users, request rate and latency budget, and training-data volume, to justify every later choice.
3. High-level design — the boxes and arrows: the serving path, candidate generation, ranking, and the offline training pipeline.
4. Deep dive — the two or three decisions that actually decide this system.
5. Follow-ups — bottlenecks, failure modes, and what you would change at ten times the scale.

What the debrief measures

The score is not "right or wrong design." A good HLD debrief reads on four dimensions, and every move below maps to one: problem solving (did you reach a two-stage retrieve-then-rank pipeline with an offline training loop), communication (did you say why you cannot score the whole catalogue per request in one sentence), technical execution (embeddings and an ANN index, a feature store, logged outcomes as labels, A/B testing and cold start), and time management (did you spend your 60 minutes on candidate generation, ranking, and the feedback loop, not on which exact model architecture to use). Reading a walkthrough teaches the design; it does not teach you to deliver it on the clock while someone pushes back.

Step 1 — Requirements: what a recommendation system must do

Spend the first few minutes here. Jumping to boxes before you have scoped the problem is the fastest way to design the wrong system.

Type	Requirement
Functional	Given a user and a context, return a personalised ranked list of items; serve several surfaces (home rows, "because you watched X", continue-watching, trending); refresh as the user interacts; return results fast enough to render inline.
Functional (optional)	Explicit controls (not-interested, thumbs), diversity and freshness rules, explanations for a row — call these out, then scope most of them for later.
Non-functional	Low latency per request (tens to ~100 ms); freshness (new items and new interactions reflected quickly); scale (hundreds of millions of users and items); measurable quality (offline metrics plus online engagement).

The sentence that earns points: "The hard part isn't choosing a model, it's serving a good ranked list in ~100 ms over a catalogue too large to score per request — so the core of the design is a two-stage retrieve-then-rank pipeline fed by an offline training loop, not a single model." Say that and you have framed the entire design before drawing a box.

Step 2 — Capacity estimation (back of the envelope)

Estimation here is about proving one thing: the hard axis is serving a good list cheaply at scale, which forces the two-stage split and the offline pipeline — not clever model choice.

Quantity	Assumption	Result
Catalogue size	millions to ~100M items	you cannot score them all per request → retrieval must shrink the set first
Users	100M+ active users	precompute per-user candidates offline, or compute them online within budget
Request rate + latency	high QPS, ~100 ms budget	cache candidates and features, retrieve with an ANN index — no full scan
Training data	billions of impression + interaction events/day	a batch pipeline, not online scoring, builds the training set and the embeddings

Two conclusions fall out, and you should say both out loud. First, you must split retrieval from ranking — because the catalogue is too large to score per request, a cheap retrieval step narrows millions of items to a few hundred before any expensive model runs. Second, the heavy work lives offline: embeddings, candidate lists and model training run as batch jobs over logged events, and only the light retrieve-then-rank path runs online inside the latency budget. You have now pointed the interview at the real structure — two systems glued by logs — before anyone asked.

Step 3 — API and data model

The external API is deliberately small: one call returns a ranked list; the two internal stages sit behind it.

Operation	Shape
get_recommendations(user_id, context, k)	the public call: given a user and a context (surface, time, device), return the top k ranked items.
retrieve(user) -> candidates	internal stage one: union several sources into a few hundred candidate items, cheaply.
rank(user, candidates) -> ordered	internal stage two: score the candidates with the ranking model and sort them.

The data model carries the state that both stages read and that the offline pipeline writes:

Entity	Notes
user	profile plus a learned user embedding; the vector the retrieval and ranking stages key on.
item	the catalogue entry plus a learned item embedding and content features (genre, cast, length).
interaction	the log row — (user, item, action, timestamp) where action is impression / click / watch-time; the raw material for training labels.
feature store	precomputed batch features and freshly computed real-time features, keyed by user and item, read at ranking time.

Step 4 — High-level design: candidate generation plus ranking

Now the boxes. A client sends get_recommendations(user, context) to a recommendation service, which runs the two-stage path and returns a ranked list. Around that sit the pieces that make it fast and keep it learning:

• Candidate generation (retrieval): unions several cheap sources — an approximate nearest neighbour (ANN) query over precomputed user/item embeddings, plus heuristic sources like trending, continue-watching and because-you-watched-X — into a few hundred candidates.
• Ranking service: a heavier ML model scores those few hundred candidates using user, item and context features pulled from a feature store, then sorts them into the final list.
• Feature store: serves batch features (precomputed offline) and real-time features (computed online) to the ranking model within the latency budget.
• Logging stream: every impression and interaction is written to a stream — this is the raw data the offline side consumes.
• Offline training pipeline: batch jobs turn logged events into training data, retrain the ranking model, and recompute embeddings and the ANN index; the resulting artefacts are pushed back to the serving path.
• Experimentation layer: new models and retrieval sources ship behind A/B tests that measure real engagement, not just offline metrics.

Say explicitly that this is two systems glued by logs — an online, low-latency serving path (retrieve → rank) and an offline data-and-training path — and that the arrow from logged interactions back to training data is the feedback loop that makes it a recommender at all. Naming that unprompted is a signal the interviewer is listening for.

Deep dive: candidate generation and retrieval

This is the first decision that actually decides the round. The catalogue is too large to score per request, so where do the few hundred candidates come from, and why does that shape everything else?

The core move is to retrieve cheaply before you rank. The dominant source is collaborative filtering expressed as learned embeddings: a user and the items are mapped into the same vector space (often by a two-tower model — one tower for the user, one for items), and similar vectors mean "likely to be watched." At serving time you take the user's vector and ask an approximate nearest neighbour (ANN) index for the closest few hundred item vectors — approximate because an exact nearest-neighbour search over a hundred million vectors cannot fit the budget. Crucially, the embeddings and the ANN index are precomputed in a batch/offline job and refreshed periodically; the online step is just a fast vector lookup.

Retrieval source	What it is for
Embedding / ANN (collaborative filtering)	the personalised core — ANN over learned user/item vectors surfaces items similar to what this user engages with.
Trending / popularity	a cheap, always-available source; strong for new or low-signal users and for freshness.
Continue-watching / recent	surfaces in-progress and recently viewed items directly from the user's own history.
Because-you-watched-X (item-to-item)	items whose embeddings are near a specific thing the user just watched; drives the "more like this" rows.

Real systems union several of these sources and dedupe, so no single retriever has to be perfect — breadth at stage one is cheap, and ranking sorts out the quality at stage two. Retrieval optimises recall (did the good items make the shortlist); precision is ranking's job.

"The catalogue is a hundred million items and you have a hundred milliseconds — how do you avoid scoring all of them?" You don't score them — you retrieve. Precompute user and item embeddings offline, build an ANN index, and at request time take the user's vector and fetch the nearest few hundred items, unioned with cheap sources like trending; only those few hundred reach the ranking model. The expensive model never sees the full catalogue.

System Design
HLD · Search & discovery · 60 min
Design a recommendation system out loud, with follow-ups

Practise on Zynter →

Deep dive: ranking, features and training data

This is the decision most candidates under-think. Retrieval hands you a few hundred candidates; the follow-up is always "how do you order them well, and where does the model's training signal even come from?"

Ranking is a heavier model on a small set. Because stage one already shrank millions of items to a few hundred, you can afford a richer model here — one that scores each candidate for this user, in this context, and sorts. The score is driven by three kinds of feature:

Feature type	Examples
User features	long-term watch history and genres, embedding, demographics, device and session activity.
Item features	genre, cast, length, age, overall popularity, item embedding.
Context features	time of day, day of week, device, the surface being filled, what was just watched.

Those features come from a feature store: heavy, slow-changing features are precomputed in batch and cached, while real-time features (what the user did in the last few minutes) are computed online and merged in at request time — so the model sees fresh signal without recomputing everything per request.

The training label is the hard part, and it comes from the logs. You do not hand-label anything; you learn from logged impressions and their outcomes — did the user click, and how long did they watch. A common objective is expected watch time rather than raw clicks, because click-through alone rewards clickbait. Two biases must be named: position bias (items shown higher get more clicks regardless of quality) and the feedback bias that comes from only ever logging outcomes for items the system already chose to show — which is why the model can keep recommending what it already shows. You evaluate offline with ranking metrics like AUC or NDCG, but the decision to ship is made on online engagement from an A/B test, because offline wins do not always move real behaviour.

"Where does the training label come from, and why might the model keep recommending what it already shows?" The labels are logged outcomes — click and watch-time on the items you displayed — so the model only ever learns from items it already surfaced. That feedback bias, plus position bias, narrows what it shows over time; you counter it by logging impressions (not just clicks), adding exploration, and judging ships on online engagement, not offline AUC alone.

System Design
HLD · Search & discovery · 60 min
Defend ranking, features and training data out loud

Practise on Zynter →

Deep dive: the feedback loop, serving and cold start

Retrieval and ranking decide what to show; this decides whether the system stays fast and keeps learning. The follow-up is always the brand-new user.

Serving hits the budget by precomputing. The online path cannot afford to build everything per request, so candidates and features are precomputed and cached where possible: embeddings and the ANN index are built offline, batch features are refreshed on a schedule, and only the light retrieve-then-rank step with real-time features runs per request.

The feedback loop is the point. Every impression and interaction is logged to a stream, and that stream becomes tomorrow's training data: a batch pipeline turns raw events into labelled examples, the ranking model and embeddings are retrained periodically, and the fresh artefacts are pushed back to serving. That arrow — shown items → logged outcomes → new training data → better model → shown items — is the loop that makes the system a recommender rather than a static ranker.

Cold start is the case the interviewer reaches for. A model trained on history has nothing to go on when there is no history:

• New user (no history): fall back to popularity and trending, use any onboarding signals (genres they picked, country, device), and lean on context; personalise as the first interactions arrive.
• New item (no interactions): lean on content features (genre, cast, metadata, a content-based embedding) so it can be retrieved before it has engagement data; add light exploration so it gets shown enough to gather signal.

Ship behind A/B tests. New retrieval sources, features or models go out to a slice of traffic first, and the decision to roll out is made on real engagement, not on an offline metric that looked good — the same point the ranking deep dive made, now as a release discipline.

"A brand-new user signs up with zero history — what do you show, and how does the system start learning them?" You cannot personalise yet, so you fall back to trending and popularity, use onboarding signals and context, and keep the list diverse. The moment they interact, those impressions and outcomes are logged, flow into the training pipeline, and feed the next retrain — so the feedback loop bootstraps a personal model from the first few clicks.

The follow-ups that decide the recommendation system round

Scan the probes from each deep dive and they rhyme. Master these five and you are ready for a version of the question that is not drawn exactly like this one.

• "Why two stages instead of one model?" → You cannot score a hundred million items per request in ~100 ms, so cheap retrieval shrinks the catalogue to a few hundred and an expensive ranker scores only those. (communication)
• "How do you retrieve candidates without scanning the catalogue?" → Precompute user/item embeddings offline, build an ANN index, and fetch the nearest few hundred vectors at request time, unioned with trending and item-to-item sources. (problem solving)
• "Where does the training label come from?" → Logged impressions and their outcomes — click and watch-time — not hand labels; you optimise something like expected watch time and watch for position and feedback bias. (technical execution)
• "A brand-new user with no history — what do you show?" → Fall back to trending, popularity and onboarding signals, keep it diverse, and let the first logged interactions feed the retrain that personalises them. (problem solving)
• "Your offline AUC went up — do you ship it?" → Not on that alone; ship behind an A/B test and decide on real engagement, because offline metrics do not always move online behaviour. (technical execution)

Each maps to a debrief dimension. Reading this list teaches you the answers; it does not teach you to deliver them while someone interrupts — that only comes from saying them out loud.

How to prepare: a one-week plan

You do not need fifty systems. You need this one cold and the framework automatic, so the next "design X" is just new nouns in the same five steps.

• Days 1–2: Learn the five-step framework until it is reflex. Do the recommendation system end to end on paper — requirements, estimate, API, data model, the two-stage boxes plus the offline pipeline.
• Days 3–4: Drill the retrieval deep dive: embeddings, the two-tower idea, ANN over precomputed vectors, and unioning cheap sources. Say in one sentence why you retrieve before you rank.
• Days 5–6: Own ranking and the loop — user/item/context features from a feature store, labels from logged outcomes, position and feedback bias, offline-versus-online metrics, cold start, and A/B testing. Answer each of the five follow-ups out loud.
• Day 7: A full timed 60-minute round, narrating the whole way, with someone (or something) interrupting. This is the rep that counts.

This post is the deep walkthrough behind one row of the broader Top 25 system design interview questions — once the recommendation system is automatic, the same spine carries the rest of the list.

Frequently asked questions

How do I design a recommendation system step by step?

Follow the five-step HLD framework. Clarify requirements (a personalised ranked list across several surfaces, fast, fresh, measurable); estimate to show the catalogue is too large to score per request and that most work must run offline; define the small API (get_recommendations on the outside, retrieve and rank inside) with a user/item/interaction/feature-store model; draw the two-stage design (a recommendation service running candidate generation with an ANN over precomputed embeddings plus heuristic sources, then a ranking service backed by a feature store, with a logging stream feeding an offline training pipeline and an A/B layer); then deep-dive the decisions that matter — candidate generation, ranking and training data, and the feedback loop with cold start.

Why split a recommendation system into candidate generation and ranking?

Because you cannot score a huge catalogue for every request inside a tight latency budget. Candidate generation is a cheap retrieval step that shrinks millions of items to a few hundred — using an ANN query over precomputed embeddings plus sources like trending and item-to-item — optimising recall. Ranking is then an expensive model that scores only those few hundred with rich user, item and context features, optimising precision. The split lets you be both fast and accurate: broad and cheap first, precise and heavy second.

How does candidate generation use embeddings and ANN?

A model (often a two-tower model) maps users and items into the same vector space so that nearby vectors mean "likely to engage." Those embeddings are computed offline in a batch job and indexed in an approximate nearest neighbour (ANN) structure. At request time the system takes the user's vector and asks the ANN index for the closest few hundred item vectors — approximate because an exact search over a hundred million items would blow the latency budget — and unions that with cheap sources like trending and continue-watching. The online step is just a fast vector lookup, not a catalogue scan.

Where does the training data for the ranking model come from?

From the logs, not from hand labels. Every impression and its outcome — whether the user clicked and how long they watched — is logged to a stream and turned into labelled training examples by a batch pipeline. A common objective is expected watch time rather than raw clicks, so the model is not rewarded for clickbait. You must account for position bias (higher-placed items get more clicks) and feedback bias (you only see outcomes for items the system already showed), which is why real systems log impressions, add exploration, and judge ships on online engagement rather than offline metrics alone.

How do you handle cold start for a new user or new item?

For a brand-new user with no history you cannot personalise yet, so you fall back to popularity and trending, use any onboarding signals (chosen genres, country, device) and context, and keep the list diverse; the first logged interactions then flow into the training pipeline and bootstrap a personal model. For a new item with no interactions you lean on content features — genre, cast, metadata, a content-based embedding — so it can be retrieved before it has engagement data, plus a little exploration so it gets shown enough to gather signal.

How do I practise an ML system design interview out loud?

Reading a walkthrough is not the same as defending a design while someone interrupts. On Zynter, the HLD track runs live voice rounds modelled on how real system design loops run — you talk through a shared design on the canvas, and a voice interviewer pushes with the follow-ups above one at a time, then gives you a debrief on problem solving, communication, technical execution, and time management. The score is a practice signal, not a hiring decision.

Practise it on Zynter

You can read every stage. The round is won on the two-stage split and the feedback loop.

Pick the Recommendation System round, draw the retrieve-then-rank pipeline and the offline training loop on the canvas, and defend it out loud to a voice interviewer that keeps asking "a hundred million items, a hundred milliseconds — how do you avoid scoring them all?" and "a brand-new user with zero history — what do you show?" — then read the debrief. Zynter.ai is on-demand mock interviews modelled on real company loops, with 150+ rounds across 20 companies including the HLD system design track (Sep 2026).

Practise the Recommendation System round →
Browse all interview tracks