DEV Community

Cover image for Design a search ranking system: the system design interview walkthrough
Gowtham Potureddi
Gowtham Potureddi

Posted on

Design a search ranking system: the system design interview walkthrough

Design a search ranking system: the system design interview walkthrough

The request to design a search ranking system — or the recommendation ranker that decides what plays next — sounds like one machine-learning model with a score() method. Feed it a query and an item, get a relevance number, sort. The trap is scale: your catalog has millions of items, a good ranking model is far too expensive to run against all of them per request, and the latency budget is a couple hundred milliseconds. You cannot score a million items with a deep model and still answer in time — so the real design is not a model, it is a funnel that cheaply narrows millions of items to a few hundred and then spends its expensive model only on those. This is one of the richer system design interview prompts because the obvious single-model answer is computationally impossible, and the interesting answer is the two-stage retrieval-then-rank shape that every production search and recommendation system shares.

The round turns on three decisions. First, the multi-stage funnel — why you split the work into a fast, coarse retrieval step and a slow, precise ranking step. Second, candidate generation — how retrieval narrows the catalog, using lexical matching and semantic embeddings with approximate nearest-neighbour search. Third, the ranking model and its signals — how the model scores the survivors using rich features from a feature store, and how it learns from the clicks and watches users leave behind, despite the biases in that data. This walkthrough follows the path a strong candidate draws on the whiteboard — requirements, estimate, the offline/online split, high-level design, then the deep dives — and at each step it names the follow-up the interviewer uses to tell a memorised answer from an engineer who understands the tradeoff.

Title card for a system design interview walkthrough on designing a search ranking system, showing a funnel that narrows millions of catalog items through retrieval to a few hundred candidates and then an expensive ranking model to a final ordered list, on a dark citrus-and-lavender background.

How the search ranking round is actually run

A search ranking round runs about 60 minutes on a shared canvas — you draw the boxes and arrows while you talk, and the interviewer interrupts. It is a favourite discovery problem because the one-line brief ("rank results for a query") hides the single most important pattern in applied ML systems: the retrieval-then-ranking funnel, split across an offline world that builds models and indices and an online world that serves each request in milliseconds. Get that two-stage, two-environment shape clean — and show you know why you cannot run one model over the whole catalog — and you have shown the backbone of every recommender and search engine in one problem.

The interviewer rarely lets "train a ranking model and score the items" stand. "Your catalog has ten million items and your ranking model takes a few milliseconds per item — that is hours per query, not milliseconds. So what does the model actually score?" That question — you cannot rank everything, so what do you even feed the ranker — is the one a copied answer can never rehearse, and the one this round is really testing.

The rounds here are modelled on how the loop runs, not scraped from any candidate's recall. What matters for prep is the shape: a blank canvas, a clock, and a follow-up after every decision you make.

The five-step framework for any HLD round

Every high-level design answer fits the same spine. Say it out loud at the start so the interviewer knows you have a plan, then walk it:

  1. Clarify requirements — what is ranked for whom, the catalog size, the latency budget, and what "good" means.
  2. Estimate — catalog size, queries per second, and the ranking model's per-item cost, to justify the funnel.
  3. High-level design — the offline/online split and the retrieval → ranking → re-rank pipeline.
  4. Deep dive — the two or three decisions that actually decide this system.
  5. Follow-ups — bottlenecks, failure modes, and what you would change at ten times the scale.

What the debrief measures

The score is not "right or wrong design." A good HLD debrief reads on four dimensions, and every move below maps to one: problem solving (did you reach for a retrieval funnel instead of scoring the whole catalog), communication (did you explain the offline/online split and why retrieval trades precision for speed, in a sentence), technical execution (ANN-based candidate generation, a learning-to-rank model over a feature store, implicit click labels with position-bias correction), and time management (did you spend your 60 minutes on the funnel, features and signals, not on the model's hyperparameters). Reading a walkthrough teaches the design; it does not teach you to deliver it on the clock while someone pushes back.

Step 1 — Requirements: what a search ranking system must do

Spend the first few minutes here. Jumping to boxes before you have scoped the problem is the fastest way to design the wrong system.

Type Requirement
Functional Given a query (or a user context, for recommendations), return an ordered list of the most relevant items from a large catalog; personalise with user history; refresh as the catalog and user behaviour change; log interactions (clicks, watches) as training signal.
Functional (optional) Filters and business rules, diversity/freshness re-ranking, exploration of new items, explanations — call these out, then scope most of them for later.
Non-functional Low-latency serving (~100–200ms) over a million-plus item catalog; the ranking model is expensive per item, so it cannot run on the whole catalog; relevance measured by engagement (CTR, watch time) and ranking metrics (NDCG); offline training vs online serving must not skew; scalable and highly available.

The sentence that earns points: "A good ranker is too expensive to run on the whole catalog in the latency budget, so the design is a two-stage funnel — cheap, wide retrieval narrows millions of items to a few hundred, then an expensive model ranks only those — split across an offline world that builds the models and indices and an online world that serves each request." Say that and you have framed the entire design before drawing a box.

Step 2 — Capacity estimation (back of the envelope)

Estimation here is about proving one thing: the ranking model's per-item cost times the catalog size blows the latency budget, which is exactly what forces the funnel.

Quantity Assumption Result
Catalog ~millions of items too many to score individually per request
Ranking cost a few ms per item (rich model) millions × ms = minutes to hours per query → impossible to rank all
Retrieval narrows to ~hundreds of candidates a ~99.99% cut → ranking now scores hundreds in a few ms total
Latency ~100–200ms end to end retrieval (ANN, cheap) + ranking (hundreds of items) must both fit
Signals clicks/watches per impression huge implicit-feedback stream → training data, with biases

Two conclusions fall out, and you should say both out loud. First, you cannot rank the whole catalog — the expensive model's cost per item makes a full scan take minutes, so a cheap retrieval step must cut millions to hundreds before the ranker runs; the funnel is not an optimisation, it is the only way the latency budget is met. Second, the work splits offline and online — building embeddings, the ANN index, the models and the feature store happens offline in batch, and each request flows online through retrieval and ranking using those precomputed artifacts. You have now pointed the interview at the funnel and the offline/online split — the real deep dives — before anyone asked.

Step 3 — API and data model

The interface is a single ranked-results call; the design lives in the offline artifacts and the online funnel behind it.

Call Operation
search(query / user, context) → ranked_items[] online: retrieve candidates, augment with features, rank, re-rank, return the ordered list
logInteraction(user, item, action, position) record a click/watch (and its displayed position) as training signal
buildIndex() / trainModel() (offline) batch jobs: embed the catalog, build the ANN index, train the ranking model, populate the feature store

The data model spans the catalog, the retrieval artifacts, the ranking features, and the interaction log:

Entity Notes
item catalog entry with attributes; embedded offline into a vector for semantic retrieval and indexed lexically.
ann_index an approximate-nearest-neighbour index over item embeddings, built offline, queried online to retrieve similar items fast.
feature_store item, user and context features, served online for ranking and read offline for training — the same store both sides to avoid train-serve skew.
interaction user, item, action (click/watch), position, ts — the implicit-feedback stream that becomes labels.

Step 4 — High-level design: the offline/online, retrieval/ranking pipeline

Now the boxes. Every production search and recommendation system is the same 2×2 — offline vs online, retrieval vs ranking:

  • Offline (build artifacts): batch jobs embed the catalog, build the ANN index over those embeddings, train the ranking model, and load item/user features into the feature store. These artifacts are published to the online world.
  • Online — retrieval: a request (a query or a user context) is turned into an embedding; the retrieval step uses the ANN index (and a lexical inverted index) to pull a few hundred candidates from the million-item catalog — fast and coarse.
  • Online — ranking: the candidates are augmented with features from the feature store (item, user, context, cross features), and the ranking model scores each, producing a precise order.
  • Re-ranking: an optional final pass applies diversity, freshness, dedup and business rules before returning the list.
  • Feedback loop: the user's clicks and watches are logged and flow back into the offline training data, so the model keeps learning.

Say explicitly that artifacts flow offline → online (models, ANN index, feature store are built in batch and loaded for serving) while each request flows left to right (retrieve → rank → re-rank), and that you use the same feature store offline and online so the model trains and serves on identical features. Naming the 2×2 and the shared feature store unprompted is a signal the interviewer is listening for.

A search ranking architecture diagram showing an offline environment that embeds the catalog, builds an ANN index, trains the ranking model and populates a feature store, with those artifacts published to an online environment where a request is embedded, retrieval pulls a few hundred candidates from the ANN and lexical indices, the candidates are augmented with features from the shared feature store, a ranking model scores them, a re-ranking pass applies diversity and business rules, and logged clicks and watches flow back into offline training.

Deep dive: the multi-stage ranking funnel

This is the decision that decides the round, because it is why the system has the shape it does. You cannot run an expensive, feature-rich model over a million-item catalog in a latency budget, so you split the work into stages that trade off precision against cost in opposite directions.

Retrieval — fast and coarse. The first stage narrows the catalog from millions to a few hundred candidates — a roughly 99.99% reduction — using cheap methods that sacrifice precision for speed (embedding similarity via ANN, lexical matching, popularity). Its only job is recall: make sure the genuinely relevant items are somewhere in the candidate set, even if the order is rough.

Ranking — slow and precise. The second stage scores those few hundred candidates with an expensive model and rich features that would have been impossible to run on the full catalog. Because it processes hundreds, not millions, it can afford precision: a learning-to-rank or click-probability model that carefully orders the survivors. Many large systems add a re-ranking stage on top for diversity, freshness and business rules — the industry two-stage pattern has grown a filtering and an ordering stage at the biggest shops.

The pattern to name is recall then precision: retrieval optimises for not missing the good items cheaply, ranking optimises for ordering them well expensively, and the two together hit a latency budget neither could alone.

"Your ranking model is great but takes milliseconds per item — how do you ever rank a ten-million-item catalog in under 200ms?" You don't rank the catalog. A cheap retrieval stage (ANN over embeddings + lexical) cuts millions to a few hundred candidates optimising for recall, then the expensive model ranks only those for precision, with an optional re-rank for diversity. Name the funnel, recall-then-precision, and that the ranker never sees the full catalog.

A multi-stage ranking funnel diagram showing a wide top labelled millions of catalog items narrowing through a retrieval stage optimised for recall to a few hundred candidates, then through an expensive ranking stage optimised for precision to a small ordered set, then a re-ranking stage applying diversity and business rules to the final displayed list, with each stage annotated by how many items it handles and whether it is cheap and coarse or expensive and precise.

Deep dive: candidate generation — lexical and semantic retrieval

This is the stage most candidates under-specify, and it is where "did we even find the right items" is decided. Retrieval has to surface the relevant handful from millions, cheaply, and it does so from several complementary sources.

Lexical retrieval. The classic approach matches the query's terms against an inverted index (an ElasticSearch-style index) — fast, exact on keywords, and strong when the query and the item share words. It misses semantic matches ("film" vs "movie") and struggles with vague queries.

Semantic retrieval. The modern approach embeds the query and every catalog item into the same vector space (learned offline) and uses approximate nearest-neighbour (ANN) search to pull the items whose embeddings are closest — catching meaning rather than exact words. The embeddings and ANN index are built offline (Faiss-style); online, you embed the request and query the index in milliseconds.

Multiple sources, merged. Production retrieval usually unions several candidate sources — lexical, semantic/ANN, recently-popular, and (for recommendations) collaborative-filtering or user-history candidates — because each catches items the others miss, and the ranker sorts out the merged set. Retrieval is tuned for recall: a relevant item the retrieval step drops can never be ranked, so it is better to over-retrieve and let ranking prune.

"A user searches 'feel-good films' — a keyword index returns little. How does retrieval still find the right titles?" Semantic retrieval: embed the query and the catalog into one vector space offline and use ANN to pull items close in meaning, not just matching words — unioned with lexical and popularity candidates. Retrieval optimises recall, so you over-retrieve and let ranking prune. Name lexical + semantic/ANN, the embedding space, and multi-source recall.

System Design
HLD · Search & discovery · 60 min
Design a search ranking system out loud, with follow-ups

Practise on Zynter →

Deep dive: the ranking model and learning from clicks

This is the decision that proves you understand the data, not just the architecture. The ranker scores candidates with rich features, and it learns from the messy, biased record of what users clicked — and handling that bias is the senior move.

Features and the model. Once retrieval hands over a few hundred candidates, you augment each with features from the feature store — item attributes, user history and preferences, and context (time, device), plus cross features — and score them. The model is typically learning-to-rank or, more commonly, a classifier predicting the probability of a positive interaction (click, watch, conversion) for each user-item pair, from a gradient-boosted tree or a deep network; you then order by that probability. The critical operational rule: train and serve on the same feature store so the features at training time match the features at serving time — train-serve skew silently wrecks a ranker, and reproducing offline features as of request time (point-in-time correctness) is what prevents it.

Learning from implicit feedback. You rarely have explicit relevance labels, so you learn from implicit signals — clicks, watches, dwell time. The trap is position bias: users click the top result partly because it is on top, not because it is best, so naive training teaches the model to reinforce whatever it already ranked highly. You correct for it with position-aware training (click models, inverse-propensity weighting) and you explore — occasionally show items the model is unsure about — to gather unbiased signal and avoid a feedback loop that only ever learns from its own past rankings. You evaluate offline with ranking metrics (NDCG, MRR) and online with A/B tests on real engagement, because offline metrics and live behaviour diverge.

"You train on what users clicked — but they mostly click whatever you put on top. How do you keep the model from just reinforcing its own rankings?" Correct for position bias (click models / inverse-propensity weighting), add exploration to gather unbiased signal, and keep train and serve on the same feature store with point-in-time features. Validate offline with NDCG and online with A/B tests. Name position bias, the exploration fix, and train-serve parity.

A ranking model and feedback-loop diagram showing a few hundred retrieved candidates augmented with item, user and context features from a shared feature store that serves both offline training and online serving, a learning-to-rank or click-probability model scoring each candidate, logged clicks and watches with their displayed positions flowing back as implicit labels, a position-bias correction and an exploration step feeding the offline training data, and evaluation by NDCG offline and A/B tests online.

System Design
HLD · Search & discovery · 60 min
Defend the funnel, retrieval and ranking signals out loud

Practise on Zynter →

The follow-ups that decide the search ranking round

Scan the probes from each deep dive and they rhyme. Master these five and you are ready for a version of the question that is not drawn exactly like this one.

  • "Ten-million-item catalog, 200ms budget — what does the model score?" → Only the few hundred candidates a cheap retrieval stage surfaces; the ranker never sees the full catalog. (problem solving)
  • "A keyword index misses 'feel-good films' — how do you retrieve?" → Semantic retrieval: embed query and catalog into one space and use ANN, unioned with lexical and popularity candidates, tuned for recall. (technical execution)
  • "How does the ranker learn what's relevant?" → From implicit clicks/watches as labels, with a learning-to-rank or click-probability model over feature-store features. (communication)
  • "Users click the top result because it's on top — isn't your training biased?" → Yes; correct for position bias (click models / IPW) and add exploration so the model doesn't just reinforce itself. (problem solving)
  • "Why might a model that looks great offline fail live?" → Train-serve skew and offline/online metric divergence; keep one feature store with point-in-time features and validate with A/B tests. (technical execution)

Each maps to a debrief dimension. Reading this list teaches you the answers; it does not teach you to deliver them while someone interrupts — that only comes from saying them out loud.

How to prepare: a one-week plan

You do not need fifty systems. You need this one cold and the framework automatic, so the next "design X" is just new nouns in the same five steps.

  • Days 1–2: Learn the five-step framework until it is reflex. Do the search ranking system end to end on paper — requirements, estimate, the offline/online split, the retrieval → ranking → re-rank pipeline.
  • Days 3–4: Drill the funnel and retrieval: recall-then-precision, why you can't rank the catalog, lexical vs semantic/ANN candidate generation, and multi-source recall. Say each tradeoff in one sentence.
  • Days 5–6: Own the model and signals — learning-to-rank vs click-probability, the shared feature store and train-serve skew, implicit labels and position bias, exploration, and NDCG/A-B evaluation. Answer each of the five follow-ups out loud.
  • Day 7: A full timed 60-minute round, narrating the whole way, with someone (or something) interrupting. This is the rep that counts.

This post is the deep walkthrough behind one row of the broader Top 25 system design interview questions — once the ranking funnel is automatic, the same spine carries the rest of the list.

Frequently asked questions

How do I design a search ranking system step by step?

Follow the five-step HLD framework. Clarify requirements (rank the most relevant items from a large catalog for a query or user, personalise, log interactions — ~100–200ms latency, expensive per-item model, relevance measured by engagement and NDCG); estimate to show the model's per-item cost times the catalog makes a full scan impossible, which forces a funnel; define the search/log API and the item/ANN-index/feature-store/interaction model; draw the offline/online pipeline (offline builds embeddings, ANN index, model and feature store; online does retrieval → ranking → re-rank); then deep-dive the three decisions that matter — the multi-stage funnel, candidate generation with lexical and semantic retrieval, and the ranking model learning from implicit clicks.

Why is ranking a two-stage (retrieval then ranking) system?

Because a good ranking model is too expensive to run over the whole catalog inside the latency budget. If the model costs a few milliseconds per item and the catalog has millions of items, scoring all of them takes minutes per query — impossible. So you split the work: a fast, coarse retrieval stage narrows millions of items to a few hundred candidates (a roughly 99.99% reduction) using cheap methods like approximate nearest-neighbour search over embeddings and lexical matching, optimising only for recall; then the expensive model ranks just those few hundred, optimising for precision with rich features it could never have afforded on the full catalog. The two stages trade off precision against cost in opposite directions and together meet a budget neither could alone.

How does candidate generation (retrieval) work?

Retrieval surfaces a few hundred relevant candidates from millions of items, cheaply, usually from several complementary sources. Lexical retrieval matches the query's terms against an inverted index — fast and exact on keywords but blind to meaning. Semantic retrieval embeds the query and every catalog item into the same vector space offline and uses approximate nearest-neighbour (ANN) search online to pull the items closest in meaning, catching matches that share no words. Production systems usually union lexical, semantic, popularity and (for recommendations) collaborative-filtering candidates, because each finds items the others miss. Retrieval is tuned for recall rather than precision: an item the retrieval stage drops can never be ranked, so you over-retrieve and let the downstream ranker prune.

How does the ranking model learn what is relevant?

Mostly from implicit feedback — the clicks, watches and dwell time users leave behind — because explicit relevance labels are rare. The model is usually a learning-to-rank model or, more commonly, a classifier that predicts the probability of a positive interaction for each user-item pair, scored from features in a feature store (item, user and context features). The hard part is bias: users click the top result partly because it is on top, so training naively on clicks teaches the model to reinforce its own past rankings. You correct for position bias with click models or inverse-propensity weighting, add exploration to gather less-biased signal, and keep offline training and online serving on the same feature store with point-in-time features to avoid train-serve skew — then validate offline with NDCG and online with A/B tests, since the two often diverge.

What is train-serve skew and why does it matter for ranking?

Train-serve skew is when the features a model is trained on differ from the features it sees when serving live requests — for example, a feature computed one way in a nightly batch job for training and another way in the online path at serving time, or an aggregate that was recomputed with data that would not have been available at request time. A ranker is especially sensitive to it because small feature differences change the order of close candidates, so a model that looks excellent offline can quietly underperform in production. The standard fix is to compute features once and serve them from a single feature store used by both offline training and online serving, and to reproduce offline features exactly as they were at the moment of the request (point-in-time correctness), so training and serving see identical inputs.

How do I practise a system design interview out loud?

Reading a walkthrough is not the same as defending a design while someone interrupts. On Zynter, the HLD track runs live voice rounds modelled on how real system design loops run — you talk through a shared design on the canvas, and a voice interviewer pushes with the follow-ups above one at a time, then gives you a debrief on problem solving, communication, technical execution, and time management. The score is a practice signal, not a hiring decision.

Practise it on Zynter

You can name a ranking model. The round is won on the funnel and learning from biased clicks.

Pick the Recommendation / Search Ranking round, draw the offline/online retrieval-then-ranking pipeline on the canvas, and defend it out loud to a voice interviewer that keeps asking "what does the model actually score?" and "how do you stop it reinforcing its own rankings?" — then read the debrief. Zynter.ai is on-demand mock interviews modelled on real company loops, with 150+ rounds across 20 companies including the HLD system design track (Sep 2026).

Practise the Ranking round →
Browse all interview tracks

Top comments (0)