DEV Community: Feng Zhang

DoorDash Data Scientist Interview Cheatsheet 2026

Feng Zhang — Wed, 22 Jul 2026 14:26:03 +0000

DoorDash Data Scientist interviews usually test two things at the same time: can you write clean analytical code, and can you reason about a messy three-sided marketplace without acting like the data is cleaner than it is.

If you are preparing for the 2026 cycle, the full DoorDash Data Scientist interview prep cheatsheet on PracHub is a useful reference. This post condenses the main patterns into a standalone guide you can use while studying.

The core areas are:

SQL and Python data manipulation
Window functions
A/B testing
Causal inference
Marketplace metric design
Delivery quality diagnostics
Switchback experiments
Unit economics, pricing, and power analysis

The main theme across all of them: DoorDash questions are rarely pure syntax questions. They are business analytics questions where the hard part is often defining the metric correctly.

SQL: treat business definitions as part of the problem

DoorDash SQL questions usually involve orders, customers, restaurants, Dashers, events, cities, and time. You may be asked to calculate monthly spend, identify top restaurants, compare customer behavior across periods, or derive delivery-quality metrics.

The interviewer is checking whether you can turn vague business language into correct query logic.

Before writing code, clarify:

What counts as a completed order?
Which timestamp should be used: created, confirmed, picked up, delivered, canceled?
What timezone should reporting use?
Are canceled or refunded orders included?
Is the metric order-level, customer-level, restaurant-level, or market-level?

Common SQL patterns include:

DATE_TRUNC('month', created_at)

for monthly metrics,

SUM(CASE WHEN condition THEN 1 ELSE 0 END) * 1.0 / COUNT(*)

for rates, and ROW_NUMBER, RANK, or DENSE_RANK for top-N queries.

The mistake I see most often is row inflation after joins. If you join an order table to item-level or event-level data, COUNT(*) may no longer mean number of orders. Use COUNT(DISTINCT order_id) when the grain requires it.

Also be careful with LEFT JOIN. A filter in the WHERE clause on the right-side table can turn it into an inner join and remove customers, restaurants, or cities with zero activity.

Window functions: know the grain before ranking or comparing

Window functions come up often because marketplace data is sequential. Customers reorder, Dashers receive dispatches, orders move through statuses, and restaurants have performance trends over time.

You should be comfortable with:

LAG(value) OVER (
  PARTITION BY customer_id
  ORDER BY order_ts
)

for prior-order comparisons,

LEAD(status) OVER (...)

for what happened after an event, and

ROW_NUMBER() OVER (
  PARTITION BY city_id
  ORDER BY orders DESC, restaurant_id
)

for deterministic top-N logic.

A good answer explains tie behavior. If the interviewer asks for one top restaurant per city, RANK() may return multiple rows. ROW_NUMBER() with a stable tie-breaker is safer.

For rolling or cumulative metrics, distinguish row-based windows from calendar-based windows. A "7-day rolling order count" is not the same as "last 7 rows." If the table has missing days, that difference matters.

A reliable workflow is:

Aggregate to the right grain first, such as restaurant-day or customer-month.
Apply window functions after that.
Filter top-N results in an outer query or CTE.

That prevents mixing raw order rows with grouped metrics, which can duplicate revenue or distort rates.

A/B testing: user-level randomization is often wrong

Many DoorDash experimentation questions are traps for people who default to "split users 50/50 and compare conversion."

That can work for independent consumer UI changes. It can fail badly for marketplace interventions.

If a change affects dispatch, batching, fees, delivery radius, Dasher incentives, or restaurant availability, one treated unit can affect untreated units. A treated order can change Dasher supply, merchant wait time, ETA accuracy, and customer experience nearby.

That is interference, and it violates the usual assumption that one unit's treatment does not affect another unit's outcome.

For marketplace changes, consider:

Geo-level randomization
Market-level randomization
Zone-level randomization
Time-block randomization
Geo-time switchback experiments

A switchback design might randomize treatment by zone × time_block, such as 2-hour or 4-hour blocks. This is useful when market conditions differ across cities or zones, but the policy can alternate over time.

A common analysis equation is:

$$Y_{g,t}=\alpha_g+\gamma_t+\beta T_{g,t}+\epsilon_{g,t}$$

where geography fixed effects control for persistent market differences, time fixed effects control for common time shocks, and treatment estimates the policy effect.

Use clustered standard errors at the assignment level. If treatment was assigned by zone-hour, raw order count alone does not tell you the true amount of independent evidence.

Power: millions of orders can still be underpowered

Clustered experiments reduce effective sample size. If you randomize across 10 markets, you do not magically have millions of independent experimental units just because millions of orders occurred.

A simple design effect is:

$$DE = 1 + (m-1)\rho$$

where m is cluster size and ρ is intra-cluster correlation.

That means power depends on the number of independent clusters and the variance at that level. For switchbacks, use historical variance at the market-time or zone-time level.

For a two-sample mean comparison, a rough sample size formula is:

$$n \approx \frac{2\sigma^2(z_{1-\alpha/2}+z_{1-\beta})^2}{\delta^2}$$

For proportions, replace variance with p(1-p).

In an interview, you do not need to compute every value perfectly. You do need to say what minimum detectable effect matters, what variance estimate you would use, and how clustering changes the calculation.

Metrics: pick one decision metric and guardrails

DoorDash experiments often involve tradeoffs. A batching algorithm may raise orders per Dasher hour but hurt food quality. A promotion may improve D7 retention but increase refunds. A dispatch change may reduce delivery time while lowering Dasher earnings.

A strong metric setup has:

One primary decision metric
Guardrail metrics
Diagnostic metrics
Pre-planned segments

For logistics changes, primary metrics might include delivery time, on-time rate, or contribution profit per order. Guardrails might include cancellation rate, merchant wait time, Dasher idle time, customer rating, refund rate, or reorder rate.

Segments matter because effects can differ by market density, peak versus off-peak, new versus existing customers, restaurant category, distance band, Dasher mode, and supply-demand balance.

Do not claim success from an aggregate lift if suburban zones degrade while dense urban zones improve. That is exactly the kind of marketplace reasoning interviewers expect.

Causal inference: start with the estimand

Causal inference questions usually start with a metric change or a proposed rollout: late deliveries increased, a merchant variety launch changed retention, a new incentive changed Dasher behavior.

Do not jump straight to a model. Start by defining the causal question.

Are you estimating:

Average treatment effect?
Treatment effect on treated?
Segment-specific effect?
Local effect for compliers?

Then choose the method.

Randomized experiments are best when feasible. If not, DoorDash-style questions often point toward difference-in-differences, matched controls, synthetic control, or staggered rollout analysis.

For difference-in-differences:

$$\hat{\tau} = (Y_{treated,post} - Y_{treated,pre}) - (Y_{control,post} - Y_{control,pre})$$

The key assumption is parallel trends. You should say you would check pre-period trends, run placebo tests, and compare against matched markets.

Regression adjustment can improve precision, but it does not remove bias by itself. Use pre-treatment covariates only. Controlling for post-treatment variables, such as actual delivery time, can block part of the treatment effect you are trying to measure.

Example: diagnosing late deliveries

A good answer starts with definitions.

"Late" could mean late relative to quoted ETA, promised delivery window, or food-ready time. The business may care about order-level lateness, customer perception, merchant SLA, or retention.

Then decompose total delivery time:

Merchant prep time
Dasher assignment latency
Travel to merchant
Pickup wait
Travel to customer
Batching delay

Compare affected and unaffected markets, dayparts, merchant types, weather conditions, and distance bands. Once you have a leading hypothesis, choose the design.

If prep-time estimates look wrong, test merchant-facing prep prompts or adjusted buffers. If Dasher supply is the issue, a zone-hour switchback test of incentives may fit better than customer-level randomization.

The final recommendation should tie back to launch criteria: reduce late rate without hurting Dasher earnings, merchant wait, cancellations, refunds, or contribution margin.

What to practice

For SQL, practice joins, conditional aggregation, date grouping, ranking, rolling metrics, and percentiles. For experimentation, practice explaining why user-level randomization may fail and how switchbacks work. For causal inference, practice turning a vague metric movement into a clear estimand, method, and decision.

You can find related DoorDash-style and marketplace analytics prompts in the PracHub interview questions library.

If you want the full structured version with topic-by-topic practice cards, use the DoorDash Data Scientist interview prep cheatsheet as your study checklist.

Supervised ML Fundamentals, Evaluation And Feature Engineering Explained — Tech Interview Concept (2026)

Feng Zhang — Wed, 15 Jul 2026 14:25:45 +0000

Supervised ML interview questions usually are not about reciting definitions. The better signal is whether you can pick a model, design the evaluation, explain tradeoffs, and tie the choice to business cost.

If you want the original interview-focused version, PracHub has a concise concept page on supervised ML fundamentals, evaluation, and feature engineering. This post rewrites the same ideas as a practical guide for developer and data science readers.

Start with the decision context

Before comparing algorithms, ask what the model output is used for.

Are you optimizing:

Probability accuracy?
Ranking quality?
Forecast accuracy?
A hard business action, such as block, approve, alert, or reorder?
Interpretability for product, operations, or compliance teams?
Low-latency scoring?

That question changes the model and the evaluation.

A fraud model with a rare positive class should not be judged by raw accuracy. A demand forecast should not use random cross-validation if time order matters. A conversion model used for bidding may need calibrated probabilities, while a search relevance model may care more about ranking metrics.

A good interview answer often starts with: "I would first clarify whether we need calibrated probabilities, ranking quality, forecast accuracy, or an action threshold tied to business cost."

Linear and logistic regression are still worth knowing well

Linear regression estimates coefficients by minimizing squared error:

$$
\min_\beta \sum_i (y_i - x_i^\top\beta)^2
$$

The classic assumptions are linearity, independent errors, homoscedasticity, no perfect multicollinearity, and exogeneity:

$$
E[\epsilon \mid X]=0
$$

Breaking these assumptions does not automatically make predictions useless. It does affect inference, confidence intervals, and how much you should trust coefficient interpretation.

Logistic regression models class probability as:

$$
P(y=1 \mid x)=\sigma(x^\top\beta)
$$

It minimizes log loss, not squared error. In tabular classification, logistic regression is often a strong baseline, especially with sparse high-dimensional features. It is also easier to explain and often better calibrated than many complex models out of the box.

Do not skip the baseline. If a boosted tree barely beats regularized logistic regression but is harder to explain, slower to score, and poorly calibrated, the simpler model may be the better production choice.

Regularization is about controlling variance

Regularization penalizes model complexity to reduce overfitting.

Common forms:

L2 regularization adds $\lambda|\beta|_2^2$. It shrinks coefficients smoothly and works well with correlated predictors.
L1 regularization adds $\lambda|\beta|_1$. It can set coefficients to zero, which makes it useful for sparse feature selection.
Elastic Net combines L1 and L2. It is useful when groups of correlated predictors are present.
L0 regularization counts nonzero coefficients, $|\beta|_0$. It directly targets subset selection, but exact optimization is usually combinatorial.
L∞ regularization constrains the maximum absolute coefficient. It is less common in applied interviews, but it tests whether you understand norm geometry and constraints.

Feature scaling matters here. If one feature is measured in cents and another in years, a regularization penalty treats their coefficients differently unless you scale the inputs. Scaling is also needed for distance-based models and many gradient-based linear models.

Tree models such as Random Forests, Gradient Boosted Trees, XGBoost, and LightGBM are mostly invariant to monotonic scaling. Transformations can still help with outliers or skewed distributions.

Random Forests vs Gradient Boosting

This is a common interview comparison because both are tree ensembles, but they behave differently.

Random Forests reduce variance through bagging. They train many decorrelated trees on bootstrap samples and random subsets of predictors. They are stable, parallelizable, and less sensitive to hyperparameter settings. They are often a safe first model when labels are noisy or you need a benchmark that does not take much tuning.

Gradient Boosting trains trees sequentially. Each new tree fits residuals or gradients from the current model. This tends to reduce bias and often works very well on structured tabular data. XGBoost and LightGBM are common examples.

The tradeoff is tuning. Boosted trees need careful control of:

Learning rate
Number of trees
Max depth
Subsampling
Column sampling
Early stopping

If trees are too deep or boosting rounds run too long, boosted models can overfit label noise.

A solid answer sounds like this:

"Random Forests mainly reduce variance through bagging, while Gradient Boosted Trees reduce bias through sequential additive learning. I would compare them based on predictive performance, sensitivity to noisy data, tuning cost, calibration, and the business metric. Boosted trees may win on PR-AUC or lift, but Random Forests may be easier to tune and less brittle."

That answer is much better than "Gradient Boosting is more accurate" or "Random Forests avoid overfitting."

Class imbalance changes the metric

Accuracy is a trap when positives are rare.

If the positive class rate is 0.5%, a classifier that always predicts negative gets 99.5% accuracy. That number is useless.

For imbalanced classification, consider:

precision
recall
F1
PR-AUC
Top-k lift
Recall at fixed precision
Cost-weighted metrics

ROC-AUC can also mislead when positives are extremely rare. A model can have a strong ROC-AUC while precision is too low for production use.

Sampling and class weights can help training, but they do not replace a good evaluation plan. You still need to choose a decision threshold based on cost.

For example, a fraud model might optimize:

$$
EV(t)=TP(t)\cdot B - FP(t)\cdot C - FN(t)\cdot L
$$

Here, $t$ is the threshold, $B$ is the benefit of catching fraud, $C$ is the cost of a false positive, and $L$ is the loss from a false negative.

The model produces scores or probabilities. The threshold is a business policy layered on top.

Calibration matters when probabilities drive decisions

Ranking quality and probability accuracy are different.

A model can rank cases well but produce poorly calibrated probabilities. That may be fine for a top-k review queue. It is a problem if probabilities feed expected-value calculations, capacity planning, pricing, or downstream decision systems.

Logistic regression is often reasonably calibrated. Boosted trees may need Platt scaling or isotonic regression.

Useful calibration checks include:

Calibration curves
Brier score
Observed vs predicted rates by score bucket

If a bucket of users predicted at 20% conversion actually converts at 8%, that model may still rank users well, but its probabilities are not reliable.

Time-series forecasting needs temporal validation

Forecasting has a different data-generating process from ordinary tabular classification.

Random train/test splits leak future information. They can make a weak model look strong, especially when seasonality, promotions, customer behavior shifts, or inventory effects are present.

Use temporal validation, such as:

A simple holdout from a later time period
Rolling-origin evaluation
Backtesting across several forecast windows

Before using complex models, compare against simple baselines:

Seasonal naive
Moving average
Simple exponential smoothing

Then test models such as ARIMA, Prophet, tree models with lagged variables, or sequence models if the problem calls for them.

Metrics should match the operational decision. For demand forecasting, common choices include WAPE, sMAPE, pinball loss, or a service-level cost tied to stockouts and overstock.

Feature engineering without leakage

Feature engineering should encode signal available at prediction time. That last phrase matters.

Common tabular features include:

Lagged demand
Rolling means
Customer tenure
Frequency counts
Recency
Price bands
Target encodings for categoricals
Missingness indicators
Log-transformed monetary values

The risk is leakage. If you compute a rolling average using data after the prediction timestamp, you have leaked future information. If you use target encoding without proper out-of-fold or time-aware computation, the model may learn the answer indirectly.

For repeated entities, grouped splits may be needed. For time-dependent data, temporal holdouts are safer than random splits. Segment-level error checks can also reveal that a model works overall but fails for a product category, region, or customer group.

How to answer this in an interview

A strong structure is:

Clarify the target and decision use.
Describe the data conditions: size, sparsity, label noise, missingness, class balance, time dependence.
Pick baselines first.
Compare model families conditionally.
Choose metrics tied to business cost.
Discuss calibration, thresholds, and validation design.
Name leakage risks.

For example:

"I would start with regularized logistic regression as a baseline because it is interpretable and often well calibrated. I would compare Random Forests and Gradient Boosted Trees using a temporally held-out set if time matters. Since positives are rare, I would use PR-AUC, recall at fixed precision, and expected cost rather than accuracy. If probabilities drive actions, I would check calibration and apply Platt scaling or isotonic regression if needed."

That answer shows model knowledge, evaluation judgment, and production awareness.

For more interview prompts around these topics, PracHub has a set of technical interview practice questions you can use to test your reasoning under pressure.

Common mistakes to avoid

Using accuracy by default.

Accuracy is often the wrong metric for imbalanced classification.

Comparing models without conditions.

"XGBoost is better" is not enough. Better for what data, what metric, what latency limit, and what decision?

Ignoring validation design.

Random splits can leak information in time-series and repeated-entity problems.

Forgetting calibration.

A high-ranking model may still produce bad probabilities.

Treating feature engineering as harmless.

Rolling features, target encodings, and aggregations can leak future or label information if computed incorrectly.

If you want the original concept-card version with the compact interview framing, see PracHub's post on supervised ML fundamentals, evaluation, and feature engineering.

Amazon Data Scientist Interview Cheatsheet 2026

Feng Zhang — Wed, 08 Jul 2026 14:25:57 +0000

Amazon Data Scientist interviews test whether you can turn messy data into a product decision you can defend. You need SQL or pandas fluency, experiment design, metric judgment, ML fundamentals, and clear communication when the prompt is underspecified.

This post is a condensed, blog-friendly version of the PracHub Amazon Data Scientist interview cheatsheet, focused on the parts that tend to separate candidates who have done real analysis from candidates who only know textbook definitions.

What Amazon is usually testing

The interview loop can cover several areas:

Data manipulation with SQL and Python
Product analytics and root-cause analysis
A/B testing and statistical inference
Machine learning fundamentals and evaluation
Applied ML system thinking, including RAG
Behavioral answers mapped to Amazon Leadership Principles

The bar is not "can you recite a formula?" It is closer to: can you define the right metric, compute it correctly, explain your assumptions, catch validity problems, and recommend an action?

Python and pandas: show that your metrics are trustworthy

For pandas questions, interviewers want analysis-grade data manipulation. Expect messy inputs, duplicate rows, mixed date formats, joins across tables, missing values, and business logic hidden in the prompt.

Common pattern:

df.groupby(keys).agg(
    revenue=("revenue", "sum"),
    orders=("order_id", "nunique"),
    customers=("customer_id", "nunique")
)

That code is easy. The interview is about whether the aggregation is valid.

Before grouping, ask:

What is the row grain?
Are there duplicate events or repeated order lines?
Are timestamps in the same timezone?
Are we joining many-to-one, one-to-many, or many-to-many?
Do we need currency normalization before summing revenue?
How should ties be handled in rankings?

A good answer sounds like this:

"I would first validate the grain. If this table is order-line level, I should not count rows as orders. I would count distinct order IDs, normalize currency through the exchange-rate table, then aggregate by customer and month."

That shows you know metric code can be syntactically correct and still wrong.

Useful pandas tools to have ready:

pd.to_datetime() for date cleanup
.dt.date, .dt.to_period("M") for time buckets
merge() for lookup joins
drop_duplicates() for business-key dedupe
rank(), sort_values(), cumcount() for within-group ranking
np.where(), pd.cut(), boolean masks for segments

The biggest mistake is aggregating before fixing grain. Once duplicate rows flow into your metric table, every chart and model downstream inherits the error.

A/B testing: do not jump straight to p-values

Amazon-style experimentation questions usually ask you to design or analyze an online controlled experiment. The interviewer is testing causal reasoning, metric design, statistical mechanics, and judgment.

Start with clarification:

What is the randomization unit: user, session, product, request?
What is the primary metric?
What are the guardrails?
Is the assignment stable?
Is the analysis window long enough for delayed outcomes?
Are users counted once or multiple times?

For a binary metric, such as conversion, you may compute:

Control rate: x_c / n_c
Treatment rate: x_t / n_t
Absolute lift: p_t - p_c
Relative lift: (p_t - p_c) / p_c
Confidence interval for the difference
Two-proportion z-test, if sample size assumptions are reasonable

But a better interview answer does not stop there.

Before inference, check validity:

Sample ratio mismatch
Exposure logging gaps
Duplicate users
Bot or internal traffic
Pre-period balance
Metric denominator consistency
Unit mismatch between randomization and analysis

A weak answer is:

"Treatment conversion is higher and p < 0.05, so launch."

A better answer is:

"I would first verify the planned traffic split and user-level assignment. Then I would estimate absolute and relative lift with a confidence interval. I would launch only if the lift is statistically and practically meaningful, guardrails such as latency and refund rate are neutral, and the result is stable across major pre-specified cohorts."

That last part matters. At Amazon scale, a tiny lift can be statistically significant. It still may not be worth shipping if it harms customer trust, latency, complaints, or downstream outcomes.

For sample size questions, ask for:

Baseline conversion rate
Minimum detectable effect
Alpha
Desired power
Number of variants
Metric type: binary, continuous, ratio-based, clustered

Then define the decision rule before the data arrives. For example:

"Ship if the 95% confidence interval excludes zero, the lower bound is above the practical threshold, and guardrail metrics stay within agreed limits."

Product metrics and root-cause analysis: build the metric tree

Product analytics questions often start with a vague movement:

"Revenue dropped."
"CTR is down."
"Search conversion changed."
"A dashboard metric spiked."

Do not start with random theories. Start by defining the metric and decomposing it.

For revenue, a simple tree might be:

Revenue = Traffic × Conversion Rate × Average Order Value

Then break each branch down:

Traffic:
- visits
- unique customers
- channel mix
- device mix

Conversion:
- product views
- add-to-cart rate
- checkout start rate
- order completion rate

Average Order Value:
- item price
- units per order
- discounts
- shipping or fees

Good root-cause analysis usually follows this flow:

Confirm the metric definition.
Check data quality and pipeline changes.
Compare time windows correctly.
Segment by product, geography, device, customer type, traffic source, or cohort.
Separate correlation from causal claims.
Use visualizations that match the question.

Dashboard tooling can mislead if joins, filters, and aggregation grain are wrong. If a Tableau chart says revenue dropped, you still need to know whether it uses order date or shipment date, whether canceled orders are included, and whether currency conversion happened before or after aggregation.

RAG questions: treat it as an ML system, not "add a vector database"

Retrieval-augmented generation is now fair game for Data Scientist interviews. The DS angle is less about serving infrastructure and more about evaluation, tradeoffs, and risk.

A typical RAG pipeline has:

Document selection
Chunking
Embeddings
Vector retrieval
Optional lexical retrieval such as BM25
Reranking
Prompt construction
Generation
Post-generation validation

If asked to design or evaluate a RAG system, frame the use case first:

"What corpus are we answering from? How fresh is it? What is the cost of a wrong answer? Do we need citations? Are out-of-scope questions common?"

Then split evaluation into retrieval quality and answer quality.

Retrieval metrics:

Recall@k
Precision@k
MRR
nDCG@k

Answer metrics:

Correctness
Faithfulness to retrieved context
Citation accuracy
Coverage
Refusal quality
Helpfulness
Latency and cost per query

A key distinction: an answer can be true but unsupported by the retrieved context. For factual systems, that is still a failure.

Know the RAG vs fine-tuning tradeoff:

Use RAG when facts change often, citations matter, or the corpus is large.
Use fine-tuning for tone, output format, intent classification, or consistent task behavior.
A strong system may use RAG for grounding and fine-tuning for behavior.

For Amazon-like catalogs, policies, seller docs, and support content, hybrid retrieval often makes sense. Dense retrieval handles semantic similarity. BM25 helps with exact product IDs, policy names, rare entities, and error codes. A reranker can improve the final passage list, though it adds latency and cost.

Behavioral answers still matter

For Amazon, technical strength is not enough. Prepare STAR stories tied to Leadership Principles. Your stories should be specific: what you owned, what tradeoff you made, what data you used, what changed because of your work.

Avoid generic claims like "I am customer obsessed." Show the decision.

For example:

"We had a metric discrepancy between two dashboards. I traced it to different order-date logic, aligned the definition with finance, backfilled the metric table, and added validation checks so the issue would not recur."

That kind of story connects ownership, data quality, and business impact.

How to practice

Use timed practice. For each question, force yourself to produce:

Clarifying questions
Assumptions
A clean solution path
Edge cases
A recommendation or decision rule

You can find related prompts in the PracHub interview question bank.

If you want the full topic map, examples, and practice cards for this role, use the Amazon Data Scientist interview cheatsheet on PracHub as your prep checklist.

Product Metric Design And Diagnostic Deep Dives Explained — Tech Interview Concept (2026)

Feng Zhang — Wed, 01 Jul 2026 14:25:38 +0000

A product metric design interview is usually a test of judgment, not memorization. You get an ambiguous product or integrity problem, then you need to turn it into a measurement plan a real team could act on.

The interviewer is checking whether you can keep these separate:

What the product should optimize
What the data can reliably observe
What might be biased, gamed, or misleading

This article adapts the main ideas from PracHub's guide to product metric design and diagnostic investigations, with a focus on how to structure your answer in a Data Scientist interview.

Start with the product goal, not the metric

A weak answer starts with a list:

DAU
posts
clicks
retention
reports

That sounds busy, but it does not explain what success means.

A stronger answer starts by clarifying how the product is supposed to work. For example, if the prompt is "Define success metrics for a Circles feature," you might say:

"I will treat Circles as a community product meant to deepen meaningful interaction among smaller groups. Success should be sustained, high-quality engagement without safety issues or notification fatigue."

That short framing does a lot of work. It tells the interviewer you will not blindly optimize raw activity. A feature can create more posts and still make the product worse if those posts are low quality, spammy, or annoying.

Pick a north-star metric that maps to durable value

A north-star metric should capture product value, not surface activity.

For a community product like Circles, raw joins or raw posts are easy to inflate. Users may join once and never return. Creators may post low-effort content. Spam accounts may create noisy groups.

A better primary metric could be:

weekly active circle members with meaningful two-sided interactions

Then normalize it:

meaningful interactions / eligible circle members

or:

meaningful interactions / eligible impressions

The denominator matters because each version answers a different question.

Per-member metrics ask whether members are getting value. Per-impression metrics ask whether exposed content creates useful engagement. Per-session metrics ask whether Circles changes behavior during active use. Raw counts hide these differences.

For a B2B chat product, the north-star metric might be qualified conversation starts, not total messages. A qualified conversation could require both parties to participate, or require that the conversation passes a basic quality threshold.

Define the unit of value before you define the count.

Build a metric tree

A metric tree helps you avoid treating metric design as a bag of unrelated numbers.

A useful structure is:

Outcome metric
Input metrics
Diagnostic metrics
Guardrails

For B2B chat, that might look like:

Category	Example metrics
Outcome	Qualified conversation starts
Inputs	Response rate, time-to-first-response
Diagnostics	Exposure rate, click-through rate, reply depth
Guardrails	Blocks, spam reports, opt-outs

This structure lets you explain why a metric moved.

If qualified conversations dropped, maybe fewer users saw the entry point. Maybe users clicked but did not send messages. Maybe messages were sent, but businesses stopped responding. Each diagnosis points to a different product issue.

Use guardrails to block bad launches

A positive primary metric does not mean the launch is safe.

Guardrail metrics protect user experience, integrity, and ecosystem health. Common guardrails include:

hide rate
report rate
block rate
unfollow rate
session length
notification opt-outs
harmful-content prevalence
advertiser complaints
support contacts

For Circles, guardrails might include mute rate, leave rate, reports, blocks, notification opt-outs, and displacement from broader feed engagement.

That last one is easy to miss. A feature may increase activity inside Circles while reducing healthy engagement elsewhere. If the new product fragments the social graph or pushes spammy invites, the top-line metric may look good while the broader product gets worse.

Cohort before you trust the average

Averages can hide the real story.

Cut the results by:

new vs existing users
market
device class
language
creator size
business type
group size
spam-risk tier
prior engagement

In a Meta-style interview, you should ask whether gains are broad-based or concentrated in a small segment. For example, Circles may help highly connected users while doing little for new users. A B2B chat change may help large businesses but hurt smaller ones that cannot respond quickly.

This is also where fairness and integrity concerns enter the answer. A harmful-content system that reduces measured prevalence overall may still perform poorly for a language group with weaker labels or lower reviewer coverage.

Match attribution windows to the product mechanism

The time window should match how value appears.

A chat product may need same-day response metrics and 7-day retention. A community product may need 14-day or 28-day return behavior. Harmful-content outcomes may need delayed labels because review, appeals, and classifier updates take time.

A window that is too short misses downstream value. A window that is too long adds noise and confounding.

Say this explicitly in the interview. It shows that you understand measurement as a product decision, not just a query.

Choose the right randomization unit

Experiment design starts with the unit of randomization.

User-level randomization works when the experience is isolated. Networked products are harder. For communities, pages, advertisers, threads, or circles, users interact with each other. One user's treatment can affect another user's experience.

That means you may need community-level, page-level, advertiser-level, or thread-level randomization.

You should also define the estimand:

direct effect
spillover effect
total ecosystem effect

For example, if some Circle members receive a new invite flow and others do not, their behavior may interact. A user-level A/B test may underestimate or distort the effect if treated and control users are in the same groups.

If randomization is not possible, you can propose a retrospective cohort design with matching or difference-in-differences. Keep the caveat clear: observational methods need stronger assumptions about confounding.

Think about power, especially for rare events

Rare events are hard to measure. Spam exposure, harmful-content reports, severe abuse, and appeals may have very low base rates.

A rough minimum detectable effect relationship is:

MDE ≈ (z_alpha/2 + z_beta) * sqrt(2 * sigma^2 / n)

The takeaway is that smaller effects, noisier metrics, and rare events need more data.

For low base-rate outcomes, you can consider:

aggregated exposure units
longer test duration
stratification
higher-signal proxy labels

Do not promise that a short test can detect rare harm reliably. That is exactly the kind of overconfidence interviewers watch for.

Treat proxy metrics with suspicion

Proxy metrics are useful because they are often fast and available. They are also dangerous.

For harmful content, user reports are visible and timely. But reports are biased by user awareness, culture, language, and reporting propensity. More reports could mean more harm, better reporting UX, higher user awareness, or more total usage.

Reports are not ground truth.

A stronger harmful-content evaluation combines:

user reports
human review labels
classifier scores
prevalence estimates
severity-weighted harm metrics

Severity matters. Counting all violations equally treats mild spam and severe abuse as the same kind of event.

A better metric is:

severity-weighted prevalence =
sum(exposures_i * severity_i) / total eligible exposures

The severity buckets should be transparent, and calibration checks should verify that labels are consistent enough to support decisions.

Use a diagnostic funnel for investigations

When a metric moves, avoid guessing. Use a funnel:

exposure -> action -> quality -> retention -> harm

If a product metric drops, ask:

Did fewer users become eligible?
Did fewer users see the surface?
Did fewer users act after exposure?
Did the quality of actions change?
Did retention move?
Did harm or negative feedback move?

This keeps the answer analytical. It also mirrors how real product teams debug launches.

Before interpreting a movement, check measurement validity:

logging coverage
denominator definitions
duplicate events
bot or spam filtering
experiment balance
sample-ratio mismatch
missing labels
metric backfills

You do not need to design the ingestion system in a product metric interview. You do need to know when the measurement is untrustworthy.

A compact answer pattern for interviews

For a metric design prompt, use this flow:

Clarify the product goal.
State assumptions.
Define the primary metric.
Add supporting funnel metrics.
Add guardrails.
Discuss cohorts and denominators.
Explain experiment design.
Name likely diagnostics if the result moves.

For Circles, that might be:

"Success is weekly active circle members with meaningful two-sided interactions, normalized by eligible members. I would support that with circle creation, invite acceptance, posting, comment depth, repeat participation, and 7-day or 28-day retention. Guardrails would include mutes, leaves, reports, blocks, notification opt-outs, and displacement from broader feed engagement. I would run an A/B test if possible, with user-level or circle-level assignment depending on spillovers. I would cut by new users, highly connected users, small markets, and baseline sharing behavior."

That is a defensible answer because it ties metrics to the decision the team needs to make.

If you want more prompts to practice this style, PracHub has a set of data science and product interview questions. For the full concept breakdown, use the original PracHub guide on product metric design and diagnostic investigations.

Amazon Machine Learning Engineer Interview Cheatsheet 2026

Feng Zhang — Wed, 24 Jun 2026 14:26:21 +0000

If you are preparing for an Amazon Machine Learning Engineer interview, expect more than "I know the model." You need to explain how the model works, how you would implement it, what breaks in production, and how you would decide whether it is good enough to ship.

The longer PracHub Amazon Machine Learning Engineer interview prep guide breaks this down by interview stage. This article condenses the highest-signal areas into a study guide you can use before a technical screen or onsite.

What Amazon is likely testing

For an MLE role, Amazon interviewers usually care about four things:

Can you reason from ML theory to working code?
Can you design systems that train, evaluate, and serve models reliably?
Can you debug models using metrics, data, and experiments?
Can you explain tradeoffs around latency, cost, memory, and quality?

A good answer does not stop at "use a Transformer" or "train XGBoost." You should be able to talk through tensor shapes, masks, evaluation gaps, distributed training, sparse data, online metrics, and deployment risk.

Transformers: know the internals, not just the vocabulary

Transformers are one of the highest-yield topics for an Amazon MLE interview. Be ready to explain scaled dot-product attention:

Attention(Q, K, V) = softmax((QK^T / sqrt(d_k)) + M)V

Here, M is often an additive mask. Allowed positions get 0; blocked positions get -inf. The sqrt(d_k) scaling keeps attention logits from getting too large and saturating the softmax.

For implementation questions, shape reasoning matters. Given input X with shape B x T x d_model, multi-head attention projects it into Q, K, and V, then reshapes them into something like:

B x num_heads x T x head_dim

The attention score tensor then has shape:

B x num_heads x T x T

A common bug is reshaping after a transpose without handling non-contiguous tensors. In PyTorch, that means knowing when .view() can break and when .reshape() or .contiguous() is safer.

For decoder-only models, causal masking is mandatory. Token t can only attend to positions <= t. If you forget this, the model can leak future labels during training. The loss may look great, but generation will fail.

You should also know the standard GPT-style block:

x = x + attention(LayerNorm(x))
x = x + MLP(LayerNorm(x))

This pre-norm layout is common because it helps gradient flow in deeper models. Post-norm matches the original Transformer pattern, but can be harder to train at scale.

LayerNorm is another frequent follow-up. It normalizes across the hidden dimension for each token independently:

LN(x) = gamma * (x - mean) / sqrt(variance + epsilon) + beta

Unlike BatchNorm, LayerNorm does not depend on batch statistics. That helps with variable batch sizes, sequence models, and autoregressive inference.

LLMs: connect architecture to operations

For LLM questions, you need to move between model internals and production behavior.

A strong answer covers:

Decoder-only Transformer architecture
Tokenization with BPE, WordPiece, or SentencePiece
Pretraining with next-token prediction
Instruction tuning with prompt-response data
Preference alignment methods such as RLHF or DPO
Fine-tuning choices such as full fine-tuning, LoRA, QLoRA, prefix tuning, and prompt tuning
Evaluation beyond perplexity
Serving constraints such as KV cache memory, throughput, and p99 latency

Perplexity is useful, but it is not enough. It measures next-token likelihood, not whether the model follows instructions, refuses unsafe requests correctly, produces grounded answers, or gives useful task outputs.

For a validation-system design question, structure your answer around:

Evaluation data

Use golden prompts, task-specific benchmarks, adversarial sets, regression cases from past failures, and production-like prompts sampled in a privacy-safe way.
Metrics

Include exact match where it fits, rubric scores, human preference win rate, hallucination or groundedness for RAG, toxicity or safety rates, refusal correctness, latency p50/p95/p99, tokens per second, and cost per request.
System components

Mention a model registry, prompt/version registry, evaluation runner, deterministic inference harness, result store, dashboard, and deployment gates.
Online validation

Use shadow tests, canary rollout, alerts for regressions, drift checks, and rollback criteria.

If the system is RAG-based, model quality depends on more than weights. Retrieval, chunking, embedding quality, ranking, prompt assembly, citation grounding, and index freshness all matter. Good evaluation should include retrieval recall@k, answer faithfulness, source attribution, and latency budget split across retrieval and generation.

MoE: sparse compute has systems costs

Mixture-of-Experts models often replace dense MLP layers with multiple expert networks and a learned router. A token may be sent to the top-1 or top-2 experts.

The benefit is that the model can have more total parameters without activating all of them for every token. The cost is systems complexity.

In an interview, avoid saying "MoE is more efficient" without explaining the tradeoff. Good answers mention:

Load-balancing losses
Expert collapse risk
Capacity factors
Token dropping during overload
Distributed all-to-all communication
Harder batching because routing is data-dependent
Higher risk around p99 latency

Dense models are simpler to serve. MoE models can scale parameter count better relative to FLOPs, but routing and communication make training and serving harder.

XGBoost: understand why it is fast

Amazon MLE interviews may still test classic ML, especially for tabular problems. XGBoost is a common topic because it mixes algorithm knowledge with systems thinking.

Gradient boosting builds an additive model:

y_hat_i^(t) = y_hat_i^(t-1) + eta * f_t(x_i)

Each new tree fits the residual signal, often framed as the negative gradient of the loss. This means boosting rounds are sequential. Tree t depends on predictions from earlier trees.

The parallelism is inside each tree:

across candidate splits
across features
across data partitions
across histogram bins
across workers in distributed training

XGBoost uses second-order information. Split scoring uses gradients and Hessians, with regularization terms such as lambda and gamma. You do not need to derive every line from memory, but you should be able to explain that XGBoost uses both first and second derivatives to score split quality.

For large datasets, exact split search can be expensive. Histogram-based split finding buckets continuous values into quantile bins, often far fewer than the number of raw thresholds. Workers build local histograms of gradient and Hessian sums, then reduce them. This gives better cache behavior and lower memory use, with some loss in split precision.

Also know why sparse handling matters. XGBoost learns a default direction for missing values, which helps with sparse one-hot data and missing feature values.

PyTorch implementation questions: be concrete

For "Implement a decoder-only GPT-style Transformer," start by clarifying scope:

"Should I implement a minimal PyTorch module with embeddings, positional encoding, masked multi-head attention, MLP blocks, and logits, or should I include training and generation too?"

Then state assumptions:

Input token IDs have shape B x T
Vocabulary size is V
Embedding dimension is C
Number of heads divides C
Output logits have shape B x T x V

Talk through token embeddings, positional embeddings or RoPE, stacked pre-norm blocks, causal masking, output projection, and loss.

Call out edge cases:

T exceeds configured context length
mask broadcasting is wrong
train/eval dropout behavior differs
causal mask is missing
.view() is used on a non-contiguous tensor
generation lacks a KV cache

A good implementation answer includes unit tests for shape, causal leakage, and a tiny overfit test to verify the model can learn.

Behavioral answers still need metrics

The source guide groups behavioral preparation under leadership principles, ownership, and measurable impact. For Amazon, that phrasing matters.

Do not give vague stories like "I improved model performance." Give the situation, your decision, the tradeoff, the result, and the metric. For an MLE, strong stories often include model quality, latency, cost, reliability, data quality, rollback decisions, or experiment design.

For example, a better answer sounds like:

"We had a relevance regression after a feature pipeline change. I traced the issue to offline/online feature mismatch, added validation checks before promotion, and reduced bad launches in that area."

The exact numbers depend on your experience, but the structure should make your ownership clear.

Final prep checklist

Before the interview, make sure you can answer these without notes:

Derive and explain scaled dot-product attention
Trace Transformer tensor shapes through multi-head attention
Explain causal masking and label leakage
Compare LayerNorm and BatchNorm
Discuss KV cache memory and autoregressive latency
Explain why perplexity is not enough for LLM evaluation
Design an LLM validation system with offline and online gates
Explain MoE routing and serving tradeoffs
Explain XGBoost histogram split finding and boosting-round dependency
Write a minimal PyTorch Transformer block
Tie every model choice to quality, latency, cost, or reliability

If you want to drill with targeted prompts, the PracHub interview questions library has practice questions across ML theory, system design, coding, and behavioral topics.

For the full role-specific breakdown, use the Amazon Machine Learning Engineer interview prep guide on PracHub as your main checklist.

Notifications And Lifecycle Engagement Explained — Tech Interview Concept (2026)

Feng Zhang — Wed, 17 Jun 2026 14:26:03 +0000

Notifications are easy to measure badly.

If a push campaign gets more clicks, did it create real engagement, or did it interrupt people who were already likely to open the app? If a ranking model lifts CTR, did it improve relevance, or did it learn to send curiosity bait? If dormant users come back today, do they stick around next month, or do they disable notifications?

That is what interviewers are getting at when they ask about notification and lifecycle engagement metrics. The original PracHub concept post on Notifications and Lifecycle Engagement covers the concept, but this article focuses on the answer pattern you can use in a data science or product analytics interview.

What interviewers are really testing

A weak answer sounds like this:

"I would track CTR, opens, DAU, and retention."

That is a metric list, not an evaluation plan.

A stronger answer explains:

What product decision the metric supports
What the primary outcome is
Which metrics are drivers
Which metrics are guardrails
How the experiment estimates incremental impact
How you handle fatigue, delayed outcomes, and selection bias

For a company like Meta, notifications can bring users back and help build habits. They can also annoy users, increase opt-outs, and reduce long-term trust. The interview is testing whether you can separate short-term movement from durable product value.

Start with the product goal

Before naming metrics, clarify the goal.

A notification system may be trying to:

Reactivate dormant users
Improve relevance of pushes
Increase marketplace actions
Reduce notification fatigue
Test a new ranking or sending policy
Personalize volume caps

The right metric depends on the goal. A reactivation system may care about D7_retained_active_users. A marketplace notification may care about listing_detail_views, saves, or seller_messages. A fatigue-reduction project may use disable_push_rate or mute_rate as the primary outcome.

This step matters because it ties measurement to an actual product decision.

Build a metric hierarchy

For notification experiments, organize metrics into three layers.

1. Primary metric

This is the metric you would use to make the launch decision.

Good candidates include:

D7_retained_active_users
D28_retained_active_users
Incremental sessions_per_user
meaningful_sessions
Downstream actions such as messages, purchases, comments, or listing views

The primary metric should capture user or business value, not raw notification volume.

2. Driver metrics

These explain why the primary metric moved.

Common driver metrics:

notification_open_rate
notification_click_through_rate
session_starts_from_notification
notification-attributed_sessions
downstream_conversion

CTR is useful here, but it should rarely be the launch metric.

The formula is simple:

$$CTR = \frac{\text{notification clicks}}{\text{notifications delivered}}$$

The problem is what CTR rewards. It can favor clickbait, curiosity, or over-targeting users who were already active. A notification that gets many clicks may still reduce retention if users feel spammed.

3. Guardrail metrics

Guardrails protect user trust and long-term health.

Use metrics such as:

disable_push_rate
mute_rate
uninstall_rate
hide_notification_rate
negative_feedback_rate
notifications_sent_per_user
complaints
Quality metrics like meaningful_interactions_per_session

A treatment that lifts DAU but also raises push disables may be a bad trade.

Design the experiment around assignment, not clicks

For most notification policies, randomize at the user level.

Control users get the existing policy. Treatment users are eligible for the new notification policy, ranking model, or sending rule.

The treatment should not be defined as "clicked a notification" or "received a notification." Those are post-treatment events. If you analyze only users who clicked, you introduce selection bias because the treatment itself affects who receives, sees, and clicks notifications.

Use intent-to-treat analysis as the primary estimate:

$$ITT = E[Y \mid Z=1] - E[Y \mid Z=0]$$

Here, Z is assignment to treatment.

This estimates the effect of being assigned to the new policy. Some assigned users may never receive a notification during the experiment. That is fine. ITT matches the product decision: should we launch this policy to eligible users?

You can report treatment-on-treated as a secondary diagnostic, but be careful. If exposure is affected by the treatment, exposed-user analysis can be misleading. If needed, use exposure rates or instrumental variables, with clear caveats.

Watch for interference

User-level randomization works well for many notification systems, but social notifications can create spillovers.

Examples:

"Your friend commented"
"Someone tagged you"
"A creator you follow posted"
Marketplace messages tied to listings

One user's treatment can generate messages or activity that affects another user. That violates SUTVA, the assumption that one unit's treatment does not affect another unit's outcome.

In those cases, consider cluster randomization. The cluster could be a conversation, household, creator-follower graph component, marketplace listing neighborhood, or another unit that captures likely spillovers.

Cluster experiments need more sample size because observations inside a cluster are correlated. The design effect is:

$$DE = 1 + (m-1)\rho$$

Here, m is cluster size and rho is the intra-cluster correlation.

Plan for small effects

Retention and opt-out effects can be small, so power matters.

For a two-sample comparison of means, a rough sample size formula is:

$$n \approx \frac{2\sigma^2(z_{1-\alpha/2}+z_{1-\beta})^2}{\delta^2}$$

Here, delta is the minimum detectable effect. For binary metrics, use p(1-p) as the variance.

If you have strong pre-period behavior, use CUPED or regression adjustment to reduce variance. Good covariates include pre-experiment:

sessions_per_user
notification_clicks
active_days

The covariates must be measured before assignment. This improves sensitivity without changing the estimand.

Segment by lifecycle stage

Notification impact is rarely uniform.

Analyze cohorts such as:

New users
Dormant users
Power users
Notification-heavy users
Low-intent users
Users with prior disables or mutes

A policy may help dormant users come back while annoying already-active users. A broad average can hide that pattern.

This is especially relevant for lifecycle engagement. The same push can feel helpful to one user and spammy to another. Segment analysis can support personalization, caps, or targeted rollout instead of a full launch.

Measure beyond the first click

Notifications often have delayed costs.

Common patterns:

CTR rises, but disable_push_rate rises later
DAU increases, but D28_retention falls
Sessions increase, but session quality drops
Short-term reactivation fades after users habituate

Use a window that matches the product goal. If the goal is reactivation, D1 may be too short. D7 or D28 can show whether users came back again. For long-term fatigue, use longer experiments, staggered rollouts, or holdouts.

Control multiple testing

Notification systems have many surfaces, cohorts, and outcomes. If you slice enough, some result will look significant by chance.

A clean answer says you would predefine:

Primary metric
Main guardrails
Evaluation window
High-risk cohorts

For many exploratory slices, use false discovery control such as Benjamini-Hochberg. For guardrails where false positives or false negatives are costly, Bonferroni correction may be more appropriate.

If you want more practice with these interview-style pivots, PracHub has related data science and product interview questions that cover experimentation, metrics, ranking, and causal inference.

Worked example: general notification policy

Suppose the prompt is:

"Define metrics and design experiments for notifications."

A strong answer could sound like this:

First, clarify the goal. Assume we are testing a new push notification ranking policy for a social app. The goal is to increase meaningful engagement and retention without increasing fatigue.

Primary metric:

D7_retained_active_users or incremental sessions_per_user

Driver metrics:

notification_open_rate
notification-attributed_sessions
downstream_actions, such as comments, messages, or shares

Guardrails:

disable_push_rate
mute_rate
uninstall_rate
notifications_sent_per_user
negative_feedback_rate
meaningful_interactions_per_session

Experiment design:

Randomize eligible users into control and treatment
Control keeps the current policy
Treatment is eligible for the new ranking or sending policy
Analyze by assignment using ITT
Avoid conditioning on users who clicked or received a notification
Use cluster randomization if social spillovers are strong

Decision rule:

Ship only if the primary metric improves and guardrails do not show statistically or practically meaningful harm. If the lift is concentrated in dormant users but guardrail harm appears among power users, consider personalization or volume caps instead of a full rollout.

Worked example: similar-listing notifications

Now suppose the prompt is:

"How would you evaluate a similar-listing notification feature?"

The product goal is narrower. You want to know whether notifying users about similar marketplace listings helps them find relevant items without feeling spammed.

Primary metrics could include:

Incremental listing_detail_views
saves
seller_messages
Purchase-intent actions

Guardrails:

notification_disable_rate
hide_rate
Lower engagement with future marketplace notifications

Randomize eligible users who viewed or saved a listing. Do not randomize only people who receive the notification, because eligibility is part of the treatment.

Use an evaluation window that includes delayed actions. A user may click today but message a seller two days later. Segment by intent strength too. Recent searchers may benefit, while casual browsers may find the same push irrelevant.

The common traps

Avoid these mistakes in an interview:

Optimizing only for CTR
Analyzing only users who opened the notification
Listing metrics without a launch rule
Ignoring opt-outs, mutes, and uninstalls
Treating all lifecycle cohorts the same
Missing network spillovers in social notifications
Reading too much into short-term DAU lift

A good interview answer is causal, decision-oriented, and honest about tradeoffs. You are not trying to prove notifications work. You are trying to estimate whether a specific policy creates incremental value without damaging retention or trust.

For a more compact interview-prep version of this framework, use the PracHub write-up on Notifications and Lifecycle Engagement as a reference before practicing mock answers.

Uber Data Scientist Interview Cheatsheet 2026

Feng Zhang — Wed, 10 Jun 2026 14:25:36 +0000

If you're preparing for an Uber Data Scientist interview, the hard part is not memorizing formulas. It is knowing how Uber frames data science problems: marketplace effects, experiment validity, ETA quality, and metric definitions that do not fall apart under edge cases.

This post is a condensed rewrite of PracHub's Uber Data Scientist interview prep guide, focused on the themes that come up in technical screens and onsite rounds.

What Uber is really testing

Across SQL, product analytics, experimentation, and stats, interviewers want to see whether you can:

define the metric correctly
choose the right unit of analysis
avoid leakage and bad denominators
reason about interference in a two-sided marketplace
separate model quality from business impact

That last one matters a lot. Lower prediction error does not automatically mean a better rider experience. A statistically significant A/B test result does not automatically mean "launch."

1) SQL: can you build defensible metrics from messy event data?

Uber SQL questions often look simple at first. Then they turn into deduping events, picking the correct grain, and handling time windows without leaking future information.

Topics that come up often:

Window functions you should be comfortable with

Last or first event per entity

Use ROW_NUMBER() with a deterministic sort:

ROW_NUMBER() OVER (
  PARTITION BY user_id
  ORDER BY event_ts DESC, event_id DESC
)

This is the standard pattern for "latest trip per rider" or "first exposure per user."

Rolling metrics

For time-series summaries, know how to write rolling averages by partition:

AVG(metric) OVER (
  PARTITION BY city
  ORDER BY dt
  ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
)

Top-N logic

You should know when to use RANK, DENSE_RANK, and ROW_NUMBER, and be able to explain tie behavior clearly.

Cohort conversion and CTR

A common failure mode is inflated CTR after joining impressions to clicks. If one impression maps to multiple clicks, COUNT(*) breaks the metric. You need to define the denominator once, dedupe at the right grain, and use explicit attribution windows like click_ts <= impression_ts + interval '48 hours'.

Date spine joins

These matter for rolling averages and anomaly detection. Generate all dates first, then left join events, and fill missing counts with zero.

Timezone-aware aggregation

If you analyze market-level data, local time matters. San Francisco metrics in January should not be cut on raw UTC day boundaries.

Common SQL mistakes

counting rows after a one-to-many join
using future rows in a rolling metric
treating RANK and ROW_NUMBER as interchangeable
skipping timezone conversion before DATE_TRUNC

If you want realistic drills for this style of question, PracHub has a set of data science interview practice questions that match the patterns above.

2) ETA questions: accuracy is only part of the problem

ETA is one of the clearest examples of how Uber expects product sense and statistical judgment to work together.

An interviewer is not looking for "we reduced MAE, so the model is better." They want you to think through:

what the ETA label is
how to evaluate prediction quality
whether the prediction is calibrated
how uncertainty should be measured
what user behavior changes after ETA changes
how interference breaks naive A/B testing

Start with label definition

You need to ask what ETA means in the question.

Is it:

request-to-pickup time?
pickup-to-dropoff time?
total trip duration?

The target has to match the user-facing promise. Cancellations, reassignment, batching, and no-shows all affect the label definition.

Know the evaluation metrics and what they miss

Uber cares about more than one error metric:

MAE is easy to interpret in minutes
RMSE penalizes large misses
median absolute error is more stable with outliers like airports or events
bias tells you whether the model is systematically optimistic or pessimistic

You should also say you would segment results by city, time of day, weather, airport, and trip type.

Calibration matters

If the app says 5 minutes and riders usually wait 7, the model is underestimating. That can increase conversion in the short run and hurt trust later.

Reliability curves by ETA bucket are often more useful than one aggregate accuracy score.

Uncertainty matters too

For dispatch and UX decisions, intervals can matter as much as point estimates. A 90% prediction interval should contain the actual arrival time about 90% of the time. Coverage and interval width are both relevant.

Connect ETA to business outcomes

A good answer separates model metrics from business metrics.

Examples of business outcomes:

request conversion
cancellation rate
completed trips
pickup delay
rider satisfaction

Guardrails might include:

driver idle time
acceptance rate
surge exposure
support contacts

3) Uber experiments are often not standard A/B tests

This is where many candidates get too generic.

For consumer apps, user-level randomization is often fine. At Uber, treatment can affect shared supply. One rider's treatment can change another rider's outcome. That means SUTVA may fail.

When interference matters

If treatment changes dispatch, pricing, ETA display, or demand, untreated users may still be affected.

Examples:

a rider-facing ETA change shifts demand in a neighborhood
a driver incentive changes driver supply for everyone nearby
a marketplace ranking change affects matching outcomes across groups

If you ignore that, your experiment readout may look precise and still be wrong.

Know when to propose switchback experiments

For marketplace changes, Uber often needs geo-time randomization instead of user-level assignment.

A strong answer for an ETA or dispatch experiment usually includes:

the estimand
the randomization design
primary metrics and guardrails
the inference plan

A reasonable design is a switchback experiment with city-zone-hour cells. You randomize treatment by market and time block, then analyze results with cluster-robust standard errors or a regression with time and geography fixed effects.

Do not use naive row-level standard errors if the design is clustered.

Power is different under clustering

For clustered experiments, you need to account for design effect:

DEFF = 1 + (m - 1)rho

where m is cluster size and rho is intra-cluster correlation.

That means more events inside the same cluster do not help as much as people expect. More independent clusters or time blocks usually matter more.

4) A/B testing answers need a decision framework

A lot of candidates list metrics and stop there. Uber wants a launch recommendation, not a metrics dump.

A solid structure is:

1. Define the objective

Example: Does a promo targeting change increase completed trips or gross bookings at an acceptable promo cost and contribution margin?

2. Pick the right randomization unit

rider_id for rider promos
driver_id for driver incentives
geo or switchback for marketplace changes with spillovers

3. Choose one primary metric

Possible primary metrics:

completed trips per user
conversion rate
gross bookings
variable contribution

Then add a short list of guardrails:

cancellation rate
ETA
surge rate
driver utilization
support contact rate

4. Check validity before interpretation

You should mention:

sample ratio mismatch
exposure correctness
pre-treatment balance
logging completeness
novelty or day-of-week effects

5. Make the recommendation based on practical value

Do not say "p < 0.05, ship it."

A result can be statistically significant and still be a bad launch if contribution drops, promo spend gets out of control, or marketplace health gets worse.

Final prep advice

If you're studying for this interview, spend less time on abstract ML talk and more time on clean definitions, marketplace-aware experiment design, and SQL execution details. That is where many answers get weak.

The full Uber Data Scientist interview prep guide on PracHub goes deeper on ETA evaluation, A/B testing, SQL patterns, and practice prompts. If you want to pressure-test yourself, work through timed practice questions here and say your answer out loud like you're already in the interview.

CTR And Engagement Metrics Explained — Tech Interview Concept (2026)

Feng Zhang — Wed, 27 May 2026 14:24:49 +0000

CTR questions look simple in interviews until you realize the interviewer is not asking for a formula. They want to know whether you can define exposure correctly, separate shallow clicks from useful engagement, and decide what to do when metrics move in opposite directions.

This post is adapted from PracHub's original breakdown of CTR and engagement metrics, but rewritten as a standalone guide for data science and product analytics interviews.

What interviewers are really testing

On ranking-heavy products like a home feed, carousel, Shopping surface, fresh content module, or video feed, small product changes can move several metrics at once:

CTR goes up
saves stay flat
reports rise
retention drops
impressions per user jump because the UI exposes more content

Your job is to reason through that mess without jumping to the wrong conclusion.

Interviewers want to hear that you can:

define metrics precisely
explain what changed causally, not just descriptively
separate product impact from logging issues or mix shifts
make a launch recommendation under uncertainty

If your answer stays at "CTR is clicks divided by impressions," it is too shallow.

Start with the metric definition

Yes, CTR is usually:

$$CTR = \frac{\text{clicks}}{\text{impressions}}$$

But the grain matters a lot.

You might be talking about:

item-level CTR
user-level average CTR
session-level CTR
surface-level CTR

These are different metrics with different interpretations. In experiment analysis, user-level aggregation is often the safer choice because heavy users can otherwise dominate the estimate.

That is one of the easiest points to miss in an interview. If the surface is personalized, the user is usually the right unit for inference.

CTR alone is not the goal

A strong answer treats CTR as an intermediate metric, not the business objective.

For Pinterest-style surfaces, engagement quality can include:

pin clicks
saves
closeups
outbound clicks
follows
board adds
video starts
video completes
Shopping product clicks
return visits

A +3% CTR result does not mean the launch is good. If saves are flat and hide or report rates get worse, you may have made the feed more clickbaity rather than more useful.

That is why you need a metric hierarchy.

Build a metric hierarchy before you interpret anything

A clean interview answer usually has three layers:

1. Primary success metric

Pick the metric closest to user value for that surface.

Examples:

saves per user
engaged sessions per user
shopping-engaged sessions
product detail clicks per user

2. Diagnostic metrics

These tell you where movement came from.

Examples:

impressions per user
module visibility
viewport rate
click position
downstream save rate

3. Guardrails

These stop you from shipping a bad tradeoff.

Examples:

hides
reports
session exits
latency perception
creator or content diversity
overall home feed engagement
retention

If you list ten metrics without saying which one decides the launch, your answer will sound scattered.

Exposure definition is where a lot of people fail

In ranking systems, "impression" is often the hardest metric to define correctly.

An impression should mean the user had a reasonable chance to see the item. It should not mean "the server ranked it."

For a carousel, you should distinguish between:

module rendered
module in viewport
item impression
click
post-click engagement

That distinction matters because a ranking or UI change can alter the denominator mechanically. If more items count as impressions because users scroll farther or because the module renders differently, CTR can fall even if the product got better.

Watch for denominator effects

This is one of the best points you can bring into an interview.

Suppose a recommendation launch shows more content lower in the feed. You may get:

more impressions per user
more clicks per user
more saves per user
lower raw CTR

That is not contradictory. The denominator grew faster than the numerator.

So when CTR drops, do not stop there. Look at both rates and volumes:

CTR
clicks per user
impressions per user
saves per user
save rate

A strong candidate says, "I want to know whether user value fell, or whether the exposure mix changed."

How to talk about experiment design

For personalized feeds, randomize at the user level.

Why? Because recommendation exposure and engagement history are correlated across sessions. Item-level randomization can create an inconsistent user experience and can contaminate training signals.

You should also mention experiment validity checks:

randomization balance
ramp timing
sample ratio mismatch
pre-period comparability

If the assignment is broken, the metric read is not trustworthy.

For surfaces with social or marketplace effects, spillovers can matter. In those cases, you may need to think beyond pure user-level analysis and discuss creator-level or geo-level effects.

Match the stats method to the metric

Interviewers like this because it separates people who know product metrics from people who know how to analyze them.

Binary click outcomes can use proportion-style tests. But engagement per user is often:

heavy-tailed
zero-inflated
noisy

Common approaches include:

user-level means with robust standard errors
bootstrap
winsorization sensitivity checks
delta method for ratios

One important warning: do not treat clicks and impressions as independent row-level observations when you analyze ratio metrics. That usually gives false confidence.

You should also mention power and minimum detectable effect. Tiny CTR lifts can be statistically significant on high-traffic surfaces and still be too small to matter for the business.

The sample size framing from the source is:

$$n \approx \frac{2\sigma^2(z_{1-\alpha/2}+z_{1-\beta})^2}{\Delta^2}$$

where $\Delta$ is the minimum detectable effect.

The right follow-up is, "What effect size would change a launch decision?"

CUPED is worth bringing up

If the interviewer asks how to improve experiment sensitivity, CUPED is a solid answer.

The adjustment is:

$$Y_{adj}=Y-\theta(X-\bar{X}),\quad \theta=\frac{Cov(Y,X)}{Var(X)}$$

Use it when pre-period behavior predicts post-period behavior and treatment cannot affect that pre-period covariate. It is especially useful for noisy user-level metrics like saves per user or Shopping clicks.

You do not need a long derivation. Just show that you know when it helps.

A good framework for "CTR dropped after a recommendation launch"

This is the kind of case question you might get directly.

A solid structure has four parts:

1. Validate the metric path

Ask how impressions and clicks are defined.

Check:

ranked impression vs viewport impression
deduping rules
whether the new system changed logging or counting logic

2. Validate the experiment

Ask whether it was an A/B test or full rollout.

Check:

randomization balance
sample ratio mismatch
pre-period similarity
ramp timing

3. Decompose the funnel

Since:

$$CTR = \frac{\text{clicks}}{\text{impressions}}$$

ask:

Did clicks fall?
Did impressions rise?
Did position mix shift toward lower slots?
Did visibility or viewport rates change?

4. Segment for diagnosis

After checking the overall result, cut by:

new vs returning users
heavy vs light users
platform
country
content type
fresh vs evergreen content
video vs static
shopping-intent users
position or session intent

The key judgment call is this: lower CTR may be acceptable if deeper value improves. If saves per user, long-clicks, Shopping conversions, or retention go up while low-quality clicks go down, "CTR dropped" is not enough reason to roll back.

The Shopping version of this question

For a Shopping launch, CTR is even less likely to be the final metric that matters.

Primary metrics may shift toward:

product detail clicks per user
merchant outbound clicks
add-to-cart proxies
shopping-engaged sessions

And your guardrails still matter:

overall home feed engagement
user trust
retention
cannibalization of organic pin engagement

You should also mention heterogeneity. Users with shopping intent may benefit, while casual browsers may see irrelevant commerce content. That changes how you think about targeting and interpretation.

Common mistakes

Treating CTR as the business goal

It is usually a diagnostic or intermediate metric.

Ignoring exposure changes

A CTR drop can come from more low-intent impressions, different positions, or a new UI module.

Listing metrics without a decision rule

Say what is primary, what is a guardrail, and what tradeoff is acceptable.

A much better interview line is:

"Launch if saves per user or shopping-engaged sessions improve without meaningful degradation in retention, home feed engagement, or negative feedback."

Final takeaway

If you get a CTR question in an interview, do not answer it like a spreadsheet exercise. Define exposure carefully. Use user-level reasoning. Check denominator effects. Separate shallow clicks from meaningful engagement. Then make a decision with guardrails, not with one ratio.

If you want the original concept note, formulas, and interview framing, read the full PracHub post on CTR and engagement metrics. If you want more practice in this style, PracHub also has a set of interview questions on metrics, experimentation, and product data science.

Meta Data Scientist Interview Cheatsheet 2026

Feng Zhang — Sun, 24 May 2026 04:01:10 +0000

If you're preparing for Meta data scientist interviews, one pattern shows up fast: the bar is not "can you compute a metric?" It is "can you define the right metric, design a clean experiment, and explain tradeoffs like an owner?"

This article pulls together the most interview-relevant parts of PracHub's Meta Data Scientist interview prep guide, with a focus on areas candidates often get pressed on: notification analytics, A/B testing, cluster randomization, and SQL event logs.

What Meta interviewers are usually testing

Across technical screens and onsites, the questions often sound broad:

"How would you evaluate similar-listing notifications?"
"Design an experiment for a new ads ranking model"
"Write SQL to compute engagement or call metrics"
"What would you do if there is interference between users?"

The underlying skill is the same. You need to move from raw events or product ideas to a decision-ready analysis. That means:

defining eligibility
choosing a randomization unit
picking a primary metric
adding guardrails
checking whether the observed impact is actually incremental

If your answer stays at the dashboard level, it will usually feel weak.

Notification analytics is a causal question, not a CTR question

A common interview prompt is some variation of push notifications or similar-listing alerts. The mistake many candidates make is optimizing for click-through rate.

That is too shallow.

For notification products, Meta cares about whether the notification creates net value or just interrupts people enough to get clicks. A strong answer breaks the system into a funnel:

eligibility
send
delivery
impression or open
click
landing-page engagement
downstream action
longer-term retention

For a marketplace notification, downstream actions matter more than raw clicks. Examples from the source include:

listing_view
save
seller_message
offer_sent
purchase_intent
transaction_proxy

If the product goal is better buyer discovery, then a better primary metric than notification_click_rate might be:

incremental qualified_listing_views_per_user
buyer_seller_message_threads_per_eligible_user

That framing shows you understand the product mechanism.

Guardrails matter more for notifications than people think

Notifications impose an attention cost. Your answer should include guardrails such as:

push_opt_out_rate
notification_disable_rate
app_uninstall_rate
hide_report_rate
negative_feedback_rate
session_depth
7d_retention
total notification volume per user

If you ignore fatigue, your experiment design looks incomplete.

Be precise about eligibility and exposure

Another common failure mode is saying, "compare users who got notifications with users who didn't."

That comparison is biased. Users who receive notifications are often already more active, have permissions enabled, or have more relevant inventory available.

A better answer starts with a fixed eligible population, for example users who:

viewed or saved a marketplace item in the last 7 days
have push permissions enabled
have at least one similar listing available

Then analyze intent-to-treat on randomized eligible users. You can inspect treatment-on-treated later, but only with the right causal caveats.

Watch for cannibalization and spillovers

A similar-listing notification can shift behavior from search, organic feed, saved items, or other notification channels rather than create new demand. So you should measure total marketplace engagement, not only attributed notification clicks.

If the product has social, household, or marketplace spillovers, say that directly. That is often when an interviewer pushes into cluster randomization.

A/B testing answers need an estimand, not just a p-value

Meta interviewers want to hear that you can design an experiment before data exists, not just analyze one afterward.

Start with the decision and the causal quantity. In plain terms: what launch decision does this test inform, and for whom?

For many interview prompts, your structure can be:

Define the product change and eligible population
Choose a randomization unit
Name the primary metric
Add guardrails
Discuss power, variance, and diagnostics
Explain how you'd interpret null or mixed results

Choose the randomization unit based on interference risk

User-level randomization is often fine for isolated product changes. It is not automatically correct.

If one user's treatment can affect another user's outcome, then SUTVA may fail. In Meta-style products, that comes up in:

social feeds
messaging
ads auctions
marketplaces
creator ecosystems

In those cases, you may need cluster, geo, advertiser, page, or marketplace-level randomization.

If you say, "I'd use user-level randomization if interference is low, otherwise I'd consider cluster or geo designs," that is already much stronger than forcing every problem into a 50/50 user RCT.

Power should be discussed at the right level

For repeated notifications or clustered experiments, observations are correlated. You should talk about power at the user or cluster level, not at the event level.

The source also calls out the design effect for clustered experiments:

DE = 1 + (m - 1)rho

where m is average cluster size and rho is intracluster correlation.

That matters because a huge row count can still translate into a much smaller effective sample size.

CUPED is worth mentioning if the prompt invites depth

For noisy product metrics, pre-experiment covariates can reduce variance. The source mentions CUPED, which adjusts outcomes using pre-period behavior. You do not need to derive it in every answer, but mentioning it in a Meta interview often signals practical experiment experience.

Use it when the pre-period metric strongly predicts the post-period metric, such as engagement, spend, or retention.

How to answer a "similar-listing notifications" question

A solid answer could sound like this:

First, clarify the product goal. Are you trying to increase discovery, transactions, or re-engagement among users with shopping intent?

Next, define the eligible population: users who recently viewed or saved an item, have push permissions on, and have relevant similar inventory available.

Then propose user-level randomization if interference is limited. Treatment users receive similar-listing pushes, control users stay on the current notification policy.

Pick a primary metric tied to downstream value, like incremental qualified_listing_views_per_eligible_user or buyer_seller_message_threads_per_user.

Use secondary metrics like:

notification_open_rate
save_rate
return_sessions

Add guardrails:

push_opt_out_rate
notification_settings_disable_rate
hide_report_rate
notifications received per user
7d_retention

Then say you'd analyze ITT first, check whether gains are incremental versus cannibalized from existing surfaces, and look at heterogeneous effects by intent, notification sensitivity, and inventory density.

That answer is much closer to what interviewers want than "I'd compare CTR between treatment and control."

SQL event log questions are mostly about grain and joins

The SQL side of the interview is less about syntax tricks and more about getting metric definitions right.

The source's advice is simple and useful:

1) Decide the grain first

Know what one row means before you write code:

user-day
call-day
impression-day
country-day

A lot of mistakes come from skipping this step.

2) Be careful with time windows

Use bounded windows like:

event_ts >= start
event_ts < end

That avoids double counting midnight events.

3) Aggregate before joining when needed

Joining raw event tables too early can multiply rows and inflate clicks, responses, revenue, or duration.

4) Protect ratio calculations

Use safe denominators and be explicit about what happens when the denominator is zero.

5) Clarify deduplication rules

If the metric requires one valid event per entity or one response per user, say how you would dedupe, often with ROW_NUMBER().

These are basic ideas, but they come up constantly in product analytics interviews.

What candidates most often miss

From the source, the recurring weak spots are:

optimizing for CTR alone
being vague about eligibility or exposure
ignoring interference and repeated treatment
assuming every experiment should be user-level 50/50
treating a null result as proof of no effect
skipping diagnostics like sample ratio mismatch, logging sanity, or pre-period balance

If you avoid those, your answers already sound more senior.

A better way to use a cheatsheet

Don't memorize lines. Practice turning these patterns into spoken answers.

Take a prompt like notifications, ads ranking, or call metrics, and force yourself to answer in this order:

goal
population
unit of randomization
primary metric
guardrails
power and inference risks
interpretation

If you want more realistic drills, PracHub also has interview practice questions here.

And if you're preparing specifically for Meta, the full Meta Data Scientist interview prep guide on PracHub is the better reference because it keeps these topics in one place and ties them to actual interview-style prompts.

System Design 101

Feng Zhang — Tue, 05 May 2026 03:44:03 +0000

System design is one of those skills people try to speedrun, then realize that it just does not work that way.

This article is adapted from a PracHub post on System Design 101, and the point is simple: if you want to get good at system design, real work matters more than polished tutorials.

A lot of interview prep material makes system design look like a set of reusable templates. Some patterns do repeat, but strong interview performance usually comes from having seen real systems, real constraints, and real tradeoffs.

Real system design experience beats tutorial knowledge

The fastest way to build system design judgment is through work:

building systems yourself
reading designs from other teams
seeing what failed in production
understanding why one approach beat another

That is very different from memorizing a "design Twitter" or "design Uber" walkthrough.

The source article makes a good point here. The author had led several designs that later showed up as classic interview questions. The value was not that they had seen the question before. It was that they had already gone through the parts most prep content skips:

implementation details
tradeoffs between candidate solutions
hardware assumptions
load test results
production pitfalls

That is why experienced engineers often sound more convincing in system design interviews. They are not reciting. They are talking about work they have done.

Breadth vs depth depends on your level

One useful part of the original post is the distinction between mid-level and senior interviews.

If you are mid-level

System design interviews usually test breadth more than depth.

You can pass without knowing every technology in detail. You do need to propose a reasonable solution, explain your choices, and avoid obvious mistakes. Interviewers are usually looking for sane architecture, good data flow, and awareness of tradeoffs.

If you are senior or above

Breadth alone is not enough.

You need depth too. You should be able to support decisions with experience, data, and a clear explanation of failure modes. If there is a gap in an area that matters to the problem, it can hurt a lot more at senior level than it would at mid-level.

That also changes how you should grow your career.

How to build system design skill through your job

The advice here is practical.

Early in your career, moving across teams or projects can help you build breadth. You see different architectures, constraints, and patterns. Later, staying longer in a domain helps you build depth. That is where you start to understand the details that separate an okay design from one that holds up under load.

Over time, a lot of concepts connect:

data modeling affects scaling choices
workload shape affects storage design
consistency requirements affect architecture
cost and capacity affect almost every decision

If your current role gives you none of that, it is fair to ask whether it is the right place for your growth.

What to study first

The source recommends a small set of resources and is honest about their limits.

1. Designing Data-Intensive Applications

DDIA is the foundation.

People often call it the bible of system design, but a better way to put it is that it is a starter book for distributed data systems. That is still very valuable. Most system design interviews are really about data:

what data exists
how much of it there is
how it is accessed
how it is stored
what integrity guarantees matter

DDIA helps you build that mental model.

It will not hand you interview answers. It is weaker on batch and stream processing, so you may need other material if you want more depth there.

2. System Design Primer

The System Design Primer is useful for beginners.

The warning from the source is fair: because it is crowd-sourced, some content has errors. Read it critically. Use it to learn concepts, not as something to memorize word for word.

3. Classic distributed systems papers

The source specifically calls out:

GFS
MapReduce
Bigtable
DynamoDB

If you have never read these, they are worth your time. They shaped a lot of what later systems and interview discussions borrow from.

4. Other books

The source also mentions "Designing Distributed Systems" and books focused on Kafka, Flink, or real-time analytics. The take is measured. They can help fill gaps, but DDIA and classic papers give you the stronger base.

Learn from real production cases

One of the best suggestions in the source is to study production systems from large companies.

If you work at a company with mature infrastructure, read internal design docs from other teams. If you do not, company engineering blogs and conference talks are the next best thing.

Good sources include:

company tech blogs from firms like Uber and Dropbox
InfoQ talks
architecture talks from companies like Google, Meta, and Amazon

You will not always get full schema details. Companies are careful about that. Still, these materials are closer to how systems are actually built than many interview prep articles.

Be selective with popular prep resources

The original post has opinions here, and they are useful.

Grokking is okay for basic concepts and the ID generator example, but the rest is not worth much.
Alex Xu's first book is too shallow.
The second book has more content, but quality is uneven.
The "System Design Interview" YouTube channel has a good rate limiter video, but at least one Top K solution is described as outdated enough to fail interviews.

That may sound harsh, but it matches what many engineers eventually learn: a lot of system design content is polished, simple, and incomplete.

What interviews usually care about

Most system design interviews revolve around data.

A clean way to think about the discussion is:

What are the requirements?
What data do you need to support them?
What are the size and access patterns of that data?
How will you store, retrieve, and protect it?

That is why so many weak system design answers feel off. They jump straight to components like Kafka, Redis, or sharding without first getting the data model and access patterns right.

A good interview answer should show:

reasonable infrastructure choices
correct data flow
a clear thought process

Pattern recognition matters, but only after understanding the problem

You will start to notice that many interview questions share structure.

The source gives one example: group chat and multiplayer card games can have similar data handling patterns. That is a useful observation. Still, pattern matching only helps if you actually understand the data and requirements. Otherwise you end up forcing the wrong template onto the problem.

Capacity estimation: interviews vs real work

This distinction is useful.

At work, capacity planning should be precise enough to support scaling and cost decisions. In interviews, order-of-magnitude estimates are often enough:

GB or TB?
thousands or millions of QPS?

Those estimates shape your technical choices.

If you are interviewing for senior roles, being able to do more exact back-of-the-envelope math and tie it to infrastructure choices and cost is a strong signal.

Case studies worth reviewing

The source recommends examples that do not skip schema design, which is a good filter. If the data model is vague, the rest of the architecture is often weak too.

Examples called out in the post:

Rate limiter, especially the well-known YouTube walkthrough
Chat application case study
Job scheduling system case study

The rate limiter example is considered solid for interviews, but the source notes a few missing angles, like local rate limiters as safeguards and deeper thinking around CPU or memory-based limits.

The chat and job scheduling writeups are described as good enough for entry-level interviews, with some flaws but stronger than many articles written by people with more authority and less substance.

If you want prompts to practice with after reading, PracHub also has a set of interview questions here.

The takeaway

System design skill comes from accumulated exposure to real systems.

Books help. Papers help. Interview case studies help. But the biggest jump happens when you build something, operate it, measure it, and learn what broke.

That is also the standard you should use in interviews. Your answer should sound like something you would actually build at work, not a guess assembled from buzzwords.

If you want the original version of these ideas, the source post on PracHub is here: System Design 101.

Most Common Amazon Interview Questions by Role (2026)

Feng Zhang — Tue, 05 May 2026 03:42:03 +0000

Amazon runs a different interview loop than most big tech companies. The technical bar matters, but the behavioral bar is unusually high. Every round, including coding and design, checks for Leadership Principles.

If you are preparing for Amazon, this role-by-role breakdown from PracHub is a good starting point: Most Common Amazon Interview Questions by Role (2026).

What the Amazon interview process looks like

The structure is fairly consistent across roles:

Online Assessment (OA)
For SDE roles, this is usually 1-2 coding problems. For data roles, expect SQL and analytics-style questions. It is timed, often around 90 minutes.
Phone screen
Usually one technical question and 1-2 behavioral questions tied to Leadership Principles.
Onsite, usually a virtual loop
Expect 4-5 rounds, each around 45-60 minutes. Every round includes at least one behavioral question. One interviewer is the Bar Raiser, a trained interviewer from another team who can veto the hire.

That last point matters. Amazon does not treat behavioral as a warm-up. It is part of the decision in every round.

SDE interviews: coding first, behavior in every round

For Software Development Engineer roles, the process is coding-heavy, but behavioral prep is mandatory.

What shows up most often in coding rounds

PracHub has 160 Amazon coding questions in its dataset, and the common topics are pretty predictable:

Arrays and strings
Two pointers
Sliding window
Trees and graphs
BFS and DFS
Lowest common ancestor
Dynamic programming, usually medium difficulty
Data structure implementation, such as LRU cache

One thing that catches people off guard is the framing. Amazon often wraps standard problems in practical business scenarios like:

warehouse optimization
delivery routing
inventory management

The underlying problem may still be a graph traversal or a sliding window question, but the prompt sounds like an operations problem.

System design for SDEs

PracHub lists 48 Amazon system design questions. The recurring themes are very Amazon-shaped:

Design an order management system
Design a product recommendation engine
Design a delivery tracking system
Design a pricing system with real-time updates

These are not abstract whiteboard exercises. You need to connect technical choices to scale, reliability, latency, and business impact.

Behavioral topics that come up again and again

PracHub tracks 122 Amazon behavioral questions, and some Leadership Principles show up far more often than others:

Customer Obsession
Ownership
Dive Deep
Bias for Action
Deliver Results

Interviewers explicitly map your answers to these principles. They take notes on what you demonstrated, then compare impressions across the loop. If your examples are vague, you will feel that quickly.

Data Scientist interviews: SQL, experiments, and product metrics

Amazon Data Scientist interviews have a different balance. You still need strong behavioral answers, but the technical side leans toward analytics, experimentation, and applied ML.

PracHub's Amazon set includes 65 SQL questions and 71 ML questions. Common examples include:

"Write a query to calculate customer lifetime value"
"Design an experiment to test a new recommendation algorithm"
"How would you detect fraudulent seller accounts?"
retention analysis
funnel analysis
cohort analysis

What Amazon tends to care about in ML rounds

The ML areas called out in the source are tightly tied to Amazon's product and marketplace model:

recommendation systems
fraud detection
demand forecasting
NLP for review analysis
search ranking

This is useful because it tells you where to focus. If your prep is centered on generic model trivia, you may miss what Amazon actually asks, applied questions tied to user behavior, marketplace integrity, or retail operations.

Product sense matters more than many candidates expect

Amazon DS interviews put real weight on product metrics. You need to explain how success is measured and how you would test changes. That means being comfortable with experiment design, tradeoffs in metrics, and the business meaning behind your analysis.

If you answer with technical detail but cannot define the right success metric, that is a problem.

Data Engineer interviews: heavy SQL and reliable pipelines

Data Engineer interviews at Amazon are very SQL-heavy. The source is direct about that, and it lines up with what candidates usually report.

Expect questions around:

complex SQL on large datasets
query optimization
data modeling, such as star schema for e-commerce data

The design side focuses on data systems, not general backend design.

Common pipeline design themes

Typical prompts include:

Design an ETL pipeline for order data
Handle late-arriving data
Design a data quality monitoring system
Migrate from batch to real-time processing

Amazon cares about scale and reliability here. A clean architecture diagram is not enough. You need to explain what happens when jobs fail, when data arrives late, when retries create duplicates, or when upstream quality drops.

If you skip failure modes, your answer is incomplete.

What applies to every Amazon role

Some prep advice is role-specific. Some is universal.

1. Prepare 12-15 STAR stories

This is the biggest pattern in Amazon prep. You need a bank of stories mapped to Leadership Principles.

The source is blunt on this point. It is not optional.

A lot of candidates prepare hard for coding or SQL, then improvise behaviorals. That is a bad tradeoff for Amazon. Since every round includes behavioral questions, weak stories can sink an otherwise strong loop.

2. Be precise with metrics

Amazon is data-driven, and interviewers expect specifics. "We improved performance" is weak. "We cut latency by 28%" is useful.

The same applies to product work, incident response, project delivery, and system design. Use numbers whenever you can. If your example has no measurable result, it will sound unfinished.

3. Think in terms of the flywheel

This comes up most often in system design and product discussions. Amazon likes reasoning that connects technical choices to business outcomes through reinforcing loops.

If your design improves delivery speed, does that improve customer trust, which drives more usage and increases operational efficiency? That style of thinking tends to land well in Amazon interviews.

4. Understand what the Bar Raiser is doing

The Bar Raiser is not there to fill a seat for one team. This person is judging whether you meet Amazon's hiring standard overall.

That usually means close attention to Leadership Principles, quality of judgment, and consistency across rounds. If one round says you show strong Ownership and another suggests the opposite, that will come up in the final discussion.

How I would prep, based on this breakdown

If I were targeting Amazon, I would split prep like this:

Build a Leadership Principles story bank first
Practice role-specific technical questions second
Rehearse answers with numbers, tradeoffs, and clear outcomes
For design rounds, tie the system back to customer impact and business metrics

I would not prep from random lists alone. Amazon patterns are role-dependent. SDE, DS, and DE loops overlap on behaviorals, but the technical expectations are clearly different.

If you want to practice against a large role-specific set, PracHub has Amazon questions across coding, behavioral, ML, SQL, and system design here: interview questions on PracHub.

The useful part is the distribution: 160 coding, 122 behavioral, 71 ML, 65 SQL, and 48 system design questions from Amazon. That makes it easier to focus on what your target role is likely to test instead of studying everything equally.

For the full role-by-role breakdown, go back to the original PracHub post: Most Common Amazon Interview Questions by Role (2026).