Supervised ML interview questions usually are not about reciting definitions. The better signal is whether you can pick a model, design the evaluation, explain tradeoffs, and tie the choice to business cost.
If you want the original interview-focused version, PracHub has a concise concept page on supervised ML fundamentals, evaluation, and feature engineering. This post rewrites the same ideas as a practical guide for developer and data science readers.
Start with the decision context
Before comparing algorithms, ask what the model output is used for.
Are you optimizing:
- Probability accuracy?
- Ranking quality?
- Forecast accuracy?
- A hard business action, such as block, approve, alert, or reorder?
- Interpretability for product, operations, or compliance teams?
- Low-latency scoring?
That question changes the model and the evaluation.
A fraud model with a rare positive class should not be judged by raw accuracy. A demand forecast should not use random cross-validation if time order matters. A conversion model used for bidding may need calibrated probabilities, while a search relevance model may care more about ranking metrics.
A good interview answer often starts with: "I would first clarify whether we need calibrated probabilities, ranking quality, forecast accuracy, or an action threshold tied to business cost."
Linear and logistic regression are still worth knowing well
Linear regression estimates coefficients by minimizing squared error:
$$
\min_\beta \sum_i (y_i - x_i^\top\beta)^2
$$
The classic assumptions are linearity, independent errors, homoscedasticity, no perfect multicollinearity, and exogeneity:
$$
E[\epsilon \mid X]=0
$$
Breaking these assumptions does not automatically make predictions useless. It does affect inference, confidence intervals, and how much you should trust coefficient interpretation.
Logistic regression models class probability as:
$$
P(y=1 \mid x)=\sigma(x^\top\beta)
$$
It minimizes log loss, not squared error. In tabular classification, logistic regression is often a strong baseline, especially with sparse high-dimensional features. It is also easier to explain and often better calibrated than many complex models out of the box.
Do not skip the baseline. If a boosted tree barely beats regularized logistic regression but is harder to explain, slower to score, and poorly calibrated, the simpler model may be the better production choice.
Regularization is about controlling variance
Regularization penalizes model complexity to reduce overfitting.
Common forms:
- L2 regularization adds $\lambda|\beta|_2^2$. It shrinks coefficients smoothly and works well with correlated predictors.
- L1 regularization adds $\lambda|\beta|_1$. It can set coefficients to zero, which makes it useful for sparse feature selection.
- Elastic Net combines L1 and L2. It is useful when groups of correlated predictors are present.
- L0 regularization counts nonzero coefficients, $|\beta|_0$. It directly targets subset selection, but exact optimization is usually combinatorial.
- L∞ regularization constrains the maximum absolute coefficient. It is less common in applied interviews, but it tests whether you understand norm geometry and constraints.
Feature scaling matters here. If one feature is measured in cents and another in years, a regularization penalty treats their coefficients differently unless you scale the inputs. Scaling is also needed for distance-based models and many gradient-based linear models.
Tree models such as Random Forests, Gradient Boosted Trees, XGBoost, and LightGBM are mostly invariant to monotonic scaling. Transformations can still help with outliers or skewed distributions.
Random Forests vs Gradient Boosting
This is a common interview comparison because both are tree ensembles, but they behave differently.
Random Forests reduce variance through bagging. They train many decorrelated trees on bootstrap samples and random subsets of predictors. They are stable, parallelizable, and less sensitive to hyperparameter settings. They are often a safe first model when labels are noisy or you need a benchmark that does not take much tuning.
Gradient Boosting trains trees sequentially. Each new tree fits residuals or gradients from the current model. This tends to reduce bias and often works very well on structured tabular data. XGBoost and LightGBM are common examples.
The tradeoff is tuning. Boosted trees need careful control of:
- Learning rate
- Number of trees
- Max depth
- Subsampling
- Column sampling
- Early stopping
If trees are too deep or boosting rounds run too long, boosted models can overfit label noise.
A solid answer sounds like this:
"Random Forests mainly reduce variance through bagging, while Gradient Boosted Trees reduce bias through sequential additive learning. I would compare them based on predictive performance, sensitivity to noisy data, tuning cost, calibration, and the business metric. Boosted trees may win on PR-AUC or lift, but Random Forests may be easier to tune and less brittle."
That answer is much better than "Gradient Boosting is more accurate" or "Random Forests avoid overfitting."
Class imbalance changes the metric
Accuracy is a trap when positives are rare.
If the positive class rate is 0.5%, a classifier that always predicts negative gets 99.5% accuracy. That number is useless.
For imbalanced classification, consider:
precisionrecallF1PR-AUC- Top-k lift
- Recall at fixed precision
- Cost-weighted metrics
ROC-AUC can also mislead when positives are extremely rare. A model can have a strong ROC-AUC while precision is too low for production use.
Sampling and class weights can help training, but they do not replace a good evaluation plan. You still need to choose a decision threshold based on cost.
For example, a fraud model might optimize:
$$
EV(t)=TP(t)\cdot B - FP(t)\cdot C - FN(t)\cdot L
$$
Here, $t$ is the threshold, $B$ is the benefit of catching fraud, $C$ is the cost of a false positive, and $L$ is the loss from a false negative.
The model produces scores or probabilities. The threshold is a business policy layered on top.
Calibration matters when probabilities drive decisions
Ranking quality and probability accuracy are different.
A model can rank cases well but produce poorly calibrated probabilities. That may be fine for a top-k review queue. It is a problem if probabilities feed expected-value calculations, capacity planning, pricing, or downstream decision systems.
Logistic regression is often reasonably calibrated. Boosted trees may need Platt scaling or isotonic regression.
Useful calibration checks include:
- Calibration curves
- Brier score
- Observed vs predicted rates by score bucket
If a bucket of users predicted at 20% conversion actually converts at 8%, that model may still rank users well, but its probabilities are not reliable.
Time-series forecasting needs temporal validation
Forecasting has a different data-generating process from ordinary tabular classification.
Random train/test splits leak future information. They can make a weak model look strong, especially when seasonality, promotions, customer behavior shifts, or inventory effects are present.
Use temporal validation, such as:
- A simple holdout from a later time period
- Rolling-origin evaluation
- Backtesting across several forecast windows
Before using complex models, compare against simple baselines:
- Seasonal naive
- Moving average
- Simple exponential smoothing
Then test models such as ARIMA, Prophet, tree models with lagged variables, or sequence models if the problem calls for them.
Metrics should match the operational decision. For demand forecasting, common choices include WAPE, sMAPE, pinball loss, or a service-level cost tied to stockouts and overstock.
Feature engineering without leakage
Feature engineering should encode signal available at prediction time. That last phrase matters.
Common tabular features include:
- Lagged demand
- Rolling means
- Customer tenure
- Frequency counts
- Recency
- Price bands
- Target encodings for categoricals
- Missingness indicators
- Log-transformed monetary values
The risk is leakage. If you compute a rolling average using data after the prediction timestamp, you have leaked future information. If you use target encoding without proper out-of-fold or time-aware computation, the model may learn the answer indirectly.
For repeated entities, grouped splits may be needed. For time-dependent data, temporal holdouts are safer than random splits. Segment-level error checks can also reveal that a model works overall but fails for a product category, region, or customer group.
How to answer this in an interview
A strong structure is:
- Clarify the target and decision use.
- Describe the data conditions: size, sparsity, label noise, missingness, class balance, time dependence.
- Pick baselines first.
- Compare model families conditionally.
- Choose metrics tied to business cost.
- Discuss calibration, thresholds, and validation design.
- Name leakage risks.
For example:
"I would start with regularized logistic regression as a baseline because it is interpretable and often well calibrated. I would compare Random Forests and Gradient Boosted Trees using a temporally held-out set if time matters. Since positives are rare, I would use PR-AUC, recall at fixed precision, and expected cost rather than accuracy. If probabilities drive actions, I would check calibration and apply Platt scaling or isotonic regression if needed."
That answer shows model knowledge, evaluation judgment, and production awareness.
For more interview prompts around these topics, PracHub has a set of technical interview practice questions you can use to test your reasoning under pressure.
Common mistakes to avoid
Using accuracy by default.
Accuracy is often the wrong metric for imbalanced classification.
Comparing models without conditions.
"XGBoost is better" is not enough. Better for what data, what metric, what latency limit, and what decision?
Ignoring validation design.
Random splits can leak information in time-series and repeated-entity problems.
Forgetting calibration.
A high-ranking model may still produce bad probabilities.
Treating feature engineering as harmless.
Rolling features, target encodings, and aggregations can leak future or label information if computed incorrectly.
If you want the original concept-card version with the compact interview framing, see PracHub's post on supervised ML fundamentals, evaluation, and feature engineering.
Top comments (0)