What Are They?
Statistical tests fall into two broad categories: parametric and non-parametric. The choice between them affects the validity of your conclusions — picking the wrong one can lead to misleading results.
Parametric Tests
Parametric tests make assumptions about the underlying population distribution, most commonly that the data is normally distributed. They work with the actual values in your data (means, variances) and tend to be more powerful when their assumptions are met.
Common parametric tests:
- t-test (compare means between two groups)
- ANOVA (compare means across three or more groups)
- Pearson correlation (measure linear relationship between two variables)
- Linear regression
When to use them:
- Your data is continuous
- Sample size is large enough for the Central Limit Theorem to apply (typically n > 30)
- Data is approximately normally distributed
- Variances across groups are roughly equal (homoscedasticity)
Non-Parametric Tests
Non-parametric tests make no assumptions about the population distribution. They work with ranks or categories rather than raw values, making them more flexible — but generally less powerful than parametric tests when parametric assumptions hold.
Common non-parametric tests:
- Mann-Whitney U test (equivalent of independent t-test)
- Wilcoxon signed-rank test (equivalent of paired t-test)
- Kruskal-Wallis test (equivalent of one-way ANOVA)
- Spearman correlation (rank-based correlation)
- Chi-square test (categorical data)
When to use them:
- Data is ordinal (e.g. satisfaction scores, rankings)
- Data is heavily skewed or contains outliers
- Sample size is small (n < 30)
- Data violates normality assumptions
- You are working with categorical or nominal data
Key Differences
| Parametric | Non-Parametric | |
|---|---|---|
| Distribution assumption | Yes (usually normal) | No |
| Data type | Continuous | Ordinal, categorical, or non-normal continuous |
| Statistical power | Higher (when assumptions met) | Lower |
| Sensitivity to outliers | High | Low |
| Sample size requirement | Larger | Smaller |
Role in Data Science
In data science, both approaches appear regularly across the analytics workflow.
Parametric tests are common in A/B testing (t-tests for comparing conversion rates at scale), regression modelling, and feature correlation analysis where datasets are large and roughly normal.
Non-parametric tests are essential when working with customer satisfaction scores, survey data, small samples, or any dataset where normality cannot be assumed, which is more often the case in real-world business data than textbooks suggest.
A practical rule: always check your data's distribution before choosing a test. Use a histogram, Q-Q plot, or the Shapiro-Wilk test to assess normality. If in doubt, non-parametric tests are the safer default, they sacrifice a little power in exchange for broader applicability.
Parametric tests are more powerful but demand more from your data. Non-parametric tests are more flexible and robust. In data science, knowing when to apply each and being able to justify your choice is as important as running the test itself.
Top comments (1)
One thing worth adding to the "pick the non-parametric equivalent" framing: Mann-Whitney isn't really a drop-in for the t-test. The t-test asks about means, Mann-Whitney asks whether one group tends to rank above the other, so on skewed business data they can point in different directions. If the mean is what you actually care about (revenue per user, say) a Welch t-test or a bootstrap often serves you better than switching to ranks. When you reach for the rank test, is it usually because normality broke, or because the median is genuinely the quantity you want?