When working with data, knowing the mean, median, or standard deviation is often not enough.
At some point, you may want to answer questions like:
- Is the average of two groups significantly different?
- Are two variables related?
- Did an intervention change the results?
- Are three or more groups statistically different?
- Does my sample provide enough evidence to reject the null hypothesis?
This is where statistical hypothesis testing becomes useful.
Two major families of statistical tests are:
Parametric tests and non-parametric tests
Understanding when to use each one is an important skill for anyone working in data analysis, data science, or machine learning.
The Basic Idea
Most hypothesis tests start with two competing statements.
Null hypothesis — H₀
There is no statistically significant difference or relationship.
Alternative hypothesis — H₁
There is a statistically significant difference or relationship.
The test produces a p-value.
A common significance level is:
α = 0.05
A simplified interpretation is:
p < 0.05 → evidence against H₀
p ≥ 0.05 → insufficient evidence against H₀
Strong pointer
A p-value is not the probability that the null hypothesis is true.
It tells you how compatible your observed data are with the null hypothesis under the assumptions of the test.
What Are Parametric Tests?
Parametric tests make assumptions about the underlying population distribution and its parameters.
Common assumptions can include:
- Approximately normally distributed data
- Independence of observations
- Continuous measurements
- Equal or appropriately modeled variances in some tests
Common parametric tests include:
| Test | Typical Use |
|---|---|
| t-test | Compare means |
| ANOVA | Compare means across 3+ groups |
| Pearson correlation | Measure linear association |
| Paired t-test | Compare paired measurements |
Parametric tests can be powerful when their assumptions are reasonably satisfied.
What Are Non-Parametric Tests?
Non-parametric tests generally make fewer distributional assumptions.
They are particularly useful when:
- Data are strongly skewed
- Data contain influential outliers
- The sample is small and normality is questionable
- Data are ordinal/ranked
- Parametric assumptions are not reasonably satisfied
Common examples include:
| Parametric | Non-Parametric Alternative |
|---|---|
| Independent t-test | Mann–Whitney U |
| Paired t-test | Wilcoxon signed-rank |
| One-way ANOVA | Kruskal–Wallis |
| Pearson correlation | Spearman correlation |
Strong pointer
Non-parametric does not mean "no assumptions."
These tests still have assumptions about things such as independence, measurement structure, and the form of the distributions or relationships being tested.
Install the Python Libraries
For most statistical testing in Python, SciPy is a great starting point.
pip install scipy pandas numpy
Import them:
import numpy as np
import pandas as pd
from scipy import stats
Independent t-test
Suppose we want to compare the average exam scores of two independent groups.
group_a = [72, 75, 78, 70, 74, 77, 73]
group_b = [82, 85, 88, 84, 86, 89, 83]
result = stats.ttest_ind(group_a, group_b)
print("t-statistic:", result.statistic)
print("p-value:", result.pvalue)
Interpretation:
if result.pvalue < 0.05:
print("Statistically significant difference")
else:
print("Insufficient evidence of a difference")
The independent t-test is designed to compare the means of two independent groups under its assumptions.
Mann–Whitney U Test
What if the data are heavily skewed or ordinal and the assumptions for the t-test aren't appropriate?
We can consider the Mann–Whitney U test.
group_a = [72, 75, 78, 70, 74, 77, 73]
group_b = [82, 85, 88, 84, 86, 89, 83]
result = stats.mannwhitneyu(
group_a,
group_b,
alternative="two-sided"
)
print("U-statistic:", result.statistic)
print("p-value:", result.pvalue)
This test works with the ranks of observations rather than relying on the same distributional assumptions as the t-test.
Important
Don't simply say:
"Mann–Whitney compares medians."
That's an oversimplification.
Its interpretation depends on the distributions of the two groups. Under additional conditions, differences in location can be interpreted more directly.
Paired t-test
Sometimes observations come in natural pairs.
For example:
Student's score BEFORE training
Student's score AFTER training
The same students are measured twice.
before = [65, 70, 72, 68, 75, 71]
after = [70, 74, 78, 73, 79, 76]
result = stats.ttest_rel(before, after)
print("t-statistic:", result.statistic)
print("p-value:", result.pvalue)
This is different from an independent t-test because the observations are paired.
Wilcoxon Signed-Rank Test
If the paired data do not reasonably satisfy the assumptions of a paired t-test, the Wilcoxon signed-rank test is a common alternative.
result = stats.wilcoxon(before, after)
print("Statistic:", result.statistic)
print("p-value:", result.pvalue)
Think of the relationship like this:
Paired numerical data
│
├── assumptions reasonable
│ ↓
│ Paired t-test
│
└── assumptions questionable
↓
Wilcoxon signed-rank
ANOVA
What if we have three or more independent groups?
For example:
Method A
Method B
Method C
We could use one-way ANOVA.
method_a = [72, 75, 78, 70, 74]
method_b = [80, 82, 85, 81, 84]
method_c = [88, 90, 87, 91, 89]
result = stats.f_oneway(
method_a,
method_b,
method_c
)
print("F-statistic:", result.statistic)
print("p-value:", result.pvalue)
If the ANOVA is statistically significant, it tells us that at least one group differs, but it does not tell us which groups differ.
That requires appropriate post-hoc testing.
Strong pointer
Don't run multiple t-tests instead of ANOVA just because you have several groups.
Repeated testing can increase the chance of false-positive findings unless you appropriately control for multiple comparisons.
Kruskal–Wallis Test
The non-parametric counterpart commonly used for comparing three or more independent groups is the Kruskal–Wallis test.
result = stats.kruskal(
method_a,
method_b,
method_c
)
print("H-statistic:", result.statistic)
print("p-value:", result.pvalue)
Again, a significant result tells you that there is evidence of a difference somewhere among the groups.
You may need appropriate post-hoc pairwise comparisons to determine where the differences occur.
Pearson Correlation
Suppose we want to measure the linear relationship between:
Hours studied
and:
Exam score
We can use Pearson correlation.
hours = [2, 3, 4, 5, 6, 7, 8]
scores = [55, 60, 65, 70, 74, 80, 85]
result = stats.pearsonr(hours, scores)
print("Correlation:", result.statistic)
print("p-value:", result.pvalue)
The correlation coefficient ranges from:
-1 → strong negative linear relationship
0 → no linear relationship
+1 → strong positive linear relationship
Important
Correlation does not establish causation.
Correlation ≠ Causation
Spearman Correlation
When the relationship is better represented by ranks or monotonic association, Spearman's correlation can be useful.
result = stats.spearmanr(hours, scores)
print("Spearman correlation:", result.statistic)
print("p-value:", result.pvalue)
A monotonic relationship means that as one variable increases, the other tends to consistently increase or decrease, even if the relationship isn't linear.
How Do I Choose the Test?
Here's a practical cheat sheet:
What are you testing?
│
┌────────────────┼─────────────────┐
│ │ │
2 groups 3+ groups Relationship
│ │ │
↓ ↓ ↓
┌─────────┐ ┌──────────┐ ┌──────────┐
│Independent│ │ ANOVA │ │ Pearson │
│ t-test │ │ │ │ │
└─────────┘ └──────────┘ └──────────┘
│ │ │
↓ ↓ ↓
Mann–Whitney Kruskal–Wallis Spearman
For paired observations:
Paired measurements
│
├── Parametric assumptions reasonable
│ ↓
│ Paired t-test
│
└── Assumptions questionable
↓
Wilcoxon signed-rank
. Don't Automatically Test for Normality
A common beginner workflow is:
Run Shapiro-Wilk
↓
p > 0.05?
↓
Use t-test
This is too simplistic.
Normality testing should not be the only thing determining your choice.
Also consider:
- Sample size
- Distribution shape
- Outliers
- Independence
- Measurement scale
- Study design
- Equal variance assumptions
- Robustness of the chosen test
Visual inspection can help:
import matplotlib.pyplot as plt
plt.hist(group_a)
plt.xlabel("Score")
plt.ylabel("Frequency")
plt.title("Distribution of Scores")
plt.show()
You can also use a Q-Q plot:
stats.probplot(group_a, dist="norm", plot=plt)
plt.show()
. Statistical Significance vs Practical Significance
This is one of the most important lessons in statistics.
Suppose:
p = 0.001
That may provide strong evidence against the null hypothesis under the test assumptions.
But the actual difference might be extremely small.
For example:
Group A average = 70.01
Group B average = 70.10
A large dataset can make tiny differences statistically significant.
Therefore, don't report only:
p < 0.05
Also consider:
- Effect size
- Confidence intervals
- Magnitude of the difference
- Practical/business importance
- Sample size
Golden rule
Statistical significance tells you about evidence; practical significance tells you whether the size of the effect matters.
. A Better Python Workflow
When performing a statistical test, use this workflow:
1. Understand the research/business question
↓
2. Identify the variables
↓
3. Define the groups or relationship
↓
4. Understand the study design
↓
5. Check assumptions
↓
6. Choose the appropriate test
↓
7. Run the test
↓
8. Examine the p-value
↓
9. Report effect size / confidence interval
↓
10. Interpret the result in context
This is much better than:
Load data → run test → look at p-value
. Parametric vs Non-Parametric: Quick Reference
| Question | Parametric | Non-Parametric |
|---|---|---|
| 2 independent groups | Independent t-test | Mann–Whitney U |
| 2 paired measurements | Paired t-test | Wilcoxon signed-rank |
| 3+ independent groups | One-way ANOVA | Kruskal–Wallis |
| Linear association | Pearson | Spearman for rank/monotonic association |
| Distribution assumptions | Generally stronger | Generally fewer distributional assumptions |
| Works with ordinal data | Usually not ideal | Often appropriate |
| Outlier sensitivity | Often higher | Often lower, but not immune |
Conclusion
Parametric and non-parametric tests are not competitors where one is universally better.
They are tools designed for different situations.
Think of the decision this way:
BUSINESS / RESEARCH QUESTION
↓
YOUR DATA
↓
STUDY DESIGN
↓
ASSUMPTIONS
↓
┌──────────┴──────────┐
↓ ↓
Parametric test Non-parametric
↓ ↓
t-test / ANOVA Mann-Whitney /
/ Pearson Wilcoxon / Kruskal
└──────────┬──────────┘
↓
INTERPRET RESULTS
↓
Effect size + CI + Context
The goal isn't to memorize dozens of statistical tests.
The real data-analysis skill is knowing why a particular test is appropriate for a particular question and dataset.
Once you understand that, Python becomes the tool that helps you execute the analysis not the thing making the statistical decision for you.
Top comments (0)