The Importance of Hypothesis Testing in Data Science.

Data Science: To a beginner most of the time it means data collecting and building machine models but it is more than that. It is about making informed decisions based on evidence.
Whether you are analyzing customer behavior, testing medical treatment, evaluating marketing campaign or improving business processes, Hypothesis testing provides a scientific framework for whether observed patterns are genuine or simply due to random chance.
To avoid drawing unreliable conclusions, Hypothesis testing allows scientists to validate assumptions, compare groups, measure relationship and support decision-making with statistical confidence.
In this article, we will explore the importance of hypothesis testing and examine the relevance of every major hypothesis test commonly used in Data Science.

What is Hypothesis Testing?

Hypothesis testing is a statistical method used to determine whether there is enough evidence in a sample to support a claim about an entire population.
Every hypothesis test begins with two statements:

• Null Hypothesis(H0): The default assumption that there's no effect, no difference or no relationship in the population. You assume this is true until the data proves otherwise
• Alternative Hypothesis(H1): Your research assumption: It states that there's an effect, a difference or a relationship. This is what you are actually trying to prove

Example:

• H0: A new recommendation algorithm does not increase customer purchases.
• H1: A new recommendation algorithm increases customer purchases. The decision is based on a P-Value, which measures the probability of observing the sample results if the null hypothesis were true.
• Typically: if p-vale <= alpha (0.05), reject H0

Why Hypothesis Testing Matters in Data Science

Hypothesis testing is important because it helps data scientist:

• Detect meaningful relationships
• Evaluate business changes
• Validate machine learning results
• Compare algorithms
• Build confidence in analytical findings
• Make objective decisions

Instead of saying 'Sales appear to have increased', hypothesis test allows us to conclude, 'Sales increased significantly with a p-value of 0.01'.
This distinction is what separates descriptive analysis from evidence-based decision making.

Steps in Hypothesis testing/Best practices

• Step 1: Define the business question
• Step 2: States the hypothesis - Clearly define both Null and alternative hypothesis.
• Step 3: Formulate - Decide the level of significance(alpha) and the statistical test(eg t-test, chi-square,ANOVA)
• Step 4: Analyze the data - Calculate the test statistic and its corresponding p-value from the sample data.
• Step 5: Interpret the results - Compare the p-value to your significance level.
  • If p-value <= alpha, reject the null hypothesis (p <= 0.05)
  • If p-value > alpha, fail to reject the null hypothesis

Types of Hypothesis Tests and Their Importance

1. T-Test

we have various types of t-tests:

1. One-Sample ttest
2. Paired ttest
3. Independent ttest

1.1 One-Sample T-test

Compares a sample mean to a known population mean.

Example

The average height of Kenyan men is 5'5 ft.

• H0 : The average height of kenyan men is 5'5 ft(u = 5'5 ft)
• H1: The average height of kenyan men is not equal to 5'5 ft( u != 5'5 ft)

Assumptions of One sample ttest

• Continuous scale: the data must be continuous
• Random sampling: Data must be collected from random sample of the population
• Normality: The data in the population should be approximately normally distributed

Real world Example:

from scipy.stats import ttest_1samp

# Two tailed test: Is the average rent in Nairobi equal to 120000

h0 = "Average rent is equal 120000(u = 120000)"
h1 = "Average rent is not equal to 120000(u != 120000)" 

alpha = 0.05

stat, p = ttest_1samp(housing_data["monthly_rent_kes"], popmean=120000)

print(f"T-statistics: {stat:.2f}")
print(f"P-Value: {p:.2f}")

if p <= alpha:
    print(f"Reject the null hypothesis {h0}")
else:
    print(f"Fail to reject the null hypothesis {h0}")

#Ouput
T-statistics: 25.53
P-Value: 0.00
Reject the null hypothesis Average rent is equal 120000(u = 120000)

Importance in Data Science

• Business KPIs
• Quality control
• Website performance metrics
• Manufacturing benchmarks

1.2 Independent T-test

compares the means of two independent groups

Assumptions

• Continous variable: The dependent variable must be measured on a continous scale
• Two groups: The independent variable must consist of exactly two categorical, independent groups
• Independent observations: There must be no relationship between the participants in either group
• Normal distribution: The depedent variable must be approximately/roughly normally distributed within each of the two groups
• Random Sampling: Data must be collected using a random sampling method from the population.
• Homogeneity of variance: The variance(spread) of the independent variable must be roughly in both groups

Example

A bank wants to know whether people who are approved for loans have a significant different average salary compared to people who are not approved.

• Group 1: Customers whose loans have been approved
• Group 2: Customers whose loans have not been approved We are comparing their salaries

from scipy.stats import ttest_ind

h0 = "There is no significant difference between the salaries in Group 1 and 2(u1 = u2)"
h1 = "The Average salaries for Group1 and Group2 are not equal"

data = {"customer":["Mark", "Steven", "Allan", "Bill", "Micheal", "Jesse", "Sam"],
        "salary":[85000, 92000, 78000, 110000, 97000, 45000, 52000],
        "loan approval":["Approved", "Approved", "Not Approved", "Not Approved", "Not Approved", "Approved", "Approved"]}

df = pd.DataFrame(data)

approved_salary = df[df["loan approval"] == "Approved"]["salary"]
not_approved_salary = df[df["loan approval"] == "Not Approved"]["salary"]

alpha = 0.05

t_test, p = ttest_ind(approved_salary, not_approved_salary, equal_var=False)

print("T-statistic", t_test)
print("P-value", p)

if p <= alpha:
    print(f"Reject null hypothesis: {h0}")
else:
    print(f"Fail to reject null hypothesis: {h0}")

#Output
T-statistic -1.7715946167319203
P-value 0.13671546072156335
Fail to reject null hypothesis: There is no significant difference between the salaries in Group 1 and 2(u1 = u2)

Importance in data science

Independent T-test is one of most common tests in business analytics and it is used for:

• Medical research
• Marketing experiments
• Customer segmentation
• Product comparison

1.3 Paired T-test

Compares the same users before and after a product change or a system change.

Assumptions

• Continous dependent variable: The dependent variable must be measured on continous scale
• Paired observations: The data must consist of matched pairs
• Normal distribution of differences: The differences of the pairs must follow a normal distribution

Note: The individual group do not need to have a normal distribution, only the calculated difference

Example

Netflix upgrades its recommendation algorithm.

The same users are observed before and after the upgrade

user	Before(minutes watched)	After(minutes watched)
1.	65	81
2.	44	59
3.	73	75

Research question
Did the new recommendation system increase with time?

from scipy.stats import ttest_rel

data = {"User":[1, 2, 3],
        "Before":[65, 44, 73],
        "After":[81, 59, 75]}

netflix_df = pd.DataFrame(data)

# formulate the hypothesis

H0 = "There's no difference in watch time before and after new recommendation system(u1 = u2)"
H1 = "There's a significant difference in the watchtime before and after"

alpha = 0.05

stat, p = ttest_rel(netflix_df["Before"], netflix_df["After"])

print("T-statistic", stat)
print("P-value", p)

if p <= alpha:
    print(f"Reject null hypothesis: {H0}")
else:
    print(f"Fail to reject null hypothesis: {H0}")

# Output
T-statistic -2.4394301941500904
P-value 0.13486786659642408
Fail to reject null hypothesis: There's no difference in watch time before and after new recommendation system(u1 = u2)

• Paired T-test is important because is perfect for measuring improvements.

2. Z-Test

Purpose:
Z-test tests population means when:

• Sample size is large
• Population variance is known

Assumptions required to use Z-test

• Known population variance or standard deviation: The true population variance and standard deviation must be known
• Random sampling: Your data must be collected using a simple random sampling method to ensure it is representative of the broader population
• Independence of observation: The data point within the sample or between the samples must be entirely independent of one another
• Normality: The underlying population should follow a normal distribution
• Continuous data: The variable being measured should be continuous rather than discrete

3. Chi-Square Test

The Chi-Square test is used when working with categorical data.
It helps answer questions such as:

• Is a disease related to smoking status?
• Is customer gender related to the product preference?

There are two common Chi-Square test

• Chi-Square test of independence
• Chi-Square goodness of fit test

3.1 Chi-Square test of independence

Used to determine whether two categorical variables are related. eg Is a customer's gender associated with whether they purchase a product

Gender	Product Preference
Male	Electronic appliances
Female	Fashion
Female	Beauty
Male	Camping

• Null Hypothesis(H0): There is no relationship between customer's gender and their product preference
• Alternative Hypothesis(H1): There is a relationship between customer's gender and their product preference.

example

Imagine you are a data scientist and you'd like to investigate:

• Is a student's county associated with whether they secure employment after completing a data course?

from scipy.stats import chi2_contingency

data = {"county":["Nairobi", "Nairobi", "Nairobi", "Nairobi", "Kiambu", "Kiambu",
                  "Kiambu", "Kisumu", "Kisumu", "Kisumu", "Nakuru", "Nakuru"],
                  "employed":["Yes", "Yes", "No", "Yes", "Yes", "No", "No", "Yes", "No", "No",
                              "Yes", "Yes"]}
H0 = "There is no relationship between student's county and employment"
H1 = "There is a relationship between student's county and employment"

alpha = 0.05

table = pd.crosstab(df["employed"], df["county"])

chi2, p, dof, expected = chi2_contingency(table)

print(display(table))

print("Chi-Square:", chi2)
print("P-Value:", p)
print("Degree of freedom:", dof)
print("Expected Value:", expected)

if p <= alpha:
    print(f"Reject Null hypothesis: {H0}")
else:
    print(f"Fail to reject the null hypothesis: {H0}")
df = pd.DataFrame(data)

# Output:
None
Chi-Square: 3.4285714285714284
P-Value: 0.3301450108968586
Degree of freedom: 3
Expected Value: [[1.25       1.25       1.66666667 0.83333333]
 [1.75       1.75       2.33333333 1.16666667]]
Fail to reject the null hypothesis: There is no relationship between student's county and employment

3.2 Chi-Square goodness of fit

Used to observe or determine whether the observed frequencies match the expected frequencies

• Null Hypothesis(H0): Observed frequencies match the expected frequencies
• Alternative Hypothesis(H1): Observed frequencies differ from expected frequencies eg Do customers choose products in the proportions we expected?
• Null Hypothesis(H0): Customers choose products in the expected proportions
• Alternative Hypothesis(H1): Customers did not choose products in the expected proportions

Example

A supermarket expects sales to be evenly distributed among four products.

Expected sales:

Product	Expected sales
Colgate	25
Pepsodent	25
Aquafresh	25
Sensodyne	25

Actual sales:

Product	Observed sales
Colgate	35
Pepsodent	20
Aquafresh	30
Sensodyne	15

Is the difference significant?

from scipy.stats import chisquare 

sales_data = {"product":["Colgate", "Pepsodent", "Aquafresh", "Sensodyne"],
             "expected sales":[25, 25, 25, 25],
             "observed sales":[35, 20, 30, 15]}

sales_df = pd.DataFrame(sales_data)

# Formulate the hypothesis

H0 = "Observed frequencies match the expected frequencies.(O = E)"
H1 = "Observed frequencies differ from expected frequencies"

alpha = 0.05

observed = sales_df["observed sales"]
expected = sales_df["expected sales"]

chi2, p = chisquare(f_obs=observed, f_exp=expected)

print("Chi-square: ", chi2)
print("P-value: ", p)

if p <= alpha:
    print(f"Reject the null hypothesis: {H0}")
else:
    print(f"Fail to reject null hypothesis: {H0}")

#Output:
Chi-square:  10.0
P-value:  0.01856613546304325
Reject the null hypothesis: Observed frequencies match the expected frequencies.(O = E)

4. ANOVA(Analysis Of Variance)

ANOVA is a statistical method used to determine if there are significance differences between the means of three or more independent groups.(Categorical vs numerical).
It simply compares means across three or more groups.

Two common types of ANOVA

• One-Way ANOVA.
• Two-Way ANOVA.

Note: In this article we will focus on One-Way ANOVA because it is most used ANOVA test.

Assumptions of ANOVA
For your ANOVA test to be valid, your data must meet three core asumptions.

• Independence: The observations in each group are independent of one another and the participants are randomly sampled.
• Normality: The dependent variable is approximately normally distributed within each group.
• Homogeneity of variances: The variance in each group should be roughly equal.

ANOVA is Widely used for:

• Manufacturing
• Education research
• Product testing
• Medical trials

4.1 One-Way ANOVA

It compares one independent variable across multiple groups.

Practical Example of One-Way ANOVA Test

# Housing Example: Do average rents differ across estates

from scipy.stats import f_oneway 

#Formulate the hypothesis
h0 = "There is no difference in rents across estates"
h1 = "Atleast one average rent is different"

alpha = 0.05

groups = [group["monthly_rent_kes"].values for _, group in housing_data.groupby("estate")]

f_stat, p_val = f_oneway(*groups)
print("F-Statistic", f_stat)
print("P-Value", p_val)

if p <= alpha:
    print(f"Reject null hypothesis: {h0}")
else:
    print(f"Fail to reject null hypothesis: {h0}")

It helps determine if there is any significant difference in rent across estates.

5. Mann_Whitney U Test

It is Non-parametric alternative to the independent t-test.
Instead of comparing the mean of two independent groups, it compares the rankings of observations between those groups.
The test is useful when data is not normally distributed

Difference between Mann_Whitney U Test and Independent T-test

Mann-Whitney U test => "Is the observation from one group generally higher or lower than those from the other group?"
Independent t-test => "Is the everage value between the two groups different?"

When to use Mann-Whitney U test

• There are two independent groups.
• The response variable is ordinal or not normally distributed.
• When the assumptions of independent ttest are violated.

Practical Example of Mann-Whitney U Test

from scipy.stats import mannwhitneyu
# housing exmple: Is there a significant difference in monthly rent between properties in Karen and Umoja

h0 = "There is no significant difference in monthly rent in Umoja and Karen"
# Monthly distribution is the same in karen and Umoja
h1 = "There is a difference in monthly rent in Umoja and Karen"
# monthly distribution differs between Karen and Umoja

alpha = 0.05

karen_rent = housing_data[housing_data["estate"] == "Karen"]["monthly_rent_kes"]
umoja_rent = housing_data[housing_data["estate"] == "Umoja"]["monthly_rent_kes"]

stat, p_value = mannwhitneyu(karen_rent, umoja_rent, alternative="two-sided")

print("Karen mdeian rent:", karen_rent.median())
print("Umoka mdeian rent:", umoja_rent.median())

print("U-Statistic: ", stat)
print("P-Value ", p_value)

if p <= alpha:
    print(f"Reject null hypothesis '{h0}'")
else:
    print(f"Fail to reject null hypothesis '{h0}'")

# Output
Karen median rent: 363740.0
Umoja median rent: 96559.5
U-Statistic:  15232.0
P-Value  5.768145774470841e-42
Fail to reject null hypothesis 'There is no significant difference in monthly rent in Umoja and Karen'

6. Wilcoxon Signed-Rank Test

The Wilcoxon Signed-Rank Test is the non-parametric equivalent to the paired t-test.
It compares paired observations, such as measurements taken from the same subjects before and after intervention.
Rather than comparing the mean of the paired differences, it ranks the absolute differences(while keeping track of their signs) to determine whether there is a systematic increase or decrease.

Difference between Wilcoxon Signed-Rank Test and Paired T-test

• The paired ttest => "Did the average value change"
• The Wilcoxon signed-rank test asks => "Did the typical(median) paired observations changed?"

When is Wilcoxon Signed-Rank Test used?
Use this test when:

• The same individuals are measured twice.
• The difference between paired observations are not normally distributed
• The response is ordinal or continuous

Practical Wilcoxon Signed- Rank Test

# Housing example: Did property rents change significantly from 2024 to 2025?
from scipy.stats import wilcoxon

H0 = "There is no significant change between rent in 2024 and rent in 2025"
H1 = "There is a significant difference between the rent in 2024 and rent in 2025"

alpha = 0.05

rent_2024 = housing_data["rent_2024"]
rent_2025 = housing_data["rent_2025"]

stat, p_value = wilcoxon(rent_2024, rent_2025, alternative="two-sided")

print("Test statistic: ", stat)
print("P-Value: ", p_value)

if p <= alpha:
    print(f"reject null hypothesis '{H0}'")
else:
    print(f"Fail to reject null hypothesis '{H0}'")

# Output
Test statistic:  0.0
P-Value:  0.0
Fail to reject null hypothesis 'There is no significant change between rent in 2024 and rent in 2025'

It is useful when the data is not normally distributed.

7. Kruskal-Wallis Test

The Kruskal-Wallis test is the equivalent of the one-way ANOVA Test.
It compares three or more independent groups by ranking all observation together and then determine whether one or more groups tend to have higher or lower ranks than others.

Difference between One-Way ANOVA and Kruskal-Wallis Test

• One-Way ANOVA Test => "Do the average values differ among these groups?"
• Kruskal-Wallis Test => "Do the distribution(or typical ranks) differ among these groups?"

When is Kruskal-Wallis Test used?

• There are three or more independent groups.
• The response variable is ordinal or not normally distributed.
• When the assumptions of ANOVA are violated

Practical Example using Kruskal-Wallis Test

# Housing Example: Does the monthly rent differ across properties with different numbers of bedrooms?
from scipy.stats import kruskal

H0 = "Monthly distribution is the same across all bedroom groups"
H1 = "Atleast one bedroom group has a differfent rent distribution"

alpha = 0.05

groups = [group["monthly_rent_kes"].values for name, group in housing_data.groupby("bedrooms")]

stat, p_val = kruskal(*groups)

print("Kruskal wallis H statistic: ", stat)
print("P-Value: ", p_val)

if p <= alpha:
    print(f"Reject null hypothesis '{H0}'")
else:
    print(f"Fail to reject null hypothesis '{H0}'")

# Output
Kruskal wallis H statistic:  135.76123559380773
P-Value:  1.4235272194460855e-27
Fail to reject null hypothesis 'Monthly distribution is the same across all bedroom groups'

Choosing the Right Hypothesis Test

Goal	Recommended Test
Compare one mean to a known value	One-Sample t-Test
Compare two independent groups	Independent t-Test
Compare before and after measurements	Paired t-Test
Compare three or more groups	ANOVA Test
Compare three or more non-normal groups	Kruskal-Wallis Test
Compare categorical variables	Chi-Square Test

Hypothesis Testing in Real-World Data Science

Hypothesis testing is applied across many industries:

• E-Commerce
• Manufacturing
• Healthcare
• Financial Industries
• sports analysis
• Marketing
• Education sector
• Human resources

Conclusion

Hypothesis testing being the cornerstone of data science help in transforming raw data into evidence that can support confident decisions.
Each hypothesis tests method serves a distinct purpose in the analytical workflow. Understanding when to use each test, why its appropriate and how to interpret its results enables data scientist build more trustworthy analyses, improve predictive models and deliver insights that drive meaningful business outcomes.
Successful data scientists use hypothesis tests as complementary tools for exploring data, validating assumptions and making evidence-based decisions that stand up to scrutiny.