DEV Community

Cover image for Part 3: Understanding Distributions in Data Science
Sharon-nyabuto
Sharon-nyabuto

Posted on

Part 3: Understanding Distributions in Data Science

Statistics for Data Science (Part 3)

Part 1: Why Statistics Matters in Data Science

Part 2: Descriptive Statistics: A Beginner's Guide

Introduction

In the previous article, we worked through four questions to guide our analysis: What type of data do you have?, Where is it centered?, How spread out is it?, and What does its distribution look like?.

The last question introduced distributions briefly, we saw that some variables formed a bell-shaped curve while others were right-skewed or left skewed.

In this article, we will further explore some of the common distributions, understand what they represent, and examine their key properties. We will also see why understanding the shape of your data matters and how distributions can help you make better decisions about how to analyze and interpret data.

Table of Contents

  1. Introduction
  2. What Is a Distribution?
  3. Data Distributions (Empirical Distributions)
  4. Probability Distributions
  5. Why Distributions Matter
  6. Conclusion

What Is a Distribution?

A distribution describes how the values of a variable are spread across the possible values or range of values it can take. It tells us the values that occur frequently, those occurring rarely, and how the observations are distributed across the range of the variable.

There are two related ideas that are important to distinguish:

Data Distributions (Empirical Distributions)

Describes the actual observations in our dataset. For example, when we plot histograms, we look at the empirical distributions of variables such as bill amount and distance to the facility.

Probability Distributions

Theoretical model that describes how likely different values or ranges of values are to occur.

1. Data Distributions (Empirical Distributions)

The Normal Distribution

The normal distribution is the distribution most of the statistical methods are built around. In a normal distribution, values tend to cluster around the center, with fewer observations as we move farther away.

The common visual characteristics of a normal distribution include a bell-shaped curve, one central peak, tails extending in both directions and a symmetric appearance.

A normal distribution is defined by two things:

  • The mean, which determines where the center of the curve sits.
  • The standard deviation, which determines the spread of the data (how wide or narrow the curve is).

Note: A smaller standard deviation produces a narrower, taller curve, while a larger standard deviation produces a wider, flatter curve.

The data we work with in the real world context rarely follows a theoretical distribution perfectly. However, it may resemble one closely enough for us to use what we already know about that distribution to understand and analyze the data.

Looking at our hospital dataset, age and weight were the variables that came closest to a normal distribution. Neither follows the theoretical bell-shaped curve perfectly, but both show a broadly similar pattern, increasing to a central peak before gradually decreasing.

Age_weight Distributions

Let's look at their mean, median, and standard deviation.
To display the mean, median, and standard deviation of multiple variables at once, we can use the .agg() method and specify the summary statistics we want to calculate.

healthaccess_data[["age", "weight_kg"]].agg(["mean", "median", "std"])
Enter fullscreen mode Exit fullscreen mode
Variable Mean Median Mode Standard Deviation
Age 36.8 years 36 years 36 years 18.89 years
Weight 59.32 kg 62.7 kg 62 kg 19.43 kg

For age, the mean and median are very close, while the difference is somewhat larger for weight. The plots also show that neither variable is perfectly symmetrical.

This is typical of real-world data. We rarely get a distribution that matches a theoretical distribution exactly. The question is often how closely does the data follow the pattern we expect?. To answer this question, we have to explore the empirical rule, tells us roughly what proportion of observations fall within a certain number of standard deviations from the mean.

↑ Top

The Empirical Rule

One useful property of a normal distribution is the empirical rule, also known as the 68-95-99.7 rule. It tells us how much data should fall within one, two, and three standard deviations of the mean if the data follows a normal distribution.

  • About 68% of observations fall within 1 standard deviation of the mean.
  • About 95% of observations fall within 2 standard deviations of the mean.
  • About 99.7% of observations fall within 3 standard deviations of the mean.

Empirical Rule

We can apply the empirical rule to our age and weight variables by checking the proportion of observations that fall within 1, 2, and 3 standard deviations of their respective means. We would expect approximately 68%, 95%, and 99.7% of observations to fall within these ranges if they follow normal distributions

To check this in Python, we can use the following code:

for column in ["age", "weight_kg"]:
    mean = healthaccess_data[column].mean()
    std = healthaccess_data[column].std()

    print(f"\n{column}")

    for n in [1, 2, 3]:
        lower = mean - n * std
        upper = mean + n * std

        within_std = healthaccess_data[
            (healthaccess_data[column] >= lower) &
            (healthaccess_data[column] <= upper) ]

        proportion = len(within_std) / healthaccess_data[column].notna().sum()

        print(f"Within {n} std: {proportion:.2%}")
Enter fullscreen mode Exit fullscreen mode

Output:

age
=========================
Within 1 std: 67.41%
Within 2 std: 96.35%
Within 3 std: 100.00%

weight_kg
=========================
Within 1 std: 73.82%
Within 2 std: 92.10%
Within 3 std: 100.00%
Enter fullscreen mode Exit fullscreen mode

The results show that age follows the empirical rule quite closely, with 67.41% of observations within 1 standard deviation and 96.35% within 2 standard deviations, compared with the expected 68% and 95%.
Weight is less closely aligned, with 73.82% within 1 standard deviation and 92.10% within 2. Both variables have 100% of observations within 3 standard deviations.

Overall, age appears to follow the pattern of a normal distribution more closely, while weight shows some deviation from the theoretical 68-95-99.7 pattern. However, the empirical rule should not be used on its own to assess normality; it is best considered alongside visual checks such as histograms or density plots.

Z-Scores

The empirical rule helps understand how observations are distributed around the mean. To know how far a single observation is from the mean, we use a z-score.

A z-score tells us how many standard deviations an observation is above or below the mean.

Formula:

z=x−xˉs z = \frac{x - \bar{x}}{s}

Where:

  • xx is the individual observation
  • xˉ\bar{x} is the mean
  • ss is the standard deviation

A z-score of 0 means the observation is exactly at the mean. A z-score of 2 means it is 2 standard deviations above the mean, while a z-score of -1.5 means it is 1.5 standard deviations below the mean.

In the hospital dataset, to know how a patient weighing 100 kg compares with the other patients in our dataset:

weight = 100

mean = healthaccess_data["weight_kg"].mean()
std = healthaccess_data["weight_kg"].std()

z_score = (weight - mean) / std
print(f"Z-score: {z_score:.2f}")
Enter fullscreen mode Exit fullscreen mode

Output:

Z-score: 2.09
Enter fullscreen mode Exit fullscreen mode

The z-score of 2.09 means that this patient's weight is approximately 2.09 standard deviations above the mean. Based on the empirical rule, this places the observation toward the upper end of the distribution, beyond the range where we would expect most observations to fall.

Z-scores are also useful when comparing variables measured on different scales(such as the age and weight in our dataset). Once values are converted to z-scores, they are expressed in the same terms: how many standard deviations they are from their respective means.

↑ Top

Skewed Distributions

The empirical rule and z-scores are most useful when data is approximately normally distributed. When a distribution is heavily skewed, these tools may not describe the data as well because extreme values occur more frequently than they would in a normal distribution.

Looking at the distribution of the bill amount from the hospital dataset below,

Bill amount distribution

we can see that small number of patients have exceptionally high bills, creating a long right tail. In this situation, a value several standard deviations above the mean may not be as unusual as it would be in a normal distribution.

This is also why the median is more useful than the mean for heavily skewed data. It is a better representation of a typical observation when the distribution has substantial skew.

The important point is that the shape of our data influences the statistical methods we choose. Before applying a method, we should understand whether the data is approximately normal, skewed, or follows another type of distribution.

↑ Top

2. Probability Distributions

Uniform Distribution

In a uniform distribution, every value within a given range is equally likely to occur. Unlike a normal distribution, there is no central peak; the values are spread roughly evenly across the range.

A simple example is a fair die. Each of the six outcomes has a 1-in-6 chance of occurring. If we rolled the die many times, we would expect the results to be spread fairly evenly across the six values.

In the hospital dataset, the visits by the day of week exhibit a near uniform distribution.

healthaccess_data["day_of_week"].value_counts(normalize = "percent")
Enter fullscreen mode Exit fullscreen mode

Output:

day_of_week
Monday      14.22
Tuesday     13.98
Wednesday   14.21
Thursday    14.41
Friday      14.08
Saturday    14.64
Sunday      14.47
Name: proportion, dtype: float64
Enter fullscreen mode Exit fullscreen mode

We can see that each day accounts for roughly 14% of total visits, close to the 1-in-7 (about 14.3%) we'd expect if visits were split perfectly evenly across the week. Patients arrive at the hospital fairly consistently regardless of the day, which makes day of week a reasonable real-world example of a near-uniform distribution.

Formula
For a discrete uniform distribution across n equally likely outcomes, each outcome has the same probability:

P(x)=1n P(x) = \frac{1}{n}

With 7 days in a week, we'd expect each day to account for 17≈14.3%\frac{1}{7} \approx 14.3\% of visits under a perfectly uniform distribution, which lines up closely with what we see in the data.

↑ Top

Binomial Distribution

A binomial distribution describes the number of times a particular outcome, or success, occurs in a fixed number of independent trials.

For the binomial distribution, each trial must have only two possible outcomes. These could be;

  • Success / failure
  • Yes / no
  • Admitted / not admitted
  • Died / survived etc.

Going to our hospital dataset, if we wanted to estimate how many of the next 50 patients would be admitted,
we could use a binomial distribution to model the number of admissions among those 50 patients.
Each patient gives us one trial:

Inpatient (Admitted) or Outpatient (Not Admitted)

First, we would need to get the proportion of inpatient visits;

inpatient_proportion = (healthaccess_data["visit_class"] == "Inpatient").mean()
print(f"Proportion of inpatient visits: {inpatient_proportion:.2%}")
Enter fullscreen mode Exit fullscreen mode

Output:

Proportion of inpatient visits: 28.56%
Enter fullscreen mode Exit fullscreen mode

Having got that, now we can answer the question:

What is the probability that exactly 10 out of the next 50 patients will be admitted?

from scipy.stats import binom

n = 50
k = 10
p = inpatient_proportion

probability = binom.pmf(k, n, p)
print(f"Probability of exactly 10 inpatients: {probability:.4f}")
Enter fullscreen mode Exit fullscreen mode

Output:

Probability of exactly 10 inpatients: 0.0533
Enter fullscreen mode Exit fullscreen mode

The result confirms that out of the next 50 patients, there's roughly a 5.3% chance that exactly 10 of them are inpatient.

The important distinction is that the binomial distribution is concerned with how many successes occur, rather than which individual patients experience the outcome.

↑ Top

Poisson Distribution

A Poisson distribution is used to model the number of times an event occurs within a fixed interval of time or space.

Unlike the binomial distribution, we are not starting with a fixed number of trials. Instead, we are counting how many times an event occurs during an interval.

For example, consider the number of patients arriving at the hospital daily.

One day might have:

  • 20 visits
  • another day might have 100
  • another might have 30

If patient visits occur independently and the average arrival rate is reasonably stable, a Poisson distribution can be used to model the number of arrivals per day.

We might then ask:

What is the probability that 80 patients arrive at the hospital the next day?

Assuming the ~ 89000 visits in our hospital data were evenly spread across 3 years, our daily average visits would be ~62 visits/day

To answer the question above, we would then proceed as follows:

from scipy.stats import poisson

k = 70
lam = 62 

probability = poisson.pmf(k, lam)
print(f"Probability of exactly 70 arrivals: {probability:.4f}")
Enter fullscreen mode Exit fullscreen mode

Output:


Probability of exactly 70 arrivals: 0.0290
Enter fullscreen mode Exit fullscreen mode

So on any given day, there's roughly a 2.76% chance of seeing exactly 70 arrivals.

The Poisson distribution is useful for count data, especially when we are counting events over a defined period of time or area, such as the visits per day or transactions per hour.

↑ Top

Why Distributions Matter

The shape of a distribution influences almost every decision we make with our data afterward.

  • It tells us which summary statistics to trust. The mean can be misleading for a skewed variable like bill amount, while it works well for an approximately normal variable like age.
  • It tells us how to spot unusual observations. The empirical rule and z-scores rely on approximate normality, while the IQR rule works regardless of shape.
  • It tells us which statistical methods are appropriate. Many statistical tests and models assume the underlying data is approximately normally distributed. When that assumption doesn't hold, a different approach is usually needed.

Recognizing the distribution a variable follows shapes the analytical decisions that come after it.

Conclusion

Distributions describe the shape of our data, and that shape carries information. Understanding distributions gives us a connection between descriptive statistics, which describes the data we already have, and probability, which lets us reason about data we haven't observed yet.

Next in this series, we will look at probability, and how it lets us move from describing what has happened to what is likely to happen.

↑ Back to Top

Top comments (0)