Part 4: Understanding Probability in Data Science

Statistics for Data Science Series

Part 1: Why Statistics Matters in Data Science

Part 2: Descriptive Statistics: A Beginner's Guide

Part 3: Understanding Distributions in Data Science

Part 4: Understanding Probability in Data Science

Dataset: healthaccess_data_mini.csv

Introduction

In out last article about Distributions, we saw how recognizing the shape of our data helps us decide the statistics to trust and the tools apply. We looked at probability distributions like Binomial and Poisson introducing us to the concept of probability

In this article, we will go back to the basics of probability itself, what it means, how it's calculated, and the rules that probability builds on. By the end, we'll circle back to Binomial and Poisson and see exactly what makes them "probability" distributions in the first place.

---

Table of Contents

• What Probability Means
• Outcomes and Events
• Complementary Events
• The Addition Rule (OR)
• Independent vs. Dependent Events
• Conditional Probability
• Expected Value
• Random Variables: Discrete vs. Continuous
• How Probability Distributions Describe Outcomes
• Conclusion

---

What Probability Means

Probability is a way of measuring how likely something is to happen. It's a number that captures our uncertainty about an outcome, before we know how things actually turn out.

Even without thinking about it, we use probability in our day to day lives:

"I'll probably be late to work if I pass by the store"

"It is likely to rain today"

"There's a good chance i'll catch a cold after being rained on"

"It is highly unlikely that I will sleep through my alarm"

---

Probability Between 0 and 1

Every probability is a number between 0 and 1.

• A probability of 0 means the event cannot happen.
• A probability of 1 means the event is certain to happen.
• Anything in between reflects how likely the event is, the closer to 1, the more likely.

Probabilities are also often expressed as percentages, a probability of 0.5 is the same as saying there's a 50% chance.

Formula

$$P(A) = \frac{\text{number of favorable cases}}{\text{total number of cases}}$$

In our hospital dataset, to know how likely it is that a randomly selected visit was an inpatient visit;

1. First, we identify what can happen i.e. the outcomes: the visit can be Inpatient or Outpatient.

2. Then, we identify what we want to know about i.e. the event: whether the visit was Inpatient.

You can follow along using the same dataset, download it here, or load it directly with pd.read_csv()

In Python, we can calculate it as follows:

p_inpatient = (healthaccess_data["visit_type"] == "Inpatient").mean()

print(f"Probability of inpatient: {p_inpatient:.4f}")

Output:

Probability of inpatient: 0.2857

So, the probability of selecting an inpatient visit at random is approximately 0.2857. In other words, there is about a 28.57% chance that a randomly selected visit from this dataset would be an inpatient visit.

Outcomes and Events

Probability questions are usually about the particular result or group of results we are interested in, not every possible result. Two terms that are important to understand, which we encounter often in probability are outcomes and events.

An outcome is a single possible result.

When rolling a six-sided die, the possible outcomes are:

1, 2, 3, 4, 5, 6

An event is the outcome, or group of outcomes, that we are interested in.

If we ask, “What is the probability of rolling an even number?”, the event is rolling an even number. It includes three possible outcomes:

2, 4, 6

From the hospital visits example above, we can now identify the outcomes and event more clearly.

The two possible outcomes for a visit are:

• Outpatient: 2,143 visits
• Inpatient: 857 visits

If our question is “What is the probability that a randomly selected visit was an inpatient visit?”, then being an inpatient visit is the event we are interested in. The 857 inpatient visits are the cases that satisfy that event.

---

Complementary Events

The complement of an event is everything that is not that event.

Two events are complementary when they are the only two possible possibilities: if one happens, the other cannot happen, and vice versa.

The Complement Rule states that the probabilities of an event and its complement always add up to 1, or 100%.

$$P(A') = 1 - P(A)$$

where:

• $P(A′)$ is the probability of the complement of event A.
• $P(A)$ is the probability of event A.

Using our inpatient probability:

p_outpatient = 1 - p_inpatient
print(f"Probability of outpatient: {p_outpatient:.4f}")

Output:

Probability of outpatient: 0.7144

Complementary events have three key properties: they are mutually exclusive (they cannot occur at the same time), exhaustive (together they cover all possible outcomes), and their probabilities always add up to 1.

---

The Addition Rule (OR)

The addition rule tells us the probability that either of two events happens.

Mutually exclusive, exhaustive pairs (an event and its complement) will always add up to 1, so we can always add their probabilities.

Formula (mutually exclusive events)

$$P(A \text{ or } B) = P(A) + P(B)$$

Sometimes however, the events we are interested in aren't mutually exclusive, and can overlap. Take these 2 events for example:

• "the patient was inpatient"
• "the patient died."

These aren't mutually exclusive because some patients could be inpatient and still die. Simply adding their probabilities would double-count that overlap.

Formula (general case)

$$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$$

We need to know the individual probabilities of being inpatient and dying, get the overlap, i.e. the probability of being both inpatient and having died and subtracting it from the overall probability.
In Python, we can do that as follows:

# Probability inpatient
p_inpatient = (healthaccess_data["visit_type"] == "Inpatient").mean()
print(f"P(inpatient): {p_inpatient:.4f}")

#Probability died
p_died = (healthaccess_data["discharge_outcome"] == "Died").mean()
print(f"P(died): {p_died:.4f}")

# The overlap(Both inpatient and died)
p_inpatient_and_died = ((healthaccess_data["visit_type"] == "Inpatient") &
    (healthaccess_data["discharge_outcome"] == "Died")).mean()
print(f"P(inpatient and died): {p_inpatient_and_died:.4f}")

# Actual probability (Subtracting the overlap)
p_inpatient_or_died = p_inpatient + p_died - p_inpatient_and_died
print(f"P(inpatient or died): {p_inpatient_or_died:.4f}")

Output:

P(inpatient): 0.2856
P(died): 0.0343
P(inpatient and died): 0.0194
P(inpatient or died): 0.3005

Plugging these into the formula:

$$P(\text{inpatient or died}) = 0.2856 + 0.0343 - 0.0194 = 0.3005$$

So roughly 30% of patients either were inpatient, died, or both. Without subtracting the overlap, we'd have counted those patients twice and landed on a higher, wrong number.

---

Independent vs. Dependent Events

Two events are independent if the outcome of one doesn't affect the probability of the other. Two events are dependent if the first outcome changes the probability of the other.

A simple example of independent events is tossing a coin twice. The result of the first toss does not affect the second. If the first toss is heads, the probability of getting heads on the second toss is still 50%.

For a dependent example, consider a bus with a scheduled arrival time and the traffic situation along its route.

Possible outcomes for the traffic are either heavy traffic or no traffic. The event we are interested in is: heavy traffic (Event A).

Possible outcomes for arrival are either the bus arrives on time or it arrives late. The event we are interested in is: the bus arrives late (Event B).

If Event A occurs, the probability of Event B increases. Knowing that there is heavy traffic therefore changes what we expect about the bus's arrival time, making the two events dependent.

The same idea applies to our hospital dataset. If knowing a patient's visit type(Inpatient or Outpatient) changes the probability that they have a recorded length of stay, then the two events are dependent.

For independent events, the probability of both events occurring is:

$P(A \text{ and } B) = P(A) \times P(B)$

This is known as the Multiplication Rule, and it only works when the two events are independent. When events are dependent, we need to account for how one event changes the probability of the other. This can be explained by conditional probability.

---

Conditional Probability

Conditional probability asks: given that we already know one thing happened, what's the probability of something else?

Formula

$$P(A \mid B) = \frac{P(A \text{ and } B)}{P(B)}$$

$P(A \mid B)$ reads as "the probability of A, given B."

Say we wanted to know the probability that a patient died, given that they were an inpatient, rather than the overall probability of death across all patients. That's a meaningfully different question, and likely a different number.

inpatients = healthaccess_data[healthaccess_data["visit_type"] == "Inpatient"]
p_died_given_inpatient = (inpatients["discharge_outcome"] == "Died").mean()
print(f"P(died | inpatient): {p_died_given_inpatient:.4f}")

Output:

P(died | inpatient): 0.0677

The overall probability of death is 3.43%, while the probability of death given an inpatient visit is 6.79%. Since knowing that a patient was an inpatient changes the probability of death, this suggests that admission status and mortality are dependent.

Rearranging this formula gives us the Multiplication Rule for dependent events, the probability of both A and B happening when they aren't independent:

$$P(A \text{ and } B) = P(B \mid A) \times P(A)$$

We've actually already calculated this. Using the inpatient and death probabilities above:

p_inpatient_and_died = p_died_given_inpatient * p_inpatient
print(f"P(inpatient and died): {p_inpatient_and_died:.4f}")

Output:

P(inpatient and died): 0.0193

Same result we'll use again in the addition rule above, just built from the conditional probability.

---

Expected Value

Expected value is the average outcome we'd anticipate if a random process were repeated many times. It tells what the long-run average looks like, not what happens in a single instance.

Formula:

$$E(X) = \sum xP(X=x)$$

where:

$E(X)$ is the expected value of the random variable $X$ .
$x$ is each possible value that $X$ can take.
$P(X=x)$ is the probability that $X$ takes the value $x$ .
$\sum$ means that we add these values together.

We can see this using our hospital data. If the probability of a patient being inpatient is 28.56%, then for the next 50 patients, we can estimate the expected number of inpatients using the binomial expected-value rule:

$$E(X) = np$$

where $n$ is the number of patients and $p$ is the probability of a patient being inpatient.

n = 50
p = p_inpatient

expected_inpatients = n * p

print(f"Expected number of inpatients: {expected_inpatients:.2f}")

Output:

Expected number of inpatients: 14.28

We wouldn't expect exactly 14.28 inpatients in any real batch of 50 patients, that's not even a whole number. What it tells us is that if we watched batch after batch of 50 patients come through, the average number of inpatients across all those batches would converge toward 14.28.

Expected value gives us a long-run average that can be used as a benchmark for what to expect, based on the probabilities of the possible outcomes.

---

Random Variables: Discrete vs. Continuous

A random variable is a variable whose value comes from a random process. It gives us a way to represent outcomes as numbers that we can calculate with.

For example, the number of inpatients among the next 50 patients is a random variable. It could be 0, 1, 2, and so on up to 50, but we don't know the value in advance. A patient's weight is also a random variable because we don't know its value until we measure it.

Random variables are usually represented by capital letters, such as $X$ , while a specific value is represented by lowercase $x$ . So $P(X = 10)$ means "the probability that the random variable $X$ takes the value 10."

There are two types of random variables:

• A discrete random variable can take specific, countable values. The number of inpatients among 50 patients is discrete: it can be 0, 1, 2, and so on, but never 6.5.
• A continuous random variable can take any value within a range. Weight is continuous: a patient could weigh 59.32 kg, 59.321 kg, or any other value within the range.

For a continuous variable, the probability of any exact value is technically zero because there are infinitely many possible values. Instead, we calculate the probability of a value falling within a range. For example, the probability that a patient's weight falls between 39.9 kg and 78.7 kg.

---

How Probability Distributions Describe Outcomes

A probability distribution assigns a probability to every possible outcome of a random variable. For a discrete variable, this is a probability mass function (PMF), giving $P(X = x)$ directly. For a continuous variable, it's a probability density function (PDF), describing probability across ranges rather than exact points.

Either way, a probability distribution is just a complete map of "how likely is each outcome," built from the same rules covered in this article: simple probability, complements, independence, and conditional probability.

---

Reconnecting to Binomial and Poisson

The binomial distribution is a PMF built directly from the multiplication rule for independent events: a fixed number of trials, each with two possible outcomes and a constant probability of success. When we calculated the probability of exactly 10 inpatients out of the next 50 patients, that's $P(X = 10)$ , computed using the rules above.

The Poisson distribution works the same way, but counts how often an event happens across an interval rather than across fixed trials. The average rate we used, daily patient arrivals, was itself an expected value, the same concept covered earlier in this article, applied to counts of events.

---

Conclusion

Probability gives us the tools to describe uncertainty with real numbers. The addition rule tells us the chance of either event happening, the multiplication rule tells us the chance of both, and conditional probability lets us account for when one event changes the likelihood of another. Complementary events remind us that every possible outcome, taken together, always adds up to 1.

Applied to our hospital data, these rules let us put an actual number on the relationship between admission status and mortality, showing exactly how much one affects the other.

---

Further Reading

If you want to go deeper than a single article can cover, two resources I find useful are:

• Think Stats by Allen Downey -- A free, Python-first introduction to probability and statistics using real datasets and code.
• Introduction to Probability by Joseph K. Blitzstein and Jessica Hwang --A more formal treatment of probability, with deeper coverage of probability rules and conditional probability.