Yadav Prasad G B

Posted on Oct 19

Discrete Random Probability

#maths

Random Variable

An assignment of a value to every possible outcome.
A function from sample space to the real numbers. It is of two types: discrete and continuous random variables.
It is denoted by X, and its numeric value is denoted by x.

For Example:

Tossing a coin once may land either Heads or Tails :

S = {H, T}

A random variable ( X ) can be defined as:

(X = 1) if it is Heads
(X = 0) if it is Tails

We can also have more than one random variable in an experiment. The diagram below takes a random data point and maps it to the height scale (in inches). We can also map its weight (in kg).

Probability Mass Function (PMF) :

Probability Mass Function is a function that gives the probability that a discrete random variable takes a specific value. It is denoted by P(X = x) probability that X takes the value x.

Properties of PMF:
0≤P(X=x)≤1 for all possible values x.
The sum of probabilities for all possible values of x : ∑x P(X=x)=1.

we use PMF to generally study how probabilities are distributed across all possible outcomes of a discrete random variable.

For Example : A die is rolled and the following sample space are {1, 2, 3, 4, 5, 6} assume that the random variable X is equals to the values in the sample space so the PMF for this experiment would be :

P(X = 1) = 1/6 P(X = 2) = 1/6 P(X = 3) = 1/6
P(X = 4) = 1/6 P(X = 5) = 1/6 P(X = 6) = 1/6

Binomial PMF

Binomial PMF is used to determine the probability of k successes in n independent events where each trial has only two outcomes.

if we take the coin toss example we can assign some probability value p if heads lands else 1 - p if tails lands and taking X = number of success in n trails.

the formula for this is given by,

P(X=k)= n!/(k!(n-k)!) * p^k(1−p)^n−k

why bothering finding the combination of k different sequences of occurring heads ? Let us take an example of finding the P(X = 2) given the number of coin tosses are 4.

n = 4, k = 2, X = p if heads appears else (1-p) if tails appears,

P(X = 2) = P(HHTT) + P(HTHT) + P(HTTH) + P(THHT) + P(THTH) + P(TTHH) = 6 * p^2 * (1-p)^2 which is equal to 4!/2!*(4-2)! * p^2 * (1-p)^2

Expectation

It is the average values of a random variable that you will expect when performing an experiment repeatedly.

it is denoted by E[X], generally finds the center of gravity of PMF

If a random variable X can take values x1,x2,x3,...x with probabilities P(X=xi) then the expected value is:
E[X]=∑xi P(X=xi)

Let us take an example, You are playing a Game where the chances of winning $1 is 1/6 and chances of winning $2 is 1/2 and chances of winning $4 is 1/3, so much the player should bet in-order to break even in the long term ??

E[X] = 1/6 * 1 + 1/3 * 2 + 1/3 * 4 = 2.5

therefore the player can bet upto $2.5 per game to break even in the long term if the player bets more than the expected value then he's more likely to be in loss after n games.

But how does expectation determines the center of gravity of a PMF, lets see this visually

Properties of Expectations

Let X and Y be random variables, and a, b, c be constants.

1. Linearity of Expectation
E[aX + bY + c] = a E[X] + b E[Y] + c

Expectation distributes over addition and scalar multiplication. This holds even if X and Y are not independent.

Example:

If E[X] = 2 and E[Y] = 3,

E[2X + 5Y + 1] = 2·2 + 5·3 + 1 = 22

2. Expectation of a Constant
E[c] = c

A constant always takes the same value, so its expected value is itself.

3. Expectation of a Sum
E[X + Y] = E[X] + E[Y]

A special case of linearity: the mean of the sum equals the sum of the means.

4. Independent Random Variables
If X and Y are independent:

E[XY] = E[X] · E[Y]

5. Non-Negativity
If X ≥ 0, then E[X] ≥ 0.

6. Function of a Random Variable
For any function g(X):

E[g(X)] = Σ g(x) · P(X = x)

Example

Let X be the number shown on a fair die: X ∈ {1, 2, 3, 4, 5, 6}

E[X] = ΣPX(x) . x = (1/6) (1 + 2 + 3 + 4 + 5 + 6)/6 = 3.5

Now if Y = 2X + 1:

E[Y] = 2 E[X] + 1 = 2·3.5 + 1 = 8

Variance Properties for Discrete Random Variables (PMF)

Variance measures how much the values of a random variable differ from the expected mean. It tells us the spread or dispersion of data around the mean.And the square root of the variance is called the standard deviation.

The formula for discrete random variable is given by,
Var(X) = Σ (xᵢ - E[X])² · P(X = xᵢ)

Properties of Variance

Let X and Y be discrete random variables and a, b, c be constants.

1. Variance of a Constant

Var(c) = 0

A constant does not vary.
Example: If X = 5 always, then Var(X) = 0.

2. Scaling Property

Var(aX) = a^2 * Var(X)

Multiplying a random variable by a constant scales variance by a^2.
Example: If Var(X) = 2 and Y = 3X, then Var(Y) = 3^2 * 2 = 18.

3. Adding a Constant

Var(X + b) = Var(X)

Adding a constant shifts the distribution but does not change spread.
Example: Var(X) = 2, then Var(X + 5) = 2.

4. Variance of a Sum (Independent Random Variables)

Var(X + Y) = Var(X) + Var(Y) (if X and Y are independent)

For independent variables, total variance is the sum of individual variances.
If not independent: Var(X + Y) = Var(X) + Var(Y) + 2 * Cov(X,Y)

5. Variance of a Linear Combination

Var(aX + bY) = a^2 * Var(X) + b^2 * Var(Y) + 2ab * Cov(X,Y)

If X and Y are independent, Cov(X,Y) = 0.

6. Non-Negativity

Var(X) ≥ 0

Variance is always zero or positive, since it is the average of squared deviations.

7. Variance in Terms of PMF

Var(X) = Σ xi^2 * P(X=xi) - (Σ xi * P(X=xi))^2

First term: expected value of X²
Second term: square of the mean
Difference gives the spread around the mean

Numerical Example

Discrete random variable X with PMF:

x_i	1	2	3
P(X=x_i)	0.2	0.5	0.3

Step 1: Compute the mean

E[X] = 1*0.2 + 2*0.5 + 3*0.3 = 2.1

Step 2: Compute E[X²]

E[X^2] = 1^2*0.2 + 2^2*0.5 + 3^2*0.3 = 4.9

Step 3: Compute variance

Var(X) = E[X^2] - (E[X])^2 = 4.9 - (2.1)^2 = 0.49

Step 4: Interpretation

Variance = 0.49 → small spread around the mean
Standard deviation = √0.49 = 0.7 → average deviation from the mean

Joint PMF

a Joint PMF describes the probability distribution of two discrete random variable X and Y simultaneously.It hints you the what will be the probability that X takes the value xi and Y takes the value yi.

Formally, for discrete random variables X and Y:

pX,Y(xi,yj) = P(X=xi,Y=yj)

Properties of Joint PMF

Non-negativity:

p_X,Y(xi, yj) >= 0 for all xi, yj
Sum of all probabilities = 1:

Σi Σj pX,Y(xi, yj) = 1
Marginal PMF:

You can get the individual distributions of X or Y by summing over the other variable:

For X: pX(xi) = Σj p_X,Y(xi, yj)
For Y: pY(yj) = Σi p_X,Y(xi, yj)

Conditional Probability (under Joint PMF)

For two discrete random variables X and Y, the conditional probability of X = xi given Y = yj is:

P(X = xi | Y = yj) = P(X = xi AND Y = yj) / P(Y = yj)

Numerator: joint probability that both events happen: P(X = xi, Y = yj)
Denominator: probability of the condition: P(Y = yj)

It answers the question:

“If I know Y = yj has occurred, what is the probability that X = xi occurs?”

Independent Random Variable

In general given the random variable the PMF formula is given by,

P(X=x,Y=y,Z=z) = P(X=x)⋅P(Y=y∣X=x)⋅P(Z=z∣X=x,Y=y)

Random variables X, Y, Z are independent if,

P(X=x,Y=y,Z=z) = P(X=x) * P(Y=Y) * p(Z=z)

and so, PX|Y(x|y) = PX(x)

The Hat Problem

N people throw their hats in a box and then pick one at random X is the number of people who get their own hat.

-Find E[X]

xi = 1 if i selects own hat else 0

X = x1 _ x2 + x3 + .... + xn

p(xi = 1) = 1/n [each person will have probability of 1/n in picking own hat]

E[xi] = 1/n * 1 + (1 - 1/n) * 0 = 1/n
E[X] = n * 1/n = 1

so expected mean is 1 meaning that only one person in most cases will get their hat back!

let's calculate variance

Var(X) = E[X^2] - (E[X])^2

E[X]^2 = 1
E[X^2] = Σ xi^2 + Σxixj (where i!=j)

E[xi^2] = E[xi] = 1/n

E[X^2] = n * 1/n + 1/n * 1/(n-1) * (n^2 - n)

why 1/n * 1/(n-1) * n^2 - n ???

n^2 - n (since n^2 elements in total and we remove elements where i = j)
if P(X1X2 = 1) = P(X1 = 1)P(X2 = 1| X1 = 1) then,
it should be equal to 1/n * 1/(n - 1).

thus E[X^2] = 1 + 1 = 2

Var(X) = 2 - 1 = 1

thus having E[X] = 1 and Var(X) = 1 we can inference that there will be higher chance of k = 0, 1, or 2 being the number of person getting their own hat back.

DEV Community