Bayes' Theorem In Layman Terms

DEFINITION

Bayes' Theorem states that the conditional probability of an event, based on the occurrence of another event, is equal to the likelihood of the second event given the first event multiplied by the probability of the first event.

Blaa Blaa Blaa - I find definitions to be strange, only after understanding the concept I do understand the definition.

Let's break it down and understand it one step at a time...

Marginal Probability - P(A)

If a random variable is independent then the probability of the event is irrespective of the outcomes of other random variables. In simple words, it's like looking at the probability of something occurring without taking into account any other factors.

Joint Probability - P(A,B)

The probability of 2 or more simultaneous events happening together. Eg Probability of watching TV and Eating.

Conditional Probability - P(A|B)

Probability of one (or more) event given the occurrence of another event. Eg the probability of your father having dessert given that tomorrow he is having a diabetes test is very low. If you notice carefully if there is no diabetes test tomorrow then the probability would have been almost 100%.

Expressing Joint Probability In Terms of Conditional Probability

$$P(A,B) = P(A|B) * P(B)$$

Note : P(A,B) = P(B,A) (Symmetrical)

Expressing Conditional Probability In Terms of Joint Probability

$$P(A|B) = P(A,B) / P(B)$$

Note : P(A|B) != P(B|A) (Not Symmetrical)

Finally our Bayes' Theorem using the above equations

$$P(A|B) = (P(B|A) * P(A))/P(B)$$

The numerator P(B|A) * P(A) is the joint probability equation given in (1)
P(A|B) ===> Posterior Probability
P(B|A) ===> Likelihood
P(A) ===> Prior Probability
P(B) ===> Evidence

To understand the above equation with an example

Question: What is the probability that there is fire given that there is smoke?

$$P(Fire|Smoke)=P(Smoke|Fire)*P(Fire)/P(Smoke)$$

P(Fire|Smoke) ===> Posterior Probability
P(Smoke|Fire) ===> Likelihood
P(Fire) ===> Prior Probability
P(Smoke) ===> Evidence

The probability of fire given that there is smoke is equal to the likelihood multiplied by the probability of fire divided by the probability of smoke. And this is Bayes' theorem to understand its use-case better read further.

Where is Bayes' Theorem used and why Bayes' Theorem

One very common space where you can find the theorem applied is in the evaluation of medical diagnostic tests.

Let us consider a diagnostic test that determines whether a person has a lesion that is malignant or not.

From observation, it is given that

$$P(Test=Positive|Malignant=True) = 0.85$$

The above statement means that the probability of diagnostic test results being Positive given that he/she has a malignant tumour is 85%.

What will be a normal person's understanding of the above probability???
If a person takes this diagnostic test and the result turns out to be Positive since the above statement shows that for someone with a Malignant tumour, the test detects 85% per cent correctly there is a good chance that the person assumes that he/she might have a malignant tumour and that's scary, right?

Now let's look at what Bayes' got to say about it
P(Malignant=True|Test=Positive), this is what we are going to analyse using Bayes' theorem

There are a few assumptions that we have to make

$$P(Malignant) = 0.0002$$

$$P(Test=Positive) = 0.05016$$

This assumption means that on average only 1 in 5000 will have malignant tumours and the probability of the test is positive regardless of whether the person has a malignant tumour or not is 0.05016.

$$P(Malignant=True | Test=Positive) = P(Test=Positive|Malignant=True) * P(Malignant=True) / P(Test=Positive)$$

Plugging the values that we have

$$P(Malignant=True | Test=Positive) = 0.85 * 0.0002 / 0.05016$$

P(Malignant=True | Test=Positive) = 0.003389

Wait what... this is a terrible diagnostic test because the the above probability shows that if this diagnostic test for a malignant tumour turns out to be True the probability of it being correct is only 0.33 per cent.
Note: This result was obtained on a few assumptions and if those assumptions are verified and updated the result could change.

CONCLUSION

Bayes' theorem is a significant contribution to the field of statistics and is widely used in machine learning. Bayes' Theorem provides a systematic way to update prior probabilities with new information or evidence. In other words, it helps us adjust our beliefs about the likelihood of an event occurring based on the data we observe.

LinkedIn : https://www.linkedin.com/in/praveenr2998

Comments

[Sri Vishnuvardhan A]

Nice article @praveenr2998 .

[praveenr]

Thanks @vishnuswmech

[thssgani]

"If a person takes this diagnostic test and the result is Positive there is an 85% chance that the person has a malignant tumor and that's scary, right?" - bro shouldn't this statement be like: If a person having a malignant tumor, takes this diagnostic test, there is only 85% chance that the result will be positive. because the given event is having the tumor and the probability of having the results positive is being calculated i.e., P(True Positives) = 85% which means P(false negatives) = 15%

[praveenr]

Thanks for the feedback bro, I have rephrased it.