Bayes Theorem

Table of Contents

Bayes Theorem formalizes the idea that the probability of a hypothesis given the evidence is, in part, a function of the probability of the evidence given the hypothesis.
Main forms of Bayes Theorem:
- The Comparative Form formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
- The Non–comparative Form quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal.

The Comparative Form of Bayes Theorem formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
Specifically, it defines the numeric probability of a hypothesis in light of the evidence, based on:
- the probability of the evidence given competing hypotheses,
- the probability of competing hypotheses apart from the evidence.
Bayes Theorem, Comparative Form for two hypothesis.
- H₁ is the hypothesis under consideration
- H₂ is a competing hypothesis
- Either H₁ or H₂ is true but not both
- E is the evidence
- P(- – -) means the probability that – – –
- P( – – – | …… ) means the probability that – – – given that …..
  - View Conditional Probability

You randomly select one of two decks of cards.
- One deck is standard deck of 52 cards, half red, half black.
- The other consists of only hearts and diamonds, making all the cards red.
The probability the selected deck is all red cards = ½
You randomly draw a card from the selected deck. The card is red.
Per Bayes Theorem, the probability the deck is all-red given that you drew a red card = ⅔
The ⅔ probability follows from the following, using Bayes Theorem.
- Before you selected the red card, the probability that the deck was all-red = ½
- The probability of drawing a red card given that the deck is all-red = 1.0
- The probability of drawing a red card given that the deck is half-red = ½

Abbreviations
- H₁ = hypothesis the deck is all red cards
- H₂ = hypothesis the deck is standard.
- E = the evidence that you selected a red card.
Prior Probabilities, before you select a card.
- Prior probability of H₁ = P(H₁) = 0.5
- Prior probability of H₂ = P( H₂) = 0.5.
Likelihoods of E given the hypotheses
- Probability of E given H₁ = P(E|H₁) = 1.0
- Probability of E given H₂ = P(E|H₂) = 0.5
Results: Posterior Probabilities
- Probability of H₁ given E = P(H₁|E) = 2/3
- Probability of H₂ given E = P(H₂|E) = 1/3
So,
- Probability of all red cars = P(H₁|E) = 2/3

The probability that the deck is standard increases from 50% to 67% in light of the evidence.

Issue
- You test positive for a disease. What’s the probability you’re sick?
Given
- Sensitivity of the test = P(positive | sick) = 98.1%
- Specificity of the test = P(negative | not sick) = 99.6%.
- Prevalence of disease in the population = P(sick) = 5%
Calculate
- P(sick | positive) = ?

Abbreviations
- H₁ = You’re sick
- H₂ = You’re not sick
- E = You test positive
Prior Probabilities, based on prevalence:
- Prior probability of H₁ = P(H₁) = 0.05
- Prior probability of H₂ = P( H₂) = 0.95
Likelihoods of E given the hypotheses
- Probability of E given H₁ = P(E|H₁) = 0.981
  - based on the sensitivity of the test
- Probability of E given H₂ = P(E|H₂) = 1 – 0.996 = 0.004
  - based on the specificity of the test
Results: Posterior Probabilities
- Probability of H₁ given E = P(H₁|E) = 0.928
- Probability of H₂ given E = P(H₂|E) = 0.072
So
- P(sick | positive) = P(H₁|E) = 92.8%

Issue
- You test negative for a disease. What’s the probability you’re okay?
Given
- Sensitivity of the test = P(positive | sick) = 98.1%
- Specificity of the test = P(negative | not sick) = 99.6%.
- Prevalence of the disease in the population = P(sick) = 5%
Calculate
- P(not sick | negative) = ?

Abbreviations
- H₁ = You’re sick
- H₂ = You’re not sick
- E = You test negative
Prior Probabilities, based on prevalence:
- Prior probability of H₁ = P(H₁) = 0.05
- Prior probability of H₂ = P( H₂) = 0.95
Likelihoods of E given the hypotheses
- Probability of E given H₁ = P(E|H₁) = 1 – 0.981 = 0.019
  - based on the sensitivity of the test
- Probability of E given H₂ = P(E|H₂) = 0.996
  - based on the specificity of the test
Results: Posterior Probabilities
- Probability of H₁ given E = P(H₁|E) = 0.001
- Probability of H₂ given E = P(H₂|E) = 0.999
So
- P(not sick | negative) = P(H₂|E) = 99.9%

Hypothesis true or false

Multiple competing hypotheses

The Non-comparative Form of Bayes Theorem quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal. That is, the lower P(E), the greater P(H|E).

View Bayesian Estimation for the application of the Non-comparative Form to statistical estimation.