Bayes Theorem

Outline

Bayes Theorem formalizes the idea that the probability of a hypothesis given the evidence is, in part, a function of the probability of the evidence given the hypothesis.
Main forms of Bayes Theorem:
- The Comparative Form formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
- The Non–comparative Form quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal.

The Comparative Form of Bayes Theorem formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
Specifically, it defines the numeric probability of a hypothesis in light of the evidence, based on:
- the probability of the evidence given competing hypotheses,
- the probability of competing hypotheses apart from the evidence.
Comparative Form of Bayes Theorem for Two Hypothesis:
- - H₁ is the hypothesis under consideration
  - H₂ is a competing hypothesis
  - Either H₁ or H₂ is true but not both
  - E is the evidence
  - P(- – -) means the probability that – – –
  - P( – – – | …… ) means the probability that – – – given that …..
    - View Conditional Probability

There are two decks of cards on the table, face down.
- One is a standard deck of 52 cards
- The other consists of 52 diamonds: four aces of diamonds, four kings of diamonds, and so on.
You randomly select one of the two decks, not knowing which. The probability you selected the deck of diamonds = 0.5.
You randomly draw a card from the selected deck. The card is a diamond.
According to Bayes Theorem, the fact that you drew a diamond makes it more likely that the selected deck is the deck of diamonds. In fact, per Bayes Theorem, the probability increased from 0.5 to 0.8.
The 0.8 probability is calculated from Bayes Theorem using three pieces of information:
- Before you drew the diamond, the probability that you had selected the deck of diamonds = 0.5.
- The probability of drawing a diamond from the deck of diamonds = 1.0
- The probability of drawing a diamond from the standard deck of cards = 0.25

Abbreviations
- H₁ = hypothesis the selected deck is a deck of 52 diamonds
- H₂ = hypothesis the selected deck is a standard deck of 52 cards
- E = the evidence, i.e. that you drew a diamond from the selected deck
Prior Probabilities, before you draw a card.
- Prior probability of H₁ = P(H₁) = 0.5
- Prior probability of H₂ = P( H₂) = 0.5.
Likelihoods of E given the hypotheses
- Probability of E given H₁ = P(E|H₁) = 1.0
- Probability of E given H₂ = P(E|H₂) = 0.25
Results: Posterior Probabilities
- Probability of H₁ given E = P(H₁|E) = 0.8
- Probability of H₂ given E = P(H₂|E) = 0.2

85 percent of taxis in a city are Green Cabs. The other 15 percent are Blue Taxis. A taxi sideswiped another car on a misty winter night and drove off. A witness testified the taxi was blue. The witness is tested under conditions like those on the night of the accident and she correctly identifies the color of the taxi 80% of the time. What’s the probability the sideswiper was a Blue Taxi?

Issue
- You test positive for a disease. What’s the probability you’re sick, i.e. P(sick | positive)?
Given
- Sensitivity of the test = P(positive | sick) = 98.1%
- Specificity of the test = P(negative | not sick) = 99.6%.
- Prevalence of disease in the population = P(sick) = 5%
Calculate
- P(sick | positive) = ?

Abbreviations
- H₁ = You’re sick
- H₂ = You’re not sick
- E = You test positive
Prior Probabilities, based on prevalence:
- Prior probability of H₁ = P(H₁) = 0.05
- Prior probability of H₂ = P( H₂) = 0.95
Likelihoods of E given the hypotheses
- Probability of E given H₁ = P(E|H₁) = 0.981
  - based on the sensitivity of the test
- Probability of E given H₂ = P(E|H₂) = 1 – 0.996 = 0.004
  - based on the specificity of the test
Results: Posterior Probabilities
- Probability of H₁ given E = P(H₁|E) = 0.928
- Probability of H₂ given E = P(H₂|E) = 0.072
So
- P(sick | positive) = P(H₁|E) = 92.8%

Issue
- You test negative for a disease. What’s the probability you’re not sick, i.e. P( not sick | negative)?
Given
- Sensitivity of the test = P(positive | sick) = 98.1%
- Specificity of the test = P(negative | not sick) = 99.6%.
- Prevalence of the disease in the population = P(sick) = 5%
Calculate
- P(not sick | negative) = ?

Abbreviations
- H₁ = You’re sick
- H₂ = You’re not sick
- E = You test negative
Prior Probabilities, based on prevalence:
- Prior probability of H₁ = P(H₁) = 0.05
- Prior probability of H₂ = P( H₂) = 0.95
Likelihoods of E given the hypotheses
- Probability of E given H₁ = P(E|H₁) = 1 – 0.981 = 0.019
  - based on the sensitivity of the test
- Probability of E given H₂ = P(E|H₂) = 0.996
  - based on the specificity of the test
Results: Posterior Probabilities
- Probability of H₁ given E = P(H₁|E) = 0.001
- Probability of H₂ given E = P(H₂|E) = 0.999
So
- P(not sick | negative) = P(H₂|E) = 99.9%

Two Competing Hypotheses

Single Hypothesis, True and False

Number n of Competing Hypotheses

Hypotheses are Discrete Random Variables 𝞱

Hypotheses are Continuous Random Variables 𝞱

The Non-Comparative Form of Bayes Theorem quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal. That is, the lower P(E), the greater P(H | E).

View Bayesian Estimation for the application of the Non-comparative Form to statistical estimation.

P(H|E) = P(H&E) / P(E)
- Definition of Conditional Probability
P(H|E) = P(H&E) / P((E&H) v (E&~H))
- Equivalence Rule
- E is logically equivalent to E&H v E&~H
P(H|E) = P(H&E) /( P(E&H) + P(E&~H) )
- Special Disjunction Rule
P(H|E) = P(H) P(E|H) /( P(H) P(E|H) + P(~H) P(E|~H) )
- General Conjunction Rule