# Bayes Theorem

#### Bayes Theorem

• Bayes Theorem formalizes the idea that the probability of a hypothesis given the evidence is, in part, a function of the probability of the evidence given the hypothesis.
• Main forms of Bayes Theorem:
• The Comparative Form formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
• The Noncomparative Form quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal.

#### Comparative Form of Bayes Theorem

• The Comparative Form of Bayes Theorem formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
• Specifically, it defines the numeric probability of a hypothesis in light of the evidence, based on:
• the probability of the evidence given competing hypotheses,
• the probability of competing hypotheses apart from the evidence.
• Bayes Theorem, Comparative Form for two hypothesis.
• H1 is the hypothesis under consideration
• H2 is a competing hypothesis
• Either H1 or H2 is true but not both
• E is the evidence
• P(- – -) means the probability that – – –
• P( – – – | …… ) means the probability that – – – given that …..
##### Artificial Example of Comparative Form
• You randomly select one of two decks of cards.
• One deck is standard deck of 52 cards, half red, half black.
• The other consists of only hearts and diamonds, making all the cards red.
• The probability the selected deck is all red cards = ½
• You randomly draw a card from the selected deck. The card is red.
• Per Bayes Theorem, the probability the deck is all-red given that you drew a red card = ⅔
• The ⅔ probability follows from the following, using Bayes Theorem.
• Before you selected the red card, the probability that the deck was all-red = ½
• The probability of drawing a red card given that the deck is all-red = 1.0
• The probability of drawing a red card given that the deck is half-red = ½

#### Calculating the Probability of All Red Cards

##### Using a Bayesian Calculator
• Abbreviations
• H1 = hypothesis the deck is all red cards
• H2 = hypothesis the deck is standard.
• E = the evidence that you selected a red card.
• Prior Probabilities, before you select a card.
• Prior probability of H1 = P(H1) = 0.5
• Prior probability of H2 = P( H2) = 0.5.
• Likelihoods of E given the hypotheses
• Probability of E given H1 = P(E|H1) = 1.0
• Probability of E given H2 = P(E|H2) = 0.5
• Results: Posterior Probabilities
• Probability of H1 given E = P(H1|E) = 2/3
• Probability of H2 given E = P(H2|E) = 1/3
• So,
• Probability of all red cars = P(H1|E) = 2/3

Online Bayesian Calculator

##### Using Bayes Formula

The probability that the deck is standard increases from 50% to 67% in light of the evidence.

#### Positive and Negative Test Results

##### Bayesian Calculation of P(sick | positive)
• Issue
• You test positive for a disease. What’s the probability you’re sick?
• Given
• Sensitivity of the test = P(positive | sick) = 98.1%
• Specificity of the test = P(negative | not sick) = 99.6%.
• Prevalence of disease in the population = P(sick) = 5%
• Calculate
• P(sick | positive) = ?
• Abbreviations
• H1 = You’re sick
• H2 = You’re not sick
• E = You test positive
• Prior Probabilities, based on prevalence:
• Prior probability of H1 =  P(H1) = 0.05
• Prior probability of H2 = P( H2) = 0.95
• Likelihoods of E given the hypotheses
• Probability of E given H1 = P(E|H1) = 0.981
• based on the sensitivity of the test
• Probability of E given H2 = P(E|H2) = 1 – 0.996 = 0.004
• based on the specificity of the test
• Results: Posterior Probabilities
• Probability of H1 given E = P(H1|E) = 0.928
• Probability of H2 given E = P(H2|E) = 0.072
• So
• P(sick | positive) = P(H1|E) = 92.8%
##### Bayesian Calculation of P(not sick | negative)
• Issue
• You test negative for a disease. What’s the probability you’re okay?
• Given
• Sensitivity of the test = P(positive | sick) = 98.1%
• Specificity of the test = P(negative | not sick) = 99.6%.
• Prevalence of the disease in the population = P(sick) = 5%
• Calculate
• P(not sick | negative) = ?
• Abbreviations
• H1 = You’re sick
• H2 = You’re not sick
• E = You test negative
• Prior Probabilities, based on prevalence:
• Prior probability of H1 = P(H1) = 0.05
• Prior probability of H2 = P( H2) = 0.95
• Likelihoods of E given the hypotheses
• Probability of E given H1 = P(E|H1) = 1 – 0.981 = 0.019
• based on the sensitivity of the test
• Probability of E given H2 = P(E|H2) = 0.996
• based on the specificity of the test
• Results: Posterior Probabilities
• Probability of H1 given E = P(H1|E) = 0.001
• Probability of H2 given E = P(H2|E) = 0.999
• So
• P(not sick | negative) = P(H2|E) = 99.9%

View Random Drug Test

#### Other Comparative Forms of Bayes Theorem

Hypothesis true or false

Multiple competing hypotheses

#### Non-comparative Form of Bayes Theorem

• The Non-comparative Form of Bayes Theorem quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal. That is, the lower P(E), the greater P(H|E).
• H is the hypothesis under consideration
• E is the evidence
• P(- – -) means the probability that – – –
• P( – – – | …… ) means the probability that – – – given that …..
• View Bayesian Estimation for the application of the Non-comparative Form to statistical estimation.