Bayes Theorem

Contents
Bayes Theorem
  • Bayes Theorem formalizes the idea that the probability of a hypothesis given the evidence is, in part, a function of the probability of the evidence given the hypothesis.
  • Main forms of Bayes Theorem:
    • The Comparative Form formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
    • The Noncomparative Form quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal.
Comparative Form
  • The Comparative Form of Bayes Theorem formalizes the principle that a hypothesis that better predicts the evidence is more probable, other things being equal.
  • Specifically, it defines the numeric probability of a hypothesis in light of the evidence, based on:
    • the probability of the evidence given competing hypotheses,
    • the probability of competing hypotheses apart from the evidence.
  • Bayes Theorem, Comparative Form for two hypothesis.
    • H1 is the hypothesis under consideration
    • H2 is a competing hypothesis
    • Either H1 or H2 is true but not both
    • E is the evidence
    • P(- – -) means the probability that – – –
    • P( – – – | …… ) means the probability that – – – given that …..
Artificial Example of Comparative Form
  • You randomly select one of two decks of cards.
    • One deck is standard deck of 52 cards, half red, half black.
    • The other consists of only hearts and diamonds, making all the cards red.
  • The probability the selected deck is all red cards = ½
  • You randomly draw a card from the selected deck. The card is red.
  • Per Bayes Theorem, the probability the deck is all-red given that you drew a red card = ⅔
  • The ⅔ probability follows from the following, using Bayes Theorem.
    • Before you selected the red card, the probability that the deck was all-red = ½
    • The probability of drawing a red card given that the deck is all-red = 1.0
    • The probability of drawing a red card given that the deck is half-red = ½
Calculating the Probability of All Red Cards
Using a Bayesian Calculator
  • Abbreviations
    • H1 = hypothesis the deck is all red cards
    • H2 = hypothesis the deck is standard.
    • E = the evidence that you selected a red card.
  • Prior Probabilities, before you select a card.
    • Prior probability of H1 = P(H1) = 0.5
    • Prior probability of H2 = P( H2) = 0.5.
  • Likelihoods of E given the hypotheses
    • Probability of E given H1 = P(E|H1) = 1.0
    • Probability of E given H2 = P(E|H2) = 0.5
  • Results: Posterior Probabilities
    • Probability of H1 given E = P(H1|E) = 2/3
    • Probability of H2 given E = P(H2|E) = 1/3
  • So,
    • Probability of all red cars = P(H1|E) = 2/3

Online Bayesian Calculator

Using a Probability Tree
Using Bayes Formula

The probability that the deck is standard increases from 50% to 67% in light of the evidence.

Realistic Example of Comparative Form

View Sensitivity, Specificity, Positive and Negative Predictive Values

Bayesian Calculation of P(sick | positive)
  • Issue
    • You test positive for a disease. What’s the probability you’re sick?
  • Given
    • Sensitivity of the test = P(positive | sick) = 98.1%
    • Specificity of the test = P(negative | not sick) = 99.6%.
    • Prevalence of disease in the population = P(sick) = 5%
  • Calculate
    • P(sick | positive) = ?
  • Abbreviations
    • H1 = You’re sick
    • H2 = You’re not sick
    • E = You test positive
  • Prior Probabilities, based on prevalence:
    • Prior probability of H1 =  P(H1) = 0.05
    • Prior probability of H2 = P( H2) = 0.95
  • Likelihoods of E given the hypotheses
    • Probability of E given H1 = P(E|H1) = 0.981
      • based on the sensitivity of the test
    • Probability of E given H2 = P(E|H2) = 1 – 0.996 = 0.004
      • based on the specificity of the test
  • Results: Posterior Probabilities
    • Probability of H1 given E = P(H1|E) = 0.928
    • Probability of H2 given E = P(H2|E) = 0.072
  • So
    • P(sick | positive) = P(H1|E) = 92.8%

Online Bayesian Calculator

Bayesian Calculation of P(not sick | negative)
  • Issue
    • You test negative for a disease. What’s the probability you’re okay?
  • Given
    • Sensitivity of the test = P(positive | sick) = 98.1%
    • Specificity of the test = P(negative | not sick) = 99.6%.
    • Prevalence of the disease in the population = P(sick) = 5%
  • Calculate
    • P(not sick | negative) = ?
  • Abbreviations
    • H1 = You’re sick
    • H2 = You’re not sick
    • E = You test negative
  • Prior Probabilities, based on prevalence:
    • Prior probability of H1 = P(H1) = 0.05
    • Prior probability of H2 = P( H2) = 0.95
  • Likelihoods of E given the hypotheses
    • Probability of E given H1 = P(E|H1) = 1 – 0.981 = 0.019
      • based on the sensitivity of the test
    • Probability of E given H2 = P(E|H2) = 0.996
      • based on the specificity of the test
  • Results: Posterior Probabilities
    • Probability of H1 given E = P(H1|E) = 0.001
    • Probability of H2 given E = P(H2|E) = 0.999
  • So
    • P(not sick | negative) = P(H2|E) = 99.9%

Online Bayesian Calculator

View Random Drug Test

Other Comparative Forms

Hypothesis true or false

Multiple competing hypotheses

Non-comparative Form of Bayes Theorem
  • The Non-comparative Form of Bayes Theorem quantifies the idea that the more unexpected a prediction, the more likely the hypothesis, other things being equal. That is, the lower P(E), the greater P(H|E).
  • H is the hypothesis under consideration
  • E is the evidence
  • P(- – -) means the probability that – – –
  • P( – – – | …… ) means the probability that – – – given that …..
  • View Bayesian Estimation for the application of the Non-comparative Form to statistical estimation.