Nate Silver draws on Bayes’ Theorem to forecast election outcomes. He uses electoral polls, weighed according to their proven accuracy, as evidence to update the probability that a candidate will win. His predictions for the 2008 and 2012 US presidential elections have been spot on. In the last few weeks before the 2012 election, while most forecasters were expecting a highly uncertain neck-and-neck outcome, he gave Obama more than 90% chance of winning.
Bayes’ Theorem has three inputs:
- Base Rate, BR: Prior probability that the hypothesis is true
- True Positive Rate, TPR: Probability of the evidence, given that the hypothesis is true
- False Positive Rate, FPR: Probability of the evidence, given that the hypothesis is false.
The output is the probability that the hypothesis is true, given the evidence:
Silver uses electoral polls in the same way as the father uses expert coaches in our child footballer example. Silver asks: What is the probability that a candidate will win, given the poll results? The father asks: What is the probability that my child will become a football star, given the coach’s response?
In our example we have: BR=0.1%, TPR=100%, FPR=5%, hence PP=2%. If the question is posed to a second coach, 2% becomes the new BR. Assuming the second coach is independent of the first, and equally accurate, PP is updated to 29%. And so on. The accumulation of accurate evidence leads to the truth.
You can create your own examples. Pick any hypothesis and any evidence which might be related to it. In his recent book (hat tip to Paul Q for pointing this out), Silver has his own examples (Chapter 8, p. 243). The first one is:
Hypothesis: Your spouse is cheating on you. Evidence: You found a strange pair of underwear in a drawer.
- What is the prior probability of your spouse cheating on you? Apparently, 4% of spouses cheat in any given year. So BR=4%.
- What is the probability of finding strange underwear, if your spouse is cheating on you? You could say it is quite high, but then presumably not very high, as your spouse is likely to have been very careful not to leave behind suspicious traces. So let’s say TPR=50%.
- What is the probability of finding strange underwear, if your spouse is not cheating on you? You could say it is quite small, but it is probably not too small, as you can think of a number of fortuitous circumstances under which it might have happened. So FPR=5%.
Result: The probability that your spouse is cheating on you, after you found strange underwear in a drawer, is 29%. Uncomfortably high, but you may live with it.
Silver’s example reminds me of the Othello effect in Massimo Piattelli-Palmarini’s Inevitable Illusions. Silver’s spouse found underwear in a drawer. But Othello saw his cherished handkerchief in Cassio’s lodgings. How on earth did that happen? Sure, an adulterous Desdemona would have been foolish to leave the handkerchief with Cassio, but hey: Frailty, thy name is woman! So Othello’s TPR is likely to be higher than 50%. Let’s say it is 80%. More importantly, Cassio has the handkerchief, and that’s a fact. Did Desdemona lose it, and Cassio found it? Did someone plant it in Cassio’s lodgings? Highly unlikely! Poor Othello’s FPR is painfully low. Let’s say it is 1%. The handkerchief in your lieutenant’s lodgings is a much more accurate piece of evidence than a pair of underwear in a drawer. Even if Othello sticks to a 4% Base Rate, he is right in concluding that Desdemona is betraying him with 77% probability. Moreover, if the devious Iago manoeuvres him into the Prior Indifference Fallacy – where Othello disregards the prior probability of betrayal and gives it a 50/50 chance – Desdemona becomes a bitch with 99% probability: “I don’t know if Desdemona is betraying me or not, but the expert Iago says she is, and I believe him.” The same, by the way, would happen to Silver’s spouse: under prior indifference, PP=91%.
This is the Tragedy of Othello, the Moor of Venice. But the Comedy of Othello could have gone in entirely different directions. Imagine Othello was so spellbound by Desdemona that he refused to even consider the possibility of betrayal. “There is no way my sweetheart would betray me. Period.” This is Faith: a prior certainty that no amount of evidence, however accurate, can assail. If BR=0%, then PP=0% irrespective of TPR and FPR. Had he found Desdemona in bed with Cassio, a love struck Othello would have reacted like the English aristocrat who, finding himself in the same awkward situation while giving visitors a tour of his castle, kept his composure and said: “…and this is I, in bed with my wife.” Alternatively, when confronted by Othello to explain why the handkerchief ended up with Cassio, Desdemona could have said: “Darling, imagine for a second that I was betraying you with Cassio. Do you really think I would be so dumb as to leave our special handkerchief with him? C’mon!” To which Othello would have replied: “Of course not, honey. Sorry for even bringing that up.” TPR=0% also implies PP=0%, irrespective of BR and FPR.
But you don’t need absurd Faith to have a small PP. As long as either BR or TPR are very small (“OK, never say never, but I bet you a million to one that Desdemona does not betray me!”), PP will be small: Othello will trust Desdemona and they will live happily ever after. Unless, that is, Othello is confronted with irrefutable evidence. If he sees her in bed with Cassio, he is left with one inescapable conclusion: FPR=0% implies PP=100%, however small BR and TPR. Evidence that admits no False Positives is a Smoking Gun.
