The Inverse Fallacy is often illustrated using Tversky and Kahneman’s cab problem (also reviewed in Chapter 16 of Thinking, Fast and Slow):
A cab was involved in a hit and run accident at night. Two cab companies, the Green and the Blue, operate in the city. You are given the following data:
- 85% of the cabs in the city are Green and 15% are Blue.
- A witness identified the cab as Blue. The court tested the reliability of the witness under the same circumstances that existed on the night of the accident and concluded that the witness correctly identified each one of the two colours 80% of the time and failed 20% of the time.
What is the probability that the cab involved in the accident was Blue rather than Green?
The common answer is 80%: go along with the witness. As in our child footballer example, the “expert” rules the day. But it is the wrong answer. The true probability is half of that. Let’s see:
Hypothesis: The cab involved in the accident is Blue. Evidence: A witness says so.
- What is the prior probability that the cab is Blue? 15% of the cabs are Blue: BR=15%.
- What is the probability that the witness says the cab is Blue, if indeed it is Blue? The court says TPR=80%.
- What is the probability that the witness says the cab is Blue, if it is actually Green? The court says FPR=20%.
This is a case of symmetric evidence, with an equal probability of Misses and False Alarms: TPR=1-FPR. Under symmetry:
hence PP=41%. The mistake is due to the Inverse Fallacy, which is ultimately a Prior Indifference Fallacy. Under prior indifference (BR=50%), PP=TPR=80%.
What causes prior indifference? Why is it so immediately powerful? The answer can be found by contrasting the original cab problem with a slightly modified version, where the base information is changed to:
1a. Green cabs are involved in 85% of the accidents.
The modified version is formally identical to the original: the prior probability that the cab is Blue is still 15%. Had there been no witness to the accident, 15% would have been the obvious answer in both cases. But, after the witness testimony, the common answer in the modified version is much lower than 80% and close to the true 41% posterior probability.
Why is the witness testimony much less influential in the modified version? It is because 1a is not merely a statistical Base Rate: it is a causal Base Rate. 1a gives us a reason to believe that Blue cabs are less likely to be involved in the accident. In 1a we may not even know the proportion of Green and Blue cabs, but we know that Green cabs are much more accident prone than Blue cabs. So when the witness tells us that the cab was Blue we see the need to balance this piece of information with the fact that Green cabs are run by sloppy drivers.
Statistical Base Rates are not beliefs. Hence they are ignored: they are the unknown knowns that give power to experts, whether they are accurate – like the accident witness and the child footballer’s coach – or merely confident – like Dr. Doom and other unscrupulous forecasters. Under prior indifference, we are blinded by evidence, even when it is perfectly useless. Causal Base Rates, on the other hand, are beliefs. Hence they are not ignored, but are modified by evidence according to Bayes’ Theorem. Causal Base Rates prevent prior indifference and therefore, if correct, keep us closer to the true posterior probability.
Notice that nothing substantial would change if we had witnessed the accident ourselves, and were 80% sure that the cab was Blue. Despite our confidence, we should account for the fact that the Base Rate favours Green cabs and adjust our prediction accordingly. Like the expert coach in our child footballer story, our reasoning should be: Since there are many more Green cabs than Blue cabs, the probability that the cab was Blue must be adjusted downwards, however confident I am that it was indeed blue (unless, that is, I am absolutely certain: TPR=1, in which case PP=1, irrespective of the Base Rate).
Othello would not have killed Desdemona if he had kept believing in her loyalty. It was Iago’s demolition of Othello’s Base Rate, rather than the strength of his evidence, that drove the Moor into murder. In the same vein, if investors have a reason to believe that a stock price is wrong, they can resist Mr Market’s prior indifference and buy the stock with an ample Margin of Safety.
