A father watches his child playing football. He thinks he is quite good at it, but he is well aware that really talented players are rare. Take a random sample of 20,000 children: how many are going to become good players? Perhaps 20 – one in a thousand.
But hang on. 1/1000 is the probability of extracting a top player from the child population. This is not what the father is after: he wants to know the probability that his child will turn out to be a champion. This could be properly assessed only by comparing the child to others who are more like him: children who share the same, or at least a comparable probability of becoming top players. But what does comparable mean? Among 20,000 children, there’s got to be lots of hopeless cases, and his child is clearly better than the average. Proper stratified sampling would likely show that healthier, stronger, faster children – like his own – have a higher chance: let’s say 5/1000. But this may also be completely off the mark. For instance, future genetic research may reveal a link between football prowess and a particular gene, which is found in, say, only 2% of the population. If a child does not carry the “Maradona gene”, he can forget about ever becoming a football star. But if he has it, the probability of being a star is 5% (apologies to biologists: it is an exemplification). Thanks to this discovery, we would find that the 1/1000 population Base Rate is really the product of 2% times 5%: in 20,000 children, 20 carry the gene and will become stars, 380 carry the gene but will not be stars, and the rest have no gene and therefore no chance. Or perhaps the gene is present in only 1% of the population, and those who carry it have a 10% probability of becoming champions. Or maybe it is such a rare gene that has a 0.5% frequency, and the lucky ones have a 20% chance of success. And why not go all the way: only one in a thousand have the gene and are therefore predestined to stardom.
So what is the appropriate Base Rate? Given the current state of knowledge, it is 1/1000. But it could be completely different, depending on the definition of the appropriate reference class. We can think of the rarity of the Maradona gene as an image of the state of knowledge about football talent. The more we know, the rarer the gene and the smaller the reference class. Indeed, knowledge can be defined as a progressive narrowing down of possibilities. In Sherlock Holmes’s immortal words: “When you have eliminated the impossible, whatever remains, however improbable, must be the truth”. The smaller the reference class, the higher the Base Rate for individuals belonging to that class. In the limit, knowledge about football talent could become as complete as to allow us to narrow down the population to precisely those one-in-a-thousand children who will certainly become champions. Stardom would then be either a certainty or an impossibility.
This uncertainty about the appropriate reference class is distinctly Knightian. Given current knowledge, the Base Rate is 1/1000, but with increased knowledge it could be anywhere between 0 and 1 – like extracting from an urn with white and black balls in unknown proportions. It is in this state of uncertainty, and with the aim of increasing his knowledge, that the father asks the expert coach: What is the chance of my child becoming a top football player? Remember the coach is very accurate: he is infallible at spotting top players and mistakes an ordinary child for a top player only 5% of the times. After the coach’s response, the father no longer sees his child as a comparable member of the general population. The 1/1000 Base Rate – so clear and consequential until then – is driven to the background: it becomes an unknown known. The father no longer knows which reference class the child belongs to, hence he cannot define the relevant Base Rate. And since an undefinable Base Rate could be anywhere between 0 and 1, he picks the neutral midpoint: he becomes prior indifferent. He simply thinks: my child may or may not be a champion, attach an equal chance to the two possibilities, and let the coach decide. And if the coach says the child is a champion, he believes him.
The urge to resolve this uncomfortable state of Knightian uncertainty is what consigns the father into the hands of the expert. His drivers are hope and wishful thinking. But they could be entirely different. Othello certainly does not hope Desdemona is unfaithful, but he still wants to get out of the state of Knightian uncertainty that Iago plunged him into.
What should the father and Othello do to shun prior indifference? They should resist the sirens of Knightian uncertainty and properly place new evidence within the confines of what they already know. Priors must be constantly updated, but should never be ignored.
Or – as reprised by many but, it seems, first expressed by the US diplomat Walter Kotschnig: It is good to keep an open mind, but not so open that your brain falls out.