Dylan Evans’s Risk Intelligence test has 50 general knowledge statements, half true and half false. Participants are required to give the probability that a statement is true. So if they are certain that it is true they should answer 100%, if they are certain that it is false they should answer 0%, and so on. Perfect calibration requires that x% of the statements for which the answer is x% turn out to be true: all of the statements for which the answer is 100%, none of those for which the answer is 0%, and the same in between. In particular, if one has no idea about whether a statement is true or false, his answer should be 50%. As a result, always answering 50% will yield perfect but obviously worthless calibration.

But that is because 50% of the statements are known to be true. If, for instance, the percentage of true statements were 10%, the ‘no idea’ answer should be the 10% Base Rate and 10% should be the invariable answer in order to attain perfectly worthless calibration.

So imagine that, instead of 50 statements, half true and half false, we have 3,000 stocks, 300 of which will turn out to earn an obscene return and 2,700 won’t. Instead of asking the probability that a statement is true, we ask: what is the probability that a stock will earn an obscene return? Since the Base Rate is 10%, the ‘no idea’ answer is also 10%. That is Mr Market’s perfectly calibrated but totally useless answer: the current price is always right – all stocks are pebbles. Perfect calibration may induce investors to regard him as an expert, but in reality Mr Market has no idea. As Seth Klarman puts it:

Some investors – really speculators – mistakenly look for Mr. Market for investment guidance. They observe him setting a lower price for a security and, unmindful of his irrationality, rush to sell their holdings, ignoring their own assessment of underlying value. Other times they see him raising prices and, trusting his lead, buy in at a higher figure, as if he knew more than they. The reality is that Mr. Market knows nothing. (Margin of Safety, p. 10)

An investor who mistakes Mr Market for an expert has fallen prey to the Prior Indifference Fallacy. He is saying: I don’t know whether the price is right, but Mr Market, who is an expert, says it is, so I believe him. By centring value at the market price, he is neglecting the possibility that it might be much higher. Taking our usual example – value centred at 100, with σ=20, and a market price of 50 – the probability that the price is right, i.e. equal or greater than value, is 0.6%. This is the same as saying that the stock is a pearl, with an 89% probability of a return of 50% or higher. Under prior indifference, the probability that the price is right slides all the way up to 50%, and the probability of an obscene return drops to 11%: the pearl becomes a pebble.

But if the stock is a pebble, there is no longer a Margin of Safety in buying it. The investor’s focus ceases to be the gap between value and the current price and shifts to exploring the reasons why that price – which he now takes as a given – could change in the future. The investor is no longer like Inspector Hubbard, who thinks that Margot is probably innocent, but like the Court, who thinks she is probably guilty. And just like the best approach for the Court to avoid making a mistake is trying to prove that Margot is innocent, the best strategy for the investor becomes trying to prove that the stock will earn an obscene return. Therefore, his best evidence is no longer a Smoking Gun or a Strangler Tie, aimed at ‘convicting’ the stock, but a Barking Dog or a Perfect Alibi, aimed at ‘acquitting’ it. Like a strong suspect needs a good alibi, a pebble needs a good story.

If it has a good story – says the prior indifferent investor – I will buy the stock. This is the right approach if the stock is a pebble, but not if it is a pearl. A pearl doesn’t need a good story – it already has a good price. Once he has found a pearl, the investor’s task is to make sure that the price is truly good, and not the ominous sign of a value trap.

An investor who, faced with a Risk Intelligence test, aims at true calibration will also start with a 10% Base Rate: he knows that obscene returns are rare. So he will answer 0% if he thinks there is no way that a stock will earn such a large return, and 10% if he thinks the stock is fairly priced. Above that, his answer will reflect the extent to which he thinks the stock is undervalued. So, for example, if he thinks a stock, currently priced at 50, is worth 100, with a standard deviation of 20, he will, as we know, answer 89%: the stock is a pearl. If he thinks it is worth 75, he will answer 50%: not quite a pearl, but still an attractive investment. And so on. If he is perfectly calibrated, 9 out of 10 pearls, half of the stocks in the 50% group and, in general, x% of the stocks in the x% group will turn out to earn an obscene return.

Needless to say, no investor is even near to perfect calibration. But by concentrating on pearls or thereabouts – stocks with an ample Margin of Safety – while steering clear of value traps, the investor can still do very well. He doesn’t need perfection: a few hits will suffice.

### 2 Responses to “A Calibrated Investor”

1. Great article! Did the Evans Risk Intelligence test also show overconfidence among participants? If investors have a tendency toward believing that they are better at estimating probabilities than they are truly, then does this imply that most investors should just stick with index funds, i.e., consciously choose “perfectly worthless calibration”?

2. Yes, John, overconfidence is a major obstacle to Risk Intelligence. Test results show that the percentage of true statements for which the answer is x% is significantly less than x% for x>50 and significantly more for x<50. I just wish that 90% of my pearls earned obscene returns! But that’s exactly why the Margin of Safety is so important. As Ben Graham stressed repeatedly, intrinsic value is an elusive concept. When an investor says that value is 100, twice the market price of 50, the key is not the precision of his calculation, but how far it is from the market price, i.e. the size of the Margin of Safety. Buying stocks with an ample Margin of Safety reduces the dangerous impact of overconfidence. We need a lot less than 90% accuracy to be successful investors.

As for index funds, sure. But it’s like a basketball coach saying to the rookie: My best advice for you? Sit on the bench. Mr. Market does that every day. If an investor can’t ignore his ‘guidance’ then, yes, he should sit and watch the game.

This site uses Akismet to reduce spam. Learn how your comment data is processed.