A long time ago I came across a verse by the 17th century French poet Nicolas Boileau, that has stayed with me ever since:
Ce que l’on conçoit bien s’énonce clairement, et les mots pour le dire arrivent aisément.(L’Art poetique, Chant I)
That is: What is well conceived can be expressed clearly, and the words to say it come easily.
The first sentence is, I think, profoundly true. It says: a meaningful concept can be stated clearly. If it’s not clear, it should be rephrased. If it can’t be rephrased, it is not meaningful. But the second sentence is wrong. Precise and concise articulation is difficult. As, in a very different context, Richard Branson puts it:
Complexity is your enemy. Any fool can make something complicated. It is hard to make something simple.
which is well complemented by what Albert Einstein did not say:
Everything should be made as simple as possible, but not simpler.
(The original sentence is in the marvellous Herbert Spencer lecture of 1933, “On the Methods of Theoretical Physics”:
These fundamental concepts and postulates, which cannot be further reduced logically, form the essential part of a theory, which reason cannot touch. It is the grand object of all theory to make all these irreducible elements as simple and as few in number as possible, without having to renounce the adequate representation of any empirical content whatever (Ideas and Opinions, p.272)).
This long preamble to say that I have been thinking about this often-heard phrase:
Absence of evidence is not evidence of absence.
It sounds true, doesn’t it? It is Popper’s black swan: not having seen one (absence of evidence) does not mean that it doesn’t exist (evidence of absence). Same for the Higgs boson, extra-terrestrials, or your car keys: if you can’t find them, it doesn’t mean you’ve lost them.
But then, look under the carpet. Do you see a green rhinoceros? No? Well: Absence of evidence is not evidence of absence – or is it? We need to rephrase.
First, the two ‘evidences’ are not the same thing: absence of evidence is about the sign associated with a hypothesis; evidence of absence is about the hypothesis itself. So a better sentence is: Absence of evidence does not mean that the hypothesis must be false.
Second, sometimes it does mean it – as is the case with the rhino: if you don’t see it – absence of evidence – it obviously means that it’s not there – evidence of absence: the hypothesis “There is a rhino under the carpet” is certainly false.
To see what’s going on, let’s take the example of smoke and fire:
Smoke is confirmative evidence of fire: when there is smoke, there is often a fire. But not always: sometimes there can be a smokeless fire – a False Negative – and some other times there can be smoke without a fire – a False Positive. Therefore, absence of evidence (no smoke) does not mean that the hypothesis “There is a fire” must be false. And we add: presence of evidence (smoke) does not mean that the hypothesis must be true.
However, if we imagine instead that smokeless fire is impossible, then FNR=0: no False Negatives – if there is a fire, there must be smoke. Hence, absence of evidence (no smoke) does mean evidence of absence: the hypothesis “There is a fire” must be false.
On the other hand, if we imagine that, as the saying goes, “There is no smoke without fire”, then FPR=0: no False Positives – if there is smoke, there must be a fire. Hence, presence of evidence (smoke) does mean evidence of presence: the hypothesis “There is a fire” must be true.
We see, therefore, that whether our phrase is true or not depends on the hypothesis, and on whether its associated evidence is or is not conclusive.
So a better sentence is: Absence of inconclusive evidence does not mean that the hypothesis must be false; and presence of inconclusive evidence does not mean that it must be true. However, absence of conclusive evidence does mean that the hypothesis must be false; and presence of conclusive evidence does mean that it must be true.
To add further clarity, let’s replace ‘absence of evidence’ with ‘negative evidence’ and ‘presence of evidence’ with ‘positive evidence’, keeping in mind that positive and negative are two sides of the same evidence. In our example, smoke is positive evidence and no smoke is negative evidence. And let’s replace ‘it means that the hypothesis must be false’ with ‘it falsifies the hypothesis’; and ‘it means that the hypothesis must be true’ with ‘it verifies the hypothesis’. Thus the sentence becomes:
Inconclusive negative evidence does not falsify the hypothesis; and inconclusive positive evidence does not verify it. However, conclusive negative evidence does falsify the hypothesis; and conclusive positive evidence does verify it.
Of course, it may be the case that FNR=0 and FPR=0: evidence is not only conclusive, but perfect. With perfect evidence, there is no error: if there is smoke, there is certainly a fire; if there is no smoke, there is certainly no fire. Such is the case with the rhino under the carpet: absence of evidence is evidence of absence; and presence of evidence would be, unmistakably, evidence of presence.
But if only one of the two errors is zero, evidence can be conclusive on one side and inconclusive on the other. In particular:
If FPR=0 and FNR>0, conclusive positive evidence verifies the hypothesis; but inconclusive negative evidence does not falsify it. We call this a Smoking Gun.
If FNR=0 and FPR>0, conclusive negative evidence falsifies the hypothesis; but inconclusive positive evidence does not verify it. We call this a Barking Dog.
To complete the 2×2 matrix, we need to consider disconfirmative evidence. Just as smoke is confirmative evidence for the hypothesis “There is a fire”, we can take rain as an example of disconfirmative evidence. In this case, the ‘errors’ are True Positives – rain with fire – and True Negatives – no rain with no fire. Again, to consider conclusive evidence we can imagine TNR=0: no True Negatives – if there is no rain, there must be a fire; or TPR=0: no True Positives – if there is rain, there cannot be a fire. Hence:
If TPR=0 and TNR>0, conclusive positive evidence falsifies the hypothesis; but inconclusive negative evidence does not verify it. We call this a Perfect Alibi.
If TNR=0 and TPR>0, conclusive negative evidence verifies the hypothesis; but inconclusive positive evidence does not falsify it. We call this a Strangler’s Tie.
Popper’s black swan – the instance that is most often associated with the phrase ‘Absence of evidence is not evidence of absence’ – is a Perfect Alibi for the hypothesis “All swans are white”: conclusive positive evidence – seeing a black swan – falsifies the hypothesis; but inconclusive negative evidence – failure to see a black swan – does not verify it.
It is, however, only one among many possibilities. Simple, but not simpler.
