Epimenides the Cretan
Taleb and Avital Pilpel describe a regress in risk modeling: data is used to estimate a distribution, while assumptions about that distribution help determine whether the data is sufficient. They argue that this dependence is especially troubling for rare events.
THE IDEA
Epimenides the Cretan
Avital Pilpel and I expressed the regress argument as follows, as the epistemic problem of risk management, but the argument can be generalized to any form of probabilistic knowledge. It is a problem of self-reference by probability measures.
We can state it in the following way. If we need data to obtain a probability distribution to gauge knowledge about the future behavior of the distribution from its past results, and if, at the same time, we need a probability distribution to gauge data sufficiency and whether or not it is predictive of the future, then we face a severe regress loop. This is a problem of self-reference akin to that of Epimenides the Cretan stating whether or not Cretans are liars. Indeed, it is too uncomfortably close to the Epimenides situation, since a probability distribution is used to assess the degree of truth but cannot reflect on its own degree of truth and validity. And, unlike many problems of self-reference, those related to risk assessment have severe consequences. The problem is more acute with small probabilities.
When to use it
Assessing the risk of extreme events in financial markets
Modeling the probability of catastrophic failures in engineering
Evaluating the reliability of climate change predictions
Determining the sample size needed for a clinical trial when the effect size is unknown
FROM THEORY TO EVERYDAY LIFE
In practice
The Epimenides Paradox
Epimenides, a Cretan, is associated with the statement 'All Cretans are liars.' Its self-reference raises questions about truth and the meaning of liar, but it is not automatically the same contradiction as a statement declaring itself false: the universal claim could be false because another Cretan tells the truth. Taleb invokes it as an analogy for model validity.
Risk Assessment Regress
Estimating a rare event from historical data requires assumptions about how observations relate to future events. Judging whether the sample is sufficient also depends on a model. The regress argument asks how those assumptions are justified; it does not by itself show that every validation method is useless.
A MOMENT TO REFLECT
Put it to work.
What evidence or checks independent of a model’s fitting data could help assess its assumptions?
In what ways does self-reference undermine the reliability of probabilistic forecasts in finance?
Can we ever break the regress loop, or is it inherent to all knowledge claims about uncertainty?
CONTINUE THE CONVERSATION
Source texts
- The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)