The Dependence on Theory for Rare Events
When an event is rare relative to the available record, observations provide limited guidance about its frequency. Estimating its probability then depends more heavily on assumptions about outcomes beyond the observed data.
THE IDEA
THE DEPENDENCE ON THEORY FOR RARE EVENTS
During my deserto period, when I was getting severe but entertaining insults, I found myself debating a gentleman then employed by a firm called Lehman Brothers. That gentleman had made a statement in The Wall Street Journal saying that events we saw in August 2007 should have happened once every ten thousand years. Sure enough, we had three such events three days in a row. The Wall Street Journal ran his picture and if you look at it, you can safely say, “He does not look ten thousand years old.” So where is he getting his “once in ten thousand years” probability? Certainly not from personal experience; certainly not from the records of Lehman Brothers—his firm had not been around for ten thousand years, and of course it didn’t stay around for another ten thousand years, as it went under right after our debate. So, you know that he’s getting his small probabilities from a theory. The more remote the event, the less we can get empirical data (assuming generously that the future will resemble the past) and the more we need to rely on theory.
Consider that the frequency of rare events cannot be estimated from empirical observation for the very reason that they are rare. We thus need a prior model representation for that; the rarer the event, the higher the error in estimation from standard inductive methods (say, frequency sampling from counting past occurrences), hence the higher the dependence on an a priori representation that extrapolates into the space of low-probability events (which necessarily are not seen often).*
But even outside of small probabilities, the a priori problem is always present. It seems salient with respect to rare events, but it pervades probabilistic knowledge. I have been working on two versions of this problem with two collaborators, Avital Pilpel, a philosopher of science (he walks fast), and Raphael Douady, a mathematician (he is sometimes a good walker, when he is not busy).
When to use it
Risk management in finance: assessing tail risk of portfolio losses using extreme value theory.
Disaster preparedness: estimating probability of 100-year floods or major earthquakes for infrastructure planning.
Medical research: estimating incidence of rare diseases using Bayesian methods and prior models.
Engineering: predicting failure of safety-critical systems where failures are rare but catastrophic.
FROM THEORY TO EVERYDAY LIFE
In practice
Financial market crashes
A risk manager uses a Gaussian model to estimate the probability of a 10-standard-deviation market drop. Historical data shows no such event, but the model assigns a tiny probability. When a crash occurs, the estimate is exposed as a product of theory, not observation.
Estimating 100-year flood levels
Hydrologists estimate the 1% annual exceedance probability flood using theoretical distributions and limited historical records. The rarer the flood, the more the estimate depends on assumptions about the tail of the distribution, as empirical data is sparse.
A MOMENT TO REFLECT
Put it to work.
Think of a rare event in your field. How was its probability estimated? Was the estimation reliant on theory rather than data?
Why might frequency-based methods fail for rare events? What assumptions do they rely on?
How can we assess the reliability of the theoretical models used to estimate rare event probabilities?
CONTINUE THE CONVERSATION
Source texts
- The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)