Love of Certainties
Taleb warns that sample volatility and correlation can be unstable in settings dominated by extreme events. His broader criticism concerns treating a single historical estimate as a dependable property of the future; non-Gaussian data alone do not make those statistics meaningless.
THE IDEA
Love of Certainties
If you ever took a (dull) statistics class in college, did not understand much of what the professor was excited about, and wondered what “standard deviation” meant, there is nothing to worry about. A standard deviation can be unreliable as a summary of risk in the extreme-event settings Taleb calls Extremistan. Clearly it would have been more beneficial, and certainly more entertaining, to have taken classes in the neurobiology of aesthetics or postcolonial African dance, and this is easy to see empirically.
Standard deviations exist for distributions with finite variance, including many non-Gaussian distributions. Some heavy-tailed models have no finite variance, while others have a variance that is difficult to estimate reliably from a small sample. A Gaussian distribution is fully specified by its mean and variance; that does not make all distributions with those moments Gaussian. The Poisson distribution, for example, is a separate distribution whose variance equals its mean.
Similar care is needed with correlation and regression: their usefulness depends on the model, the data, and the question, rather than on normality alone. Yet they are deeply ingrained in our methods; it is hard to have a business conversation without hearing the word correlation.
To examine whether a correlation estimate is stable outside of Mediocristan, take a historical series involving two variables that are patently from Extremistan, such as the bond and the stock markets, or two securities prices, or two variables like, say, changes in book sales of children’s books in the United States, and fertilizer production in China; or real-estate prices in New York City and returns of the Mongolian stock market. Measure correlation between the pairs of variables in different subperiods, say, for 1994, 1995, 1996, etc. The estimated correlation may vary substantially with the period selected. Yet people talk about correlation as if it were something real, making it tangible, investing it with a physical property, reifying it.
The same illusion of concreteness affects what we call “standard” deviations. Take any series of historical prices or values. Break it up into subsegments and measure its “standard” deviation. Surprised? Every sample will yield a different “standard” deviation. Some variation is expected from sampling alone. The question is whether the estimates are stable enough for the decision and whether the model captures the losses that matter.
Note here that, as with the narrative fallacy, when you look at past data and compute one single correlation or standard deviation, you do not notice such instability.
When to use it
Risk management in finance: Avoid overreliance on volatility measures like standard deviation for assets with fat-tailed distributions.
Business forecasting: Consider the instability of correlations when diversifying product lines or markets.
Policy analysis: When evaluating economic indicators, account for the possibility that standard measures may be misleading in non-normal conditions.
Scientific research: Choose appropriate statistical methods (e.g., using medians and quantiles) when working with data from Extremistan.
FROM THEORY TO EVERYDAY LIFE
In practice
Correlation Instability Across Periods
Measure the correlation between bond and stock returns in different subperiods, such as 1994, 1995, and 1996. Large differences would prompt questions about sampling variation, changing conditions, or extreme observations. They would not by themselves prove that correlation is meaningless.
Standard Deviation Variation Across Samples
Compute standard deviations for subsegments of a price or return series. Differences may reflect sampling variation or changing conditions. Assess whether the statistic is defined for the model and whether the available data support a useful estimate.
A MOMENT TO REFLECT
Put it to work.
In what situations have you relied on standard deviation or correlation without considering the underlying distribution of the data?
How might your reliance on averages and standard deviations lead to overconfidence in predicting rare events?
Think of a real-world decision or analysis you’ve encountered—was it in Mediocristan or Extremistan? How should that change your interpretation of statistical measures?
Often confused with
CONTINUE THE CONVERSATION
Source texts
- The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)