We Do Not Understand Confidence Levels
A forecast’s uncertainty can matter as much as its central estimate. Taleb’s packing example shows how the same expected temperature can call for different preparations when the possible spread changes.
THE IDEA
We Do Not Understand Confidence Levels
Figure 11.1 Unfiltered Data Containing Signal and Noise
Figure 11.2 Same Data with Its Noise Removed
Professionals forget the following reality. It is not the estimate or the forecast that matters so much as the degree of confidence with the opinion. Consider that you are going on a trip one fall morning and need to formulate an idea about the weather conditions prior to packing your luggage. If you expect the temperature to be 60 degrees, plus or minus 10 degrees (say in Arizona), then you would take no snow clothes and no portable electric fan. Now, what if you were going to Chicago, where you are told that the weather, while being 60 degrees, will nevertheless vary by about 30 degrees? You would have to pack winter and summer clothes. Here the expectation of the temperature carries little importance concerning the choice of clothing; it is the variance that matters. Your decision to pack is markedly different now that you are told that the variability would be around 30 degrees. Now let us push the point further; what if you were going to a planet where the expectation is also going to be around 60 degrees, but plus or minus 500 degrees? What would you pack?
We can see that my activity in the market (and other random variables) depends far less on where I think the market or the random variable is going so much as it does on the degree of error I allow around such a confidence level.
When to use it
Investment decisions: consider uncertainty and possible losses alongside expected return.
Project planning: Ask for best-case, worst-case, and most-likely estimates to account for variability.
Weather preparation: Use probabilistic forecasts (e.g., 70% chance of rain) rather than a single temperature.
Medical decisions: Understand the range of possible outcomes from a treatment, not just the average effect.
FROM THEORY TO EVERYDAY LIFE
In practice
Packing for a Trip
In the Arizona example, an expected temperature of 60 degrees with a spread of ±10 suggests one packing plan. In Chicago, a spread of ±30 calls for a wider range of clothing. A planet with a spread of ±500 may make the trip infeasible despite the same 60-degree estimate. These spreads are not mathematical variances.
A MOMENT TO REFLECT
Put it to work.
When you hear a forecast or estimate, do you also ask for the range of possible outcomes? Why or why not?
Think of a time when you focused too much on a single number (e.g., stock price, grade) and ignored the uncertainty. What happened?
How can you incorporate confidence intervals into your daily decision-making, such as in budgeting or planning?
CONTINUE THE CONVERSATION
Source texts
- Fooled by Randomness The Hidden Role of Chance in the Markets and in Life (Nassim Nicholas Taleb)