How to Medicate Half the Population
Taleb uses a blood-pressure anecdote and a simplified thought experiment to criticize intervention based on noisy measurements. His broader point is to distinguish the size of an effect from statistical significance and to consider the costs of acting on uncertain information.
THE IDEA
How to Medicate Half the Population
I was once attending a lunch party at the country house of a friend when someone produced a handheld blood pressure measuring tool. Tempted, I measured my arterial pressure, and it turned out to be slightly higher than average. A doctor, who was part of the party and had a very friendly disposition, immediately pulled out a piece of paper prescribing some medication to lower it—which I later threw in the garbage can. I subsequently bought the same measuring tool and discovered that my blood pressure was much lower than average, except once in a while, when it peaked episodically. In short, it exhibits some variability. Like everything in life.
This random variability is often mistaken for information, hence leading to intervention. Let us play a thought experiment, without making any assumption on the link between blood pressure and health. Further, assume that "normal" pressure is a certain, known number. Take a cohort of healthy persons. Suppose that because of randomness, half the time a given person's pressure will be above that number, and half the time, for the same person, the measurement will be below. So on about half the doctor's visits they will show the alarming "above normal." If the doctor automatically prescribes medication on the days the patients are above normal, then half the normal population will be on medication. Unnecessary treatments can expose patients to harm without a compensating benefit. Clearly I am simplifying here; sophisticated doctors are aware of the variable nature of the measurements and do not prescribe medication when the numbers are not compelling (though it is easy to fall into the trap, and not all doctors are sophisticated). But the thought experiment can show how frequent visits to the doctor, particularly outside the cases of a life-threatening ailment or an uncomfortable condition—just like frequent access to information—can be harmful. The example extends the warning in Chapter 7 of Antifragile about harm from overreacting to noise.
This is more serious than you think: it seems that medicine has a hard time grasping normal variability in samples—it is hard sometimes to translate the difference between "statistically significant" and "significant" in effect. A certain disease might marginally lower your life expectancy, but it can be deemed to do so with "high statistical significance," prompting panics when in fact all these studies might be saying is they established with a significant statistical margin that in some cases, say, 1 percent of the cases, patients are likely to be harmed by it. Let me rephrase: the magnitude of the result, the importance of the effect, is not captured by what is called "statistical significance," something that tends to deceive specialists. We need to look in two dimensions: how much a condition, say, blood pressure a certain number of points higher than normal, is likely to affect your life expectancy; and how significant the result is.
Why is this serious? If you think that the statistician really understands "statistical significance" in the complicated texture of real life (the "large world," as opposed to the "small world" of textbooks), some surprises. Kahneman and Tversky showed that statisticians themselves made practical mistakes in real life in violation of their teachings, forgetting that they were statisticians (thinking, I remind the reader, requires effort). My colleague Daniel Goldstein and I did some research on "quants," professionals of quantitative finance, and realized that the overwhelming majority did not understand the practical effect of elementary notions such as "variance" or "standard deviation," concepts they used in about every one of their equations. A study by Emre Soyer and Robin Hogarth showed that many professionals and experts in the field of econometrics supplying pompous numbers such as "regression" and "correlation" made egregious mistakes translating into practice the numbers they were producing themselves—they get the equation right but make severe translation mistakes when expressing it into reality. In all cases they underestimate randomness and underestimate the uncertainty in the results. And we are talking about errors of interpretation made by the statisticians, not by the users of statistics such as social scientists and doctors.
Alas, all these biases lead to action, almost never inaction.
Taleb extends his criticism to dietary advice. He argues that "fat free" slogans reflected a failure to distinguish the effects of carbohydrates and fat when they were consumed together. He also cites work by David Freedman and a coauthor to challenge statistical arguments about salt and blood pressure. These examples express his objection to moving from a statistical association to an intervention without adequately examining the analysis or the size of the effect.
When to use it
Examining how a treatment decision accounts for measurement variability rather than treating one isolated reading as the whole record.
Business performance: monthly sales fluctuations may be noise; base decisions on longer-term trends.
Discussing how to interpret a series of health measurements, including normal variation and the purpose of collecting them.
Investment: daily market movements are largely noise; avoid impulsive trading based on short-term changes.
FROM THEORY TO EVERYDAY LIFE
In practice
Blood Pressure Variability
Taleb recalls receiving a prescription after a slightly high blood-pressure reading at a party. He says later readings were generally lower, with occasional peaks. He uses the anecdote to criticize treating one measurement as sufficient evidence, rather than as a complete assessment of whether medication was appropriate.
Medicating Half the Population Thought Experiment
In Taleb’s simplified model, each healthy person has an equal chance of reading above or below a chosen reference value. At a single visit, half the group would therefore read above it. An automatic prescription rule would treat those readings as illness, even though the model assumes everyone is healthy.
Fat-Free Slogans Misinterpretation
Taleb attributes enthusiasm for fat-free diets to regression analyses that, in his account, assigned joint effects of carbohydrate and fat consumption to fat alone. The example illustrates his criticism of statistical interpretation; the excerpt does not supply the underlying analyses.
A MOMENT TO REFLECT
Put it to work.
How can you distinguish between random variability and meaningful information in your daily decisions?
Think of a time when you or someone you observed overreacted to a single data point. What were the consequences?
Why is it important to consider both statistical significance and the magnitude of an effect when interpreting research?
References
- Research on statistical misinterpretation by experts — Kahneman and Tversky
- Study on quant understanding of variance — Goldstein and Taleb
- Study on econometricians' translation errors — Soyer and Hogarth
- Debunking of salt-blood pressure link — Freedman and coauthor
CONTINUE THE CONVERSATION
Source texts
- Antifragile (Nassim Nicholas Taleb)