Silent Evidence (The Health Club for Rats)
Silent evidence is a cognitive bias where we only observe survivors and mistakenly attribute their survival to external factors, ignoring the non-survivors. The rat health club metaphor illustrates how selection bias creates false narratives about causality.
THE IDEA
A HEALTH CLUB FOR RATS
The second, and more vicious, variety of the problem of silent evidence is as follows. When I was in my early twenties and still read the newspaper, and thought that steadily reading the newspapers was something useful to me, I came across an article discussing the mounting threat of the Russian Mafia in the United States and its displacement of the traditional Louie and Tony in some neighborhoods of Brooklyn. The article explained their toughness and brutality as a result of their being hardened by their Gulag experiences. The Gulag was a network of labor camps in Siberia where criminals and dissidents were routinely deported. Sending people to Siberia was one of the purification methods initially used by the czarist regimes and later continued and perfected by the Soviets. Many deportees did not survive these labor camps.
Hardened by the Gulag? The sentence jumped out at me as both profoundly flawed (and a reasonable inference). It took me a while to figure out the nonsense in it since it was protected by cosmetic wrapping; the following thought experiment will give the intuition. Assume that you’re able to find a large, assorted population of rats: fat, thin, sickly, strong, well-proportioned, et cetera. (You can easily get them from the kitchens of fancy New York restaurants.) With these thousands of rats, you build a heterogeneous cohort, one that is well representative of the general New York rat population. You bring them to my laboratory on East Fifty-ninth Street in New York City and we put the entire collection in a large vat. We subject the rats to increasingly higher levels of radiation (since this is supposed to be a thought experiment, I am told that there is no cruelty in the process). At every level of radiation, those that are naturally stronger (and this is the key) will survive; the dead will drop out of your sample. We will progressively have a stronger and stronger collection of rats. Note the following central fact: every single rat, including the strong ones, will be weaker after the radiation than before.
An observer endowed with analytical abilities, who probably got excellent grades in college, would be led to believe that treatment in my laboratory is an excellent health-club replacement, and one that could be generalized to all mammals (think of the potential commercial success). His logic would run as follows: Hey, these rats are stronger than the rest of the rat population. What do they seem to have in common? They all came from that Black Swan guy Taleb’s workshop. Not many people will have the temptation to go look at the dead rats.
Next we pull the following trick on The New York Times: we let these surviving rats loose in New York City and inform the chief rodent correspondent of the newsworthy disruption in the pecking order in the New York rat population. He will write a lengthy (and analytical) article on the social dynamics of New York rats that includes the following passage: “Those rats are now bullies in the rat population. They literally run the show. Strengthened by their experience in the laboratory of the reclusive (but friendly) statistician/philosopher/trader Dr. Taleb, they …”
When to use it
Evaluating the effectiveness of a medical treatment by only looking at surviving patients.
Analyzing historical events where only successful leaders are studied, ignoring the many who failed.
In business, studying only successful startups without considering the vast number that failed.
Making investment decisions based solely on funds with strong past performance, ignoring those that closed.
FROM THEORY TO EVERYDAY LIFE
In practice
The Russian Mafia and the Gulag
A newspaper attributed Russian Mafia members’ toughness to Gulag experiences. Taleb questions that causal inference: the survivors might already have been unusually resilient. His rat experiment shows how selection could produce the appearance of strengthening even when every individual is harmed.
The Rat Health Club Thought Experiment
Rats subjected to radiation become weaker individually, but the survivors are stronger than the original population. An outside observer might mistakenly conclude the radiation strengthened them, ignoring the dead rats.
A MOMENT TO REFLECT
Put it to work.
How might the silent evidence bias affect your perception of successful entrepreneurs or businesses?
Can you identify a situation in your life where you only considered survivors and drew a false conclusion?
Why is it important to examine the 'dead rats'—the failures—when analyzing any process or outcome?
CONTINUE THE CONVERSATION
Source texts
- The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)