Consider Multiple Hypotheses
Just because someone has an explanation for a phenomenon doesn't mean it is the correct one. Always consider multiple plausible explanations before concluding, as patterns and trends often have many possible causes.
THE IDEA
Consider Multiple Hypotheses
The key thing to realize is that just because someone has an explanation for some phenomenon doesn’t mean that it is the explanation for that phenomenon.
In May 2018, TV personality Roseanne Barr posted a racist message to Twitter. Outrage ensued and Barr apologized, blaming her actions on the sleeping medication Ambien. But it was too late; the Disney-owned ABC network canceled her sitcom despite a record-setting comeback.
Whatever one thinks about Roseanne, racism, Twitter, or Ambien, what happened next was interesting. The leading newswire service Reuters reported on Twitter that "JUST IN: Walt Disney shares down 2.5 percent after ABC cancels 'Roseanne' show." Reuters was correct that Disney stock dropped 2.5 percent that day—but the headline here implies that the Roseanne cancellation was somehow responsible for the decline in share price. It couldn’t have been: The 2.5 percent drop occurred before, not after, the Roseanne announcement. Indeed, the stock market as a whole had plunged dramatically that morning. Disney’s stock had actually ended its 2.5 percent slide prior to the Roseanne announcement in the early afternoon.
This is a powerful example of "if it seems too bad to be true, it probably is." Disney is an enormous conglomerate. Roseanne is a single sitcom. Disney generated about $55 billion in revenue in 2017. Season ten of Roseanne generated about $45 million in revenue in 2018. How could the loss of a series that generated 0.1 percent of Disney’s revenue drive a 2.5 percent decline in stock price? It doesn’t pass a basic plausibility check.
The problem with the Reuters tweet is that when you have a phenomenon of interest (a 2.5 percent slide in Disney’s stock price) and a possible explanation for that phenomenon (Roseanne was canceled), your story can seem compelling. Roseanne’s racist tweet was by far the most societally salient event associated with Disney’s holdings at the time—television pundits, newspaper columnists, and social media posters alike were up in arms about what she wrote and about the consequences she faced for doing so. But this does not mean that this event was the most plausible explanation for the trading results.
The real point here is that in the vast majority of cases, there will be many possible explanations for any given pattern or trend, and simply being consistent with the data does not make an explanation correct. Sometimes a proposed explanation may be correct but only a small part of the causal contribution; other times the proposed explanation may be totally wrong, unconnected to the real explanation.
In order to avoid falling for these seemingly plausible but incorrect explanations, consider as many possible explanations as you can for any trend or pattern or event that you seek to understand.
When to use it
When evaluating a study's conclusion, list at least three alternative explanations for the results.
When your child's grades drop suddenly, consider multiple hypotheses: difficulty with material, personal issues, classroom environment, etc.
In business, when sales decline, consider factors like seasonality, competition, market trends, and internal changes before attributing to a single cause.
When interpreting a data visualization, ask what other trends or biases could explain the pattern.
FROM THEORY TO EVERYDAY LIFE
In practice
Gender Bias in Recommendation Letters
Suppose a chart appears to show different language in recommendation letters for different groups. Before choosing an explanation, check whether it presents a hypothesis or measured results, how the letters were sampled and what other factors differ between the groups.
Disney Stock and the Roseanne Cancellation
Reuters described a 2.5% fall in Disney shares after ABC canceled Roseanne. The authors point out that the fall preceded the announcement and that the broader market had also dropped. That timing challenges the proposed cancellation explanation and makes a market-wide explanation worth examining.
A MOMENT TO REFLECT
Put it to work.
When you hear a surprising claim, ask yourself: What other explanations could there be for this phenomenon?
Think of a recent news headline that implied causation from correlation. What alternative hypotheses were overlooked?
How can you systematically generate multiple hypotheses before accepting an explanation?
Often confused with
CONTINUE THE CONVERSATION
Source texts
- Calling Bullshit The Art of Skepticism in a Data-Driven World (Carl T. Bergstrom Jevin D. West)