Discriminability (Neglect of Alternative Hypotheses)
People tend to evaluate the strength of evidence for a hypothesis without considering how well the same evidence fits alternative hypotheses, leading to overconfidence. This is demonstrated by the Barnum effect and a coin-spinning experiment.
THE IDEA
Discriminability
When comparing hypotheses, assess how well the evidence distinguishes them. Evidence that fits one account may fit another just as well. Prior probabilities also matter when turning relative support into confidence in a hypothesis.
Neglecting alternatives can contribute to the Barnum effect: a person accepts a generic personality description because it fits them, without asking how well it fits other people.
Study 3: Discriminability (Coin Experiment)
The excerpt describes two types of foreign coins: X-quintas (60% heads bias) and Y-quintas (75% tails bias). Participants judged whether samples came from an X or Y coin, with evidence ranging from 7/12 heads to 10/12 heads. Assessing the sample’s resemblance to an X coin alone leaves out its likelihood under Y.
A further comparison involved A libnars (60% heads) and B libnars (50% heads). Comparing the same evidence across pairs of possible sources helps examine whether judgments track how well the evidence discriminates between the alternatives.
When to use it
Interpreting medical test results: A positive test result might fit a disease hypothesis, but it's crucial to consider the false positive rate and how well the result fits alternative conditions.
Evaluating job candidates: Consider whether a strong interview reflects job-relevant ability, extensive rehearsal, or other explanations, and seek evidence that distinguishes them.
Marketing analysis: When a product performs well in one market, a marketer might attribute it to their strategy without considering alternative explanations like seasonal effects or competitor weaknesses.
Scientific research: When data support a hypothesis, researchers should also consider how well the data fit competing hypotheses to avoid confirmation bias.
FROM THEORY TO EVERYDAY LIFE
In practice
Coin Classification
Participants classify a coin as type X (60% heads) or type Y (75% tails). A sample contains 7 heads in 12 flips, close to X’s 60% rate. Confidence should depend on the likelihood of that sample under both coin types and on their prior frequencies. The fact that Y could also produce 7 heads does not by itself show that confidence in X is excessive.
Barnum Personality Sketch
A classic demonstration: People are given a personality description that is actually generic and true for most people. They rate it as highly accurate for themselves because they focus on how well it fits their own personality, ignoring that it fits everyone else just as well.
A MOMENT TO REFLECT
Put it to work.
Think of a time when you were convinced a diagnosis or explanation was correct because the evidence seemed to fit perfectly. Did you consider alternative explanations?
How might this bias affect the way we evaluate the accuracy of personality tests or horoscopes?
In everyday decision-making, how can we systematically consider alternative hypotheses before concluding?
Often confused with
References
- The fallacy of personal validation: A classroom demonstration of gullibility — Forer, B. R. (1949)
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)