Applying the Wrong Normative Model
Psychologists may misapply normative models from disciplines like statistics and logic when interpreting experimental responses, leading to erroneous claims of cognitive bias. Critics argue that researchers, not subjects, often misunderstand the logic of the problem.
THE IDEA
APPLYING THE WRONG NORMATIVE MODEL
The possibility of incorrect norm application arises because psychologists must appeal to the normative models of other disciplines (e.g., statistics, logic) in order to interpret the responses on various tasks, and these models must be applied to a particular problem or situation. Matching a problem to a normative model is rarely an automatic or clear cut procedure. Indeed, it has been claimed that psychologists have invoked inappropriate normative standards in the random number generation literature (Lopes, 1982), in the work on base rate use in Bayesian reasoning (Birnbaum, 1983), in the overconfidence literature (Gigerenzer (1991a, 1991b, 1993)), with respect to the conjunction effect in probability judgment (Tversky & Kahneman, 1983), and in labeling performance in opinion prediction experiments a so-called false consensus effect (Dawes, 1989, 1990).
The complexities involved in matching problems to norms make possible the argument that the gap between the descriptive and normative occurs because psychologists are applying the wrong normative model to the situation. It is a potent strategy for the Panglossian theorist to use against the advocate of Meliorism and such claims have become quite common in critiques of the heuristics and biases literature: “many critics have insisted that in fact it is Kahneman and Tversky, not their subjects, who have failed to grasp the logic of the problem” (Margolis, 1987, p. 158); “if a ‘fallacy’ is involved, it is probably more attributable to the researchers than to the subjects” (Messer & Griggs, 1993, p. 195); “in the examples of alleged base rate fallacy considered by Kahneman and Tversky, they, and not their experimental subjects, commit the fallacies” (Levi, 1983, p. 502); “Perhaps the only people who suffer any illusion in relation to cognitive illusions are cognitive psychologists” (Ayton & Hardman, 1997, p. 45).
When to use it
Critically evaluating research claims of cognitive biases in published literature
Designing experiments that carefully match problems to appropriate normative models
Teaching students about the importance of model selection in judgment and decision-making research
Defending against accusations of irrationality by questioning the normative benchmark used
FROM THEORY TO EVERYDAY LIFE
In practice
Base Rate Fallacy Debate
Birnbaum (1983) is cited among critics of the normative standards used in studies of base-rate reasoning. The dispute asks whether researchers matched the statistical benchmark to the task participants were actually answering.
Overconfidence and Calibration
Gigerenzer (1991) is cited in the debate over how to interpret apparent overconfidence. The methodological question is whether the chosen benchmark fits the judgment task; a disagreement with that benchmark alone does not settle whether a response is biased.
A MOMENT TO REFLECT
Put it to work.
How can researchers determine whether a normative model is appropriate for a specific experimental problem?
What are the implications of misapplying statistical norms for the validity of claimed cognitive biases?
Can you identify a real-world scenario where what looks like a bias might actually be rational under a different normative standard?
References
- — Lopes (1982)
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)