The Weighing of Evidence and the Determinants of Confidence
Overconfidence is common in judgment, but easy questions and sequential updating can also produce underconfidence. Griffin and Tversky examine these patterns and the role of feedback in calibrating confidence.
THE IDEA
C. FORECASTING, CONFIDENCE, AND CALIBRATION
13. The Weighing of Evidence and the Determinants of Confidence *
Dale Griffin and Amos Tversky
The weighing of evidence and the formation of belief are basic elements of human thought. The question of how to evaluate evidence and assess confidence has been addressed from a normative perspective by philosophers and statisticians; it has also been investigated experimentally by psychologists and decision researchers. One of the major findings that has emerged from this research is that people are often more confident in their judgments than is warranted by the facts. Overconfidence is not limited to lay judgment or laboratory experiments. The well-publicized observation that more than two-thirds of small businesses fail within 4 years (Dun & Bradstreet, 1967) suggests that many entrepreneurs overestimate their probability of success (Cooper, Woo, & Dunkelberg, 1988). With some notable exceptions, such as weather forecasters (Murphy & Winkler, 1977), who receive immediate frequentistic feedback and produce realistic forecasts of precipitation, overconfidence has been observed in judgments of physicians (Lusted, 1977), clinical psychologists (Oskamp, 1965), lawyers (Wagenaar & Keren, 1986), negotiators (Neale & Bazerman, 1990), engineers (Kidd, 1970), and security analysts (Staël von Holstein, 1972). As one critic described expert prediction, “often wrong, but rarely in doubt.”
Overconfidence is common, but not universal. Studies of calibration have found that with very easy items, overconfidence is eliminated and underconfidence is often observed (Lichtenstein, Fischhoff, & Phillips, 1982). Furthermore, studies of sequential updating have shown that posterior probability estimates commonly exhibit conservatism or underconfidence (Edwards, 1968).
When to use it
Forecasting business success or failure when starting a new venture, to avoid overestimating chances and to seek realistic feedback.
Medical diagnosis: Doctors can use calibration techniques to improve confidence judgments and reduce diagnostic errors.
Legal judgments: Lawyers and judges can account for overconfidence in predictions of case outcomes to make more balanced decisions.
Investment decisions: Security analysts can apply insights from calibration to adjust confidence in market predictions and avoid overconfident trading.
FROM THEORY TO EVERYDAY LIFE
In practice
Entrepreneurs Overestimating Success
The authors cite a historical report that more than two-thirds of small businesses failed within 4 years, alongside research on entrepreneurs’ expectations (Dun & Bradstreet, 1967; Cooper, Woo, & Dunkelberg, 1988). They use this comparison to introduce overconfidence.
Weather Forecasters Achieving Calibration
Weather forecasters, who receive immediate frequentistic feedback, produce realistic forecasts of precipitation, serving as an exception to typical overconfidence (Murphy & Winkler, 1977).
A MOMENT TO REFLECT
Put it to work.
Consider a recent prediction you made about a personal or professional outcome. How confident were you, and did the actual outcome match your confidence? What evidence did you weigh?
Reflect on a situation where you or someone you know exhibited overconfidence. What factors might have contributed to that overconfidence?
Can you think of an area where you consistently receive frequent, accurate feedback (like a weather forecaster)? How does that feedback affect your calibration?
References
- Dun & Bradstreet — Dun & Bradstreet (1967)
- Cooper, Woo, & Dunkelberg — Cooper, A. C., Woo, C. Y., & Dunkelberg, W. C. (1988)
- Murphy & Winkler — Murphy, A. H., & Winkler, R. L. (1977)
- Lusted — Lusted, L. B. (1977)
- Oskamp — Oskamp, S. (1965)
- Wagenaar & Keren — Wagenaar, W. A., & Keren, G. (1986)
- Neale & Bazerman — Neale, M. A., & Bazerman, M. H. (1990)
- Kidd — Kidd, J. B. (1970)
- Staël von Holstein — Staël von Holstein, C.-A. S. (1972)
- Lichtenstein, Fischhoff, & Phillips — Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982)
- Edwards — Edwards, W. (1968)
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)