Effectiveness of Debiasing Strategies for Hindsight and Overconfidence

The original review (Fischhoff, 1982) applied this framework to all existing studies of hindsight bias and overconfidence. Neither bias was much affected by most manipulations. The exceptions were (1) task restructuring (e.g., asking subjects to consider alternative explanations and contrary evidence) and (2) extensive training. The relative success of these two classes of manipulations affirms the theory of heuristics. Such task restructuring counters the naturally enhanced availability of reasons supporting initial beliefs. Training provides feedback that everyday life typically lacks, allowing people to test and refine judgmental skills. The limits to such instruction suggest which cognitive structures are particularly hard to acquire.

The consistency of the results, across these two domains, involving somewhat overlapping cognitive skills, makes them mutually reinforcing. A decade later, a review of some 100 hindsight studies showed the same general pattern, and some others, of varying practical usefulness (Christensen-Szalanski & Willham, 1991; also Koch & Loewenstein, 1989; Rachlinski, 1998). It found systemically stronger hindsight effects with occurrences than with non-occurrences (e.g., wars started versus wars averted), supporting the theoretical account of new information rapidly restructuring beliefs and retrieval processes, to the extent that it is noted. That review also found a mean difference of 10% between foresight and hindsight probabilities. Extrapolating that result to an applied context requires matching investigators’ item-sampling processes to those in the real-world task. For example, there is little room for hindsight bias with easily predicted events. To what extent did investigators use such events? To what extent do practitioners face them? A bias of that magnitude might have more impact in changing a prediction from 0 to 10% (so that it is no longer a complete surprise) than from 40% to 50% (values that are relatively indistinct; Kahneman & Tversky, 1979).

Although there is no comparable metaanalysis of confidence studies, similar interpretative issues arise. Indeed, the term overconfidence itself incompletely represented the domain of studies. The fundamental phenomenon is “miscalibration,” a mismatch between how confident people are and how confident they should be (given how much they know). This overall tendency typically emerges as overconfidence in hard tasks and as underconfidence in easy ones (Lichtenstein & Fischhoff, 1977). Researchers’ tasks have tended to be hard, so that overconfidence has typically been observed. That selection bias might skew lab tasks toward those of greatest applied interest – if important real-world uncertainties concern hard questions. The same matching concerns remain when stimuli are created by artificial procedures, such as random sampling (In what frame?) or full factorial designs (How natural are the combinations?).

Thus, a complex inferential web connects the lab and the world. The remainder of this chapter focuses on the complexities arising in applied settings. It asks what those dealing with them can learn from and teach to researchers concerned with fundamental psychological processes. Scientists got to work on these problems because someone perceived biased judgments. That someone might be researchers, arguing “perhaps (even) we can help,” or practitioners, sufficiently worried to tell researchers, “perhaps (even) you can help.” Regardless of who starts this interplay, the collaboration should increase the chances that applications will be good for behavioral decision making (BDM), just as BDM tries to be good for the applications (Fischhoff, 1996).