Expert Overconfidence and Belief Underadjustment

Implications for Rationality

Converging evidence suggests that experts not only rely, but over-rely, on their preconceptions in generating expectations about the future (producing overconfidence) and in revising preconceptions in response to unexpected events (producing belief underadjustment by Bayesian standards). Moreover, we can identify several belief-system defenses that experts deploy to protect their worldviews from falsification and we can identify a characteristic cognitive-style profile for those experts most inclined to invoke these defenses and to display overconfidence in the original forecasts and underadjustment in response to unexpected events. The more value experts place on parsimony and explanatory closure, the more prone they are to overconfidence (ex ante) and Bayesian under-adjustment (ex post).

In complex natural settings, though, we need to be careful in rendering normative judgment. As already noted, the belief-system defenses may themselves be defensible, at least up to a point. Reliance on these defenses may also be compatible with several conceptions of learning from history. Indeed, the data shed light on when learning is likely and on the forms that it will take. It will often be stimulated by failure, and constrained by the cognitive and organizational principles of least resistance, to take incremental forms (Nye, 1988; Tetlock, 1991). Most experts initially respond to unexpected events by tinkering with peripheral cognitions that require minimal modification of their belief systems. Rather than conceding on “fundamentals,” they argue that some new unforeseeable causal force emerged, thus nullifying the ceteris paribus clause, or that the predicted outcome “almost occurred.” The cumulative effect of such tinkering should not be understated. Whenever experts argue that their predicted future would have occurred but for some historical accident or previously unacknowledged moderator variable, they have learned in a significant sense. Their belief systems now assign a more important role to chance and recognize causality to have been more complex than previously acknowledged. From this standpoint, the more defensive close-call counterfactuals one generates, the greater the likelihood that one is learning, both in the cognitive-structural meaning of the term (evolving differentiated and integrated knowledge structures for processing future evidence) and in the efficiency-enhancement sense of the term (acquiring the ability either to make more accurate forecasts or at least to attach more realistic confidence estimates to one’s forecasts). This argument suggests a curious conclusion: the more experts are wrong, the wiser they become as they internalize more qualifications on when their expectations hold. The real threat to good judgment lies in the hubris that builds up from a succession of predictions that serendipitously turn out right. We thus have a substantive cognitive argument – in addition to the formal statistical argument invoking regression toward the mean – for expecting the poor predictive performers of yesterday to catch up to the superstar predictors of today.