Preexisting Quantities

Preexisting Quantities

One type of general knowledge concerns the values of preexisting quantitative variables (e.g., the average price of a hotel room in a city one is considering visiting). Judgments about such quantities have been the subject of a different stream of general-knowledge overconfidence studies initiated by the work of Alpert and Raiffa (1982). Suppose the focal quantity is the average price for a hotel room in a certain city. Then by some suitable means, we should be able to elicit the 1st and 99th percentiles of the person’s subjective probability distribution for price; for example, $60 and $130, respectively. Thus, the person believes that there is only a 1% chance that the actual average room rate will be less than $60 and only a 1% chance that it will be more than $130. Of course, this also implies a belief that there is 98% chance that the average rate lies somewhere between those amounts. Expressed another way, the range from $60 to $130 constitutes a 98% credible interval for price (cf. Yates, 1990, Chap. 3).

Now, suppose that a person provides 98% credible intervals for a large number of quantities (e.g., hotel room rates in cities all over the world). If that person’s judgments are “well calibrated” in the distribution sense, then those intervals should capture the actual values of the respective quantities about 98% of the time. Equivalently, the true values should fall outside those intervals only rarely, approximately 2% of the time. Those unexpected extreme values are called surprises, more specifically, “2% surprises” in our current illustration. Suppose a person’s 98% credible intervals capture the true values of the pertinent quantities less often than 98%, say, in only about 80% of the instances. This would be taken as a case of “distribution overconfidence.” One rationale for this interpretation is that the person has expressed great confidence (98% certainty) in a class of events (capturing true quantity values within 98% credible intervals) that actually occurs much less often than anticipated (80% of the time). A surprise index is simply the proportion of times that true values occur outside the bounds of specified credible intervals. Accordingly, in our example, our target person’s 2% surprise index would be 20% rather than the 2% that ought to be observed for 98% credible intervals.

Massive distribution overconfidence has been the norm in studies of people’s quantity judgments (e.g., Alpert & Raiffa, 1982). Yates et al. (1989) sought to determine whether, generalizing Wright and Phillips’s findings with discrete facts, distribution overconfidence for quantitative facts would be significantly stronger among Chinese respondents than among American ones. That is indeed what they observed, for quantities such as “the area of Australia.” Whereas the 2% surprise index for the American respondents was an enormous 53.8%, it was even larger for the Chinese – 59.8%.