Response Biases

Response Biases

Cultures sometimes differ in how they tend to use response scales (e.g., Pope, 1991; Zax & Takahashi, 1967). This suggests that cross-cultural variations in overconfidence might be simply another demonstration of differences in response biases. Here, the proposition would be that Asians, relative to Westerners, are more inclined to report probability judgments that are more extreme than they really feel are warranted. Yates, Lee, and Bush (1997) tested the response bias hypothesis using an approach that extended and refined one used by Fischhoff, Slovic, and Lichtenstein (1977). They had American and Chinese respondents state selling prices for real bets that paid respondents $2.20 (or its equivalent in Taiwan) if their answers to general-knowledge questions were right and no payment if their answers were wrong. Respondents’ prices did indeed agree with the overconfident probability judgments they had stated for those answers, providing evidence against the response bias proposition.

Proper scoring rules are procedures for compensating people for the accuracy of their probability judgments (cf. Yates, 1990, Chap. 8). The more closely a person’s judgments correspond to the events that actually occur, the better the score; the better the score, the more money, for instance, that an evaluator awards to the person. What makes a scoring rule “proper” is that, given certain reasonable assumptions, it is in the person’s material interests to be candid in reporting what his or her true judgments really are. Put another way, reporting judgments that differ from true judgments leads to the expectation of worse compensation. Yates, Lee, Shinotsuka, and Sieck (2000) used a proper scoring rule to provide another test of the response bias proposal. They promised their Chinese, Japanese, and American respondents financial bonuses for the accuracy of their general-knowledge probability judgments, bonuses tied to a proper scoring rule whose properness had been explained to the respondents. Yates et al. observed the same pattern of cross-cultural variations in overconfidence as had been seen previously. Thus, once again, there was no support for the hypothesis that such variations are due to response biases.

These studies did not support the hypothesis that the observed cross-cultural differences were merely differences in reporting style. They leave the underlying differences in judgment to be explained.