Axiom Tests: Focus-Dependence and Context Independence
Experiments reveal that binary complementarity (P(A,B) + P(B,A) = 1) is violated under certain conditions, such as when participants have little knowledge or when unpacked hypotheses are repacked differently depending on position. Asymmetric support theory accounts for these violations while maintaining the product rule.
THE IDEA
Axiom Tests: Focus-Dependence and Context Independence
Another set of investigations has concerned the behavioral axioms that are implied by the representation relating probability to support. One requirement of support theory is that the position of a given hypothesis – whether in the focal position or the alternative position – does not affect its support. This assumption of position-independence is best seen in the equation relating judged probability to support, in which there is a single support scale s(·) that applies to both the focal and alternative hypotheses. Position-independence implies binary complementarity: P(A, B) + P(B, A) = 1. Studies described here document systematic violations of binary complementarity, suggesting potential position-dependent generalizations of support theory.
Macchi, Osherson, and Krantz (1999) asked participants to judge probabilities of hypotheses about which they had virtually no knowledge. For example, participants assessed the probability either that “the freezing point of gasoline is greater than that of alcohol” or that “the freezing point of alcohol is greater than that of gasoline.” It was assumed that most participants would be able to recruit very little support for either hypothesis. The average sum of judgments for such pairs of questions was roughly .90, indicating a violation of binary complementarity. Macchi et al. (1999) explain this result by suggesting that participants recognize that the focal hypothesis has little support, but fail to recognize that the alternative hypothesis has little support as well. This explanation was corroborated by subsequent experiments in which binary complementarity was restored for responses to modified questions that made both hypotheses salient. When reminded of the alternative hypothesis, people appear to symmetrically consider both the focal and alternative hypothesis. However, when not reminded of the alternative hypothesis, people may directly map weak support for the focal hypothesis into a low probability judgment, with little regard for the comparably weak support of the alternative hypothesis.
Brenner and Rottenstreich (1999) found a different pattern of position-dependence. Recall that many studies have shown that unpacking a hypothesis into its components generally increases support. However, a judge presented with an unpacked hypothesis (e.g., Bob is a chemical, industrial, mechanical, civil, or electrical engineer) may repack the disjunction and consider it as if it were a single hypothesis (e.g., Bob is an engineer). Brenner and Rottenstreich (1999) proposed that the position of a hypothesis affects the degree of repacking. In particular, they predicted a greater tendency to repack a disjunction when it is in the focal rather than the alternative position, based on the intuition that it is more natural to compare a single possibility to a set of alternatives than to compare a set of possibilities to a single alternative. This prediction of differential repacking implies that for judgments involving an explicit disjunction D1 ∨ D2 and some other singleton hypothesis S, the sum of P(D1 ∨ D2, S) and P(S, D1 ∨ D2) should be less than 1, contrary to binary complementarity. The results of several experiments, involving both probability and frequency judgments, supported this prediction.
The results of Macchi et al. (1999) and Brenner and Rottenstreich (1999) suggest certain circumstances in which binary complementarity may fail (see also McKenzie, 1998, 1999). It should be noted, however, that a number of studies have found binary complementarity to hold quite well across large sets of judgment items (Wallsten, Budescu, & Zwick, 1992; Tversky & Koehler, 1994; Fox, 1999), suggesting that binary complementarity is a good first approximation in many cases. Additional research is needed to map out the conditions under which binary complementarity does and does not hold.
Support theory can be generalized to accommodate violations of binary complementarity by allowing separate support scales for focal and alternative hypotheses:
This model – termed asymmetric support theory by Brenner and Rottenstreich (2000) – maintains the other central axiom of support theory, known as the product rule. Defining Q(A, B) = P(A, B)/[1 − P(A, B)] to be the “odds” of hypothesis A relative to hypothesis B, the product rule states that, for disjoint hypotheses A, B, C, and D:
The product rule requires that the support for a hypothesis does not depend on the hypothesis with which it is paired. This assumption greatly constrains the relationship between judged probability and evidential support. Empirical tests by Brenner and Rottenstreich (2000) supported the descriptive validity of the product rule, even when binary complementarity is violated, consistent with the asymmetric support theory model. Fox (1999) also reports empirical tests that support the product rule when binary complementarity holds. The success of direct support ratings in reproducing probability judgments is also an indirect validation of the separability of support that underlies the product rule.
When to use it
In legal reasoning, distinguish evidence about guilt from the probability judgment it supports, and consider the alternative explicitly. A complementarity check does not replace the applicable burden of proof.
In medical diagnosis, a doctor might focus on evidence for a particular disease and neglect to evaluate the support for alternative diagnoses, especially when the alternative is not explicitly highlighted.
When estimating a project’s chance of success, make the failure alternative explicit and compare the support for both outcomes.
In a risk review, define a hazard and its complement explicitly, then check whether judgments about the two sum to 1. Two different hazards need not be complements.
FROM THEORY TO EVERYDAY LIFE
In practice
Freezing Point Hypothesis
Participants judged the probability that “the freezing point of gasoline is greater than that of alcohol” or the reverse. Because they had little knowledge of either, the average sum of the two probabilities was about 0.90, violating binary complementarity (Macchi, Osherson, & Krantz, 1999).
Unpacked Engineering Hypothesis
When presented with an unpacked hypothesis such as “Bob is a chemical, industrial, mechanical, civil, or electrical engineer,” participants tended to repack the disjunction more when it was in the focal position than in the alternative position, leading to sums less than 1 (Brenner & Rottenstreich, 1999).
A MOMENT TO REFLECT
Put it to work.
Think of a recent decision where you judged the probability of an event. Did you consider the strength of the alternative hypothesis? How might ignoring the alternative have biased your judgment?
When you hear a complex, unpacked description of a possibility (e.g., a list of many reasons), do you tend to combine them mentally? How might the position of that description (focal vs alternative) change your probability estimate?
In what real-world situations might binary complementarity hold well, and when might it fail? Can you design a simple test with a friend using everyday hypotheses?
Often confused with
References
- — Macchi, Osherson, and Krantz (1999)
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)