Recognition of the Operation of Chance Factors

intermediate2 min readeducation · sociology · sports · statistics · psychology

The recognition that chance factors influence outcomes encourages the use of statistical heuristics. This is easier in domains with obvious randomness (e.g., sports, test performance) but subtler in social judgments like interviews.

THE IDEA

Recognition of the Operation of Chance Factors

A second major factor encouraging the use of statistical heuristics is the recognition of the role of chance in producing events in a given domain or in a particular situation. For example, statistical understanding of some types of sports is undoubtedly facilitated by the manifestly random component in the movement of the objects used: “A football can take funny bounces.” The random component probably does not have to be physical in order for people to recognize it. It is possible to recognize the unpredictability of academic test performance by repeated observations of one’s own outcomes. Even with one’s own efforts and the group against which one is competing held constant, outcomes can vary. One may even recognize that one’s performance on particular occasions was particularly good or poor because of accidents: “I just happened to reread that section because Jill never called me back”; “It was very noisy in the study area that night so I didn’t get a chance to review my notes.”

In contrast, cues as to randomness in the production of events are much subtler for other kinds of events, especially for many social ones. When we interview someone, what signs would let us know that a particular topic got explored just by chance or that the person seems dour and lackluster because of an uncharacteristic attempt to appear dignified rather than because of a phlegmatic disposition? In addition, as Einhorn and Hogarth (1978) pointed out, the gatekeeping function of the interview may serve to prevent us from recognizing the error variance in our judgments: The great talent of some people not hired or admitted may never be observed. In most situations, cues as to the fact that an interview ought to be regarded as a sample from a population, rather than a portrait in miniature, are missing. The same may be true for visits to a city, country, or university. One of us long believed that reports of raininess in England were greatly exaggerated because he once stayed in London for 10 days and it only drizzled twice!

When to use it

Evaluating employee performance: recognize that a single bad week might be due to chance factors rather than incompetence.
Assessing a sports team's winning streak: consider the role of random bounces before concluding they are the best.
Making hiring decisions: treat interview performance as a sample from a population, not a perfect predictor of future success.
Interpreting test scores: account for random variation in scores across different test administrations or study conditions.

FROM THEORY TO EVERYDAY LIFE

In practice

Randomness in sports

Statistical understanding of sports like football is aided by the manifestly random component in ball movement, as in the phrase 'A football can take funny bounces.' This recognition allows players and fans to account for chance in outcomes.

Variability in academic test performance

A student may notice that their own test scores vary even with consistent effort and competition, due to accidents like being interrupted while studying or having good luck with reviewed topics. This recognition encourages use of statistical reasoning about performance.

✳

A MOMENT TO REFLECT

Put it to work.

Think of a recent decision or judgment you made. Were you aware of the role of chance in the outcome? How might recognition of chance factors have changed your conclusion?
In what domains do you underestimate the role of randomness? How could you adjust your decision-making accordingly?
Consider an interview process: how could you design it to account for random variation in performance across candidates or occasions?