Skip to main content
Cognitive Biases

Test of Stationarity

This basketball study grouped consecutive shots into blocks and compared the observed hit counts with those expected under a constant, independent hit probability. Its reported tests generally did not detect departures from that model.

THE IDEA

Test of Stationarity

To detect temporary elevations in performance—periods in which a player’s hit rate is substantially above his overall average—the test of stationarity partitions a player’s entire shooting record into nonoverlapping sets of four consecutive shots. Within each set, performance is classified as high (three or four hits), moderate (two hits), or low (zero or one hit). If a player occasionally experiences “hot” streaks, his record should contain more high-performance sets than would be expected if the shots were independent and governed by a constant hit rate.

The expected proportions of high, moderate, and low sets under the constant-rate assumption are derived from the player’s overall shooting percentage. For a player with a 50% hit rate, for example, the expected proportions are 5/16 high, 6/16 moderate, and 5/16 low. A chi-square goodness-of-fit test is then used to compare the observed frequencies of the three categories to these expected values. To avoid any artifact introduced by an arbitrary starting point, the entire analysis is repeated four times—beginning the partition into quadruples at the first, second, third, and fourth shot of the player’s record—and the results are examined for consistency.

This approach was applied across multiple studies. In an initial investigation of nine major college basketball players, none of the chi-square values approached statistical significance, offering no support for the nonstationarity hypothesis. A subsequent study that tracked 100 shots per player yielded similar findings; for every player except one (Player 9), the test failed to detect any departure from stationarity. Even in that single exception, the evidence was not robust across all four starting-point partitions, so the reported tests did not provide consistent evidence of hot-hand streaks. Failure to detect a departure does not establish that the hit probability was constant.

When to use it

  • Testing for hot streaks in sports performance data (e.g., baseball, basketball).

  • Testing whether a binary success measure for a fund changes over time, without treating that test alone as evidence of investment skill.

  • For economic data recorded as binary events, compare block counts under an explicit constant-probability model.

  • Checking for changes in defect rates in a manufacturing process over time.

  • Checking whether evidence for a sports streak persists when the grouping starts at a different observation.

  • Checking whether a quality-control sequence has enough observations for an informative goodness-of-fit comparison.

  • Examining a sequence of market gains and losses while stating the independence and constant-probability assumptions.

  • Exploring changes in a sequence of binary clinical outcomes; a treatment effect requires a suitable comparison and study design.

FROM THEORY TO EVERYDAY LIFE

In practice

01

Expected proportion calculation for a 0.5 hit rate

For a player with a hit rate of 0.5, the expected proportions of high-, moderate-, and low-performance sets in blocks of four shots are 5/16, 6/16, and 5/16 respectively. If observed high-performance sets significantly exceed expectation, it suggests nonstationarity. The original study found no such evidence.

02

Basketball Shot Analysis

In a study of 100 shots per player, shots were divided into sets of four. The number of sets with high performance (3 or 4 hits) was compared to expected under stationarity. No evidence of hot hand was found except for one player.

✳

A MOMENT TO REFLECT

Put it to work.

  1. How does the stationarity test challenge the common belief in streak shooting?

  2. How would limited data affect your confidence in a test that did not detect changing hit rates?

  3. Can you think of other domains where a similar stationarity test could be applied to detect illusory patterns?

  4. Why does the analysis repeat the grouping from different starting shots?

  5. What is the difference between failing to reject a constant-rate model and establishing that it is true?

  6. If shot difficulty changes over a game, how might that affect the independence and constant-rate assumptions?

Connected ideas

Another lens can bring the same problem into focus.

CONTINUE THE CONVERSATION

Source texts

  • Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)
  • Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)
← More cognitive biases