Assessing Uncertainty in Physical Constants
Historical measurements of fundamental physical constants often underestimate true uncertainty, leading to errors that can be disconcertingly large relative to reported error bars. Examples include Millikan's oil-drop experiment and CP violation measurements, highlighting the need for subjective judgment and awareness of cognitive biases.
THE IDEA
Expert Judgment in Measurement
37. Assessing Uncertainty in Physical Constants *
Max Henrion and Baruch Fischhoff
The examples below reflect the authors’ historical account of measurement uncertainty.
Accurate estimates of the fundamental constants of physics, such as the velocity of light or the rest mass of the electron, are central to the enterprise of science (Pipkin & Ritter, 1983). Like any measurements, they are subject to uncertainties from a variety of sources. Reliable assessments of this uncertainty are needed (1) to compare the precision of different measurements of the same quantity; (2) to assess the accuracy of other quantities derived from them; and, most crucially, (3) to evaluate the consistency of physical theory with the current best measurements. Thus, as Eisenhart pointed out, “A reported value whose accuracy is entirely unknown is worthless” (1968, p. 1201).
It is not unusual to encounter individual examples of errors in measurements of physical quantities that turn out to be disconcertingly large relative to the estimated uncertainty. One well-known case was in Millikan’s oil-drop experiment in 1912 to determine e, whose result turned out 15 years later to be off by 0.6%, or three standard deviations, due to reliance on a faulty value for the viscosity of air (Cohen, Crowe, & Dumond, 1957). A more recent example concerns measurements of |v+−|, the parameter that measures the degree of violation of CP (charge–conjugation–parity) invariance. The six measurements prior to 1973 agreed reasonably, but more accurate measurements since then differ consistently by about seven standard deviations from the pre-1973 mean, a discrepancy that remains unexplained in terms of experimental procedure (Franklin, 1984). Such extreme cases may be exceptions, but they raise the more general question of how well on the average reported uncertainties reflect actual errors, an issue on which there has been little systematic study (Roos, Hietanen, & Luoma, 1975). These historical measurements illustrate why an uncertainty estimate needs scrutiny alongside the reported value.
A comprehensive assessment of uncertainty cannot rest solely on statistical analysis. Unavoidably, it involves a considerable element of subjective judgment. Cognitive psychology offers ways to investigate how that judgment can go wrong.
When to use it
Evaluating the reliability of published measurements in experimental physics or other sciences.
Teaching students about the pitfalls of underestimating uncertainty and the role of cognitive biases in scientific practice.
Informing policy or engineering decisions that depend on accurate knowledge of physical constants.
FROM THEORY TO EVERYDAY LIFE
In practice
Millikan's Oil-Drop Experiment (1912)
Millikan's measurement of the electron charge e was later found to be off by 0.6% (three standard deviations) due to a faulty value for the viscosity of air, revealing underestimated uncertainty.
CP Violation Parameter Measurements
Six measurements of |v+−| prior to 1973 agreed reasonably, but subsequent more accurate measurements differ by about seven standard deviations from the pre-1973 mean, indicating severe underestimation of uncertainty.
A MOMENT TO REFLECT
Put it to work.
How might overconfidence in measurement precision affect the reliability of scientific conclusions?
Can you think of a modern example where reported error bars were later found to be too small? What lessons does that offer?
Why is it important to consider subjective judgment in assessing uncertainty, even in quantitative sciences like physics?
References
- Pipkin & Ritter — Pipkin, F. M. & Ritter, R. C. (1983)
- Eisenhart — Eisenhart, C. (1968)
- Cohen, Crowe, & Dumond — Cohen, E. R., Crowe, K. M., & Dumond, J. W. M. (1957)
- Franklin — Franklin, A. (1984)
- Roos, Hietanen, & Luoma — Roos, M., Hietanen, I., & Luoma, J. (1975)
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)