Base Rate Fallacy

22 Base Rate

Tuomas W. Manninen

According to the National Crime Victimization Survey by the Bureau of Justice Statistics, between 1999 and 2011, the number of whites who were killed by the police was 2151, which is nearly twice the number of blacks killed by the police during the same period, 1130. This looks more like black privilege.

Collected from the Internet

The base rate fallacy (also called neglecting base rates, base rate neglect, or base rate bias; in some legal contexts, a form of the prosecutor’s fallacy) is a mistake in probabilistic reasoning. It occurs when available general information – the base rate, which describes how common something is in a population – is ignored or downplayed, and attention is given only to specific case information (e.g., the result of a test or an anecdote). Specific evidence can legitimately change a probability; the mistake is to ignore a relevant base rate without a reason.

A simple illustration: a mother insists her son will be admitted to a highly selective school for brilliant children because he is, indeed, brilliant. However, the school accepts only 6% of applicants. If brilliance is common among applicants, it does not by itself distinguish him from those competing for the 6% of places. The 6% acceptance rate is a starting point; relevant information about his application may change his chances. This is base rate neglect.

---

Consider a more elaborate scenario, adapted from Philip K. Dick’s short story “Minority Report” (and the later film). In the fictional New York City of the 2050s, crime rates are said to have dropped by 99.8% thanks to a “PreCrime” unit using Precogs. For a separate numerical illustration, suppose a classifier has 99.8% sensitivity (true positive rate) and 99.8% specificity (true negative rate). That means only a 0.2% chance of false positives and false negatives. If you are identified by PreCrime as a future criminal, what is the chance you are actually innocent?

At first glance, with such impressive accuracy, you appear almost certainly guilty. But the crucial missing piece is the base rate of criminal activity. For this hypothetical calculation, take the 2014 rate quoted in the source as 0.006393 (639.3 per 100,000) and a hypothetical population of 23,462,000. Treat that rate as the probability that an individual belongs to the guilty category. An event rate does not automatically supply that probability in a real population; the assumption is made here to illustrate the calculation. Applying the stated sensitivity and specificity gives these expected counts, rounded to whole cases:

Result	Means	Probability	# of Cases
True Positive	guilty, identified as guilty	0.998	149,693
True Negative	innocent, identified as innocent	0.998	23,265,383
False Positive	innocent, identified as guilty	0.002	46,624
False Negative	guilty, identified as innocent	0.002	300

For every 149,693 true positives, there are 46,624 false positives – a ratio of roughly 3 to 1. So, despite the 99.8% accuracy, a positive identification means you are only about 76% likely to be guilty, not 99.8%. Neglecting the base rate (that crime is rare in the first place) makes the system look far more decisive than it actually is.

---

The same fallacy surfaces in many real-life judgments. For example, some people point to faith healings as evidence for divine intervention: a certain number of cancer patients recover after prayer. Some cancers can undergo spontaneous remission. A useful comparison needs patients with similar cancers, stages, treatments, and observation periods. Recovery after prayer alone does not establish an effect beyond the expected outcomes in a comparable group. Ignoring the base rate of natural remission leads to an unwarranted conclusion of a miracle.

A historical policy example concerns U.S. states that required drug testing for welfare applicants. The case raises a question about how many positive results policy makers should expect. Covert and Israel (2015) found that in the seven states with such laws, positive test rates ranged from 0.002% to 8.3%, with all but one below 1%, versus a national drug use rate of 9.4%. Those figures challenge the expectation of widespread positive tests. But a selected program’s test results and a national survey’s drug-use rate may cover different populations, substances, or periods; they should not be treated as interchangeable measures of prevalence.

The epigraph’s argument about police killings follows the same flawed pattern:

1. Between 1999 and 2011, the number of whites killed by the police was 2151.
2. Between 1999 and 2011, the number of blacks killed by the police was 1130.
3. Implied: 2151 > 1130.
4. Therefore, claims of “white privilege” in police violence are inaccurate.

Absolute counts do not establish which population faced the higher per-capita rate. Before relying on the epigraph’s figures, check that the source actually measures police killings and uses comparable racial categories and time periods. According to the 2010 Census, the U.S. was 72.4% white and 12.6% black. When we calculate proportional rates:

1. Whites killed per proportion of population: 2151 / 0.724 ≈ 2971 (scaled to a hypothetical equal‑sized population).
2. Blacks killed per proportion of population: 1130 / 0.126 ≈ 8968.
3. 8968 > 2971.
4. Therefore, when the accurate base rates are applied, the supplied counts imply a higher per-capita rate for black people. That disparity does not by itself establish its causes.

---

How do we avoid the fallacy? The remedy is straightforward: always ask for the base rate. What is the overall frequency of the trait, behavior, or event in the relevant population? Then incorporate that information using Bayes’ rule or simple proportional reasoning before drawing any conclusion. In the police-killing comparison, population proportions change how the supplied counts should be interpreted. In the PreCrime example, adding the low crime rate reveals the true predictive value of the test. In the welfare drug‑testing case, checking the actual prevalence of drug use among applicants exposes the faulty assumption.

Exceptions and a final tip. If reliable, specific factors genuinely alter the statistical probabilities, it is not fallacious to adjust one’s expectations. For instance, if the mother knows her brilliant son also possesses a unique talent that the school disproportionately favors, a probability higher than the base 6% might be justified – but near-certainty would require much stronger evidence. The key is that the deviation must be in line with factors known to shift the odds. Use a relevant base rate as a starting point, then ask how much the specific evidence should change it.