Law of Large Numbers

The engine driving the Kelly system is the law of large numbers. In a 1713 treatise on probability, Swiss mathematician Jakob Bernoulli propounded a law that has been misunderstood by gamblers (and investors) ever since. — William Poundstone

The Law of Large Numbers (LNN) is a fundamental theorem of probability theory. It states that if the same random experiment is repeated many times (tending to infinity), the frequency of a particular outcome converges to its theoretical probability. In other words, as the number of trials increases, the observed proportion of an event becomes a more and more reliable reflection of its true probability.

The law is the backbone of what some thinkers call Mediocristan—the domain of thin‑tailed, well‑behaved randomness where no single observation can dramatically skew the aggregate. In Mediocristan, uncertainty softens and eventually vanishes under averaging: when you sum a large number of small, independent random shocks, the total becomes predictable and tightly clustered around its mean. A coffee cup sitting on a table is a perfect illustration. The cup is made of trillions of minuscule particles, each jiggling randomly. If the cup were a single particle, its thermal dance would be wild and unpredictable. But because it is the sum of an astronomical number of tiny movements, the cup sits still—its average behaviour is rock‑steady.

History

The idea that relative frequencies stabilize over many trials was first mentioned by Girolamo Cardano in the 16th century, though without rigorous proof. Jakob Bernoulli provided the first formal demonstration in his work Ars Conjectandi (1713). In the early 1800s, Siméon‑Denis Poisson refined the theory and coined the term “la loi des grands nombres.” The principle was vividly illustrated during the 20th century by J. E. Kerrich, who, while interned in Denmark, flipped a coin 10,000 times and recorded the proportion of heads steadily approaching 50%.

Examples

Rolling a Die

Consider a fair six‑sided die and the event of rolling a 1. The probability of this event is 1/6 (about 16.67%). If you roll the die only 10 or 20 times, the observed frequency of 1 might be 5% or 30%. But as the number of rolls grows to 10,000 or more, the frequency will converge toward 16.67%.

A Coin‑Flipping Game

Imagine two teams flipping fair coins. Team Small flips only 10 coins each, while Team Big flips 100. With just 10 flips, some players will easily land extreme results—like 80% heads—purely by chance. Team Big’s players, with 100 flips, will see proportions much closer to 50%. With 1,000 flips, the clustering around 50% becomes even tighter. This illustrates how larger samples suppress variance and bring observed frequencies closer to the expected probability.

Brain Cancer Rates by State

When comparing brain cancer rates across U.S. states, small‑population states often appear at the top or bottom of the list. South Dakota might show the highest rate one year, not because of a genuine health crisis but because its small population makes the rate highly sensitive to random fluctuations. The Law of Large Numbers tells us that extreme values in small samples are more likely to be noise than signal.

Kerrich’s Coin Tosses

While interned in Denmark, mathematician J. E. Kerrich flipped a coin 10,000 times. The proportion of heads wavered early but gradually converged to very nearly 50%, providing an empirical demonstration of the law that complemented Bernoulli’s theoretical proof.

The Casino’s Edge

Casino operators rely on the Law of Large Numbers to sleep soundly. A single gambler might get lucky and win a large sum in the short term, but the casino hosts thousands of players making limited‑size bets. The house caps the maximum bet per round, ensuring that no single outcome can threaten the casino’s solvency. Even if the total amount wagered across all tables is millions of dollars, the typical bet might be around $20. With so many independent wagers, the casino’s cumulative profit converges tightly to its theoretical edge. The variance is minuscule relative to the total handle, and the overall result becomes virtually a foregone conclusion. This is the supreme law of Mediocristan in action: when you have many players, no individual can influence the aggregate beyond a negligible ripple.

Common Misconceptions

The law does not imply that a handful of observations should mirror the theoretical probability. Small samples are prone to extreme or erratic results, and the law offers no timetable for convergence. It is equally wrong to believe that a streak of one outcome forces an imminent “balancing” streak of the other—that is the gambler’s fallacy. The law has no memory; it only guarantees that, as trials continue, the cumulative fraction gets closer and closer to the true probability, not that short‑term deviations will be corrected.

The law’s reliability is also contingent on staying inside Mediocristan. It washes out uncertainty only when individual observations are well behaved—independent, with finite variance, and none so large that it can dominate the total. When these conditions break (in what is called Extremistan), averages no longer tighten around a mean and the Law of Large Numbers loses its grip.