Skip to main content
Logical Fallacies

The Problem of Stationarity

Past data can mislead when the process that produced it changes. Taleb illustrates this with an urn whose contents change while someone samples from it, then applies the concern to economic forecasting and to people adapting their behavior to known patterns or policies.

THE IDEA

In some cases, if the incidence of red balls is itself randomly distributed, we will never get to know the composition of the urn. This is called “the problem of stationarity.” Think of an urn that is hollow at the bottom. As I am sampling from it, and without my being aware of it, some mischievous child is adding balls of one color or another. My inference thus becomes insignificant. I may infer that the red balls represent 50% of the urn while the mischievous child, hearing me, would swiftly replace all the red balls with black ones. This makes much of our knowledge derived through statistics quite shaky.

The very same effect takes place in the market. We take past history as a single homogeneous sample and believe that we have considerably increased our knowledge of the future from the observation of the sample of the past. What if vicious children were changing the composition of the urn? In other words, what if things have changed?

I have studied and practiced econometrics for more than half my life (since I was nineteen), both in the classroom and in the activity of a quantitative derivatives trader. The “science” of econometrics consists of the application of statistics to samples taken at different periods of time, which we called “time series.” It is based on studying the time series of economic variables, data, and other matters. In the beginning, when I knew close to nothing (that is, even less than today), I wondered whether the time series reflecting the activity of people now dead or retired should matter for predicting the future. Econometricians who knew a lot more than I did about these matters asked no such question; this hinted that it was in all likelihood a stupid inquiry. One prominent econometrician, Hashem Pesaran, answered a similar question by recommending to do “more and better econometrics.” I am now convinced that, perhaps, most of econometrics could be useless—much of what financial statisticians know would not be worth knowing. For a sum of zeros, even repeated a billion times, remains zero; likewise an accumulation of research and gains in complexity will lead to naught if there is no firm ground beneath it. Studying the European markets of the 1990s will certainly be of great help to a historian; but what kind of inference can we make now that the structure of the institutions and the markets has changed so much?

Note that the economist Robert Lucas dealt a blow to econometrics by arguing that if people were rational then their rationality would cause them to figure out predictable patterns from the past and adapt, so that past information would be completely useless for predicting the future (the argument, phrased in a very mathematical form, earned him the Swedish Central Bank Prize in honor of Alfred Nobel). We are human and act according to our knowledge, which integrates past data. I can translate his point with the following analogy. If rational traders detect a pattern of stocks rising on Mondays, then, immediately such a pattern becomes detectable, it would be ironed out by people buying on Friday in anticipation of such an effect. There is no point searching for patterns that are available to everyone with a brokerage account; once detected, they would be self-canceling.

Somehow, what came to be known as the Lucas critique was not carried through by the “scientists.” It was confidently believed that the scientific successes of the industrial revolution could be carried through into the social sciences, particularly with such movements as Marxism. Pseudoscience came with a collection of idealistic nerds who tried to create a tailor-made society, the epitome of which is the central planner. Economics was the most likely candidate for such use of science; you can disguise charlatanism under the weight of equations, and nobody can catch you since there is no such thing as a controlled experiment. Now, the spirit of such methods, called scientism by its detractors (like myself), continued past Marxism, into the discipline of finance as a few technicians thought that their mathematical knowledge could lead them to understand markets. The practice of “financial engineering” came along with massive doses of pseudoscience. Practitioners of these methods measure risks, using the tool of past history as an indication of the future. We will just say at this point that the mere possibility of the distributions not being stationary makes the entire concept seem like a costly (perhaps very costly) mistake. This raises the problem of induction: what justifies extending past observations to a future in which the underlying conditions may change?

When to use it

  • Using past stock returns to predict future performance may fail if market structure changes.

  • Economic policy analysis based on historical data may be invalid if people's behavior adapts to policy changes.

  • In business, relying on past sales trends to forecast future demand may be unreliable if consumer preferences shift.

  • Forecasting election outcomes based on past polling data may be flawed if the electorate's composition changes.

FROM THEORY TO EVERYDAY LIFE

In practice

01

Urn Analogy

Someone replaces balls in an urn while you sample from it. Your estimate may describe the earlier contents rather than the current ones. A forecast based on past economic data faces a similar problem when the conditions generating the data change.

02

Stock Market Pattern Example (Lucas Critique)

In Taleb's illustration, traders notice that stocks tend to rise on Mondays and buy on Friday to anticipate the gain. Their response can change or erase the pattern. This illustrates how behavior can adapt to a known regularity; it does not establish that every publicly known pattern disappears immediately.

✳

A MOMENT TO REFLECT

Put it to work.

  1. Think of a situation you believe you can predict based on past data. Could the underlying conditions have changed? How would you test for stationarity?

  2. How does the Lucas critique apply to your own decision-making? Can patterns in your personal or professional life be self-canceling?

  3. What are the limits of using historical data for financial or economic predictions?

Connected ideas

Another lens can bring the same problem into focus.

References

  1. Lucas Critique — Robert Lucas

CONTINUE THE CONVERSATION

Source texts

  • Fooled by Randomness The Hidden Role of Chance in the Markets and in Life (Nassim Nicholas Taleb)
← More logical fallacies