Inverse Problem Fallacy

From Reality to Representation

Let me take another angle. The passage from theory to the real world presents two distinct difficulties: inverse problems and pre-asymptotics.

Inverse Problems. Recall how much more difficult it is to re-create an ice cube from the results of the puddle (reverse engineering) than to forecast the shape of the puddle. In fact, the solution is not unique: the ice cube can be of very many shapes. I have discovered that the Soviet-Harvard method of viewing the world (as opposed to the Fat Tony style) makes us commit the error of confusing the two arrows (from ice cube to puddle; from puddle to ice cube). It is another manifestation of the error of Platonicity, of thinking that the Platonic form you have in your mind is the one you are observing outside the window. We see a lot of evidence of confusion of the two arrows in the history of medicine, the rationalistic medicine based on Aristotelian teleology, which I discussed earlier. This confusion is based on the following rationale. We assume that we know the logic behind an organ, what it was made to do, and thus that we can use this logic in our treatment of the patient. It has been very hard in medicine to shed our theories of the human body. Likewise, it is easy to construct a theory in your mind, or pick it up from Harvard, then go project it on the world. Then things are very simple.

This problem of confusion of the two arrows is very severe with probability, particularly with small probabilities.

As we showed with the undecidability theorem and the self-reference argument, in real life we do not observe probability distributions. We just observe events. So I can rephrase the results as follows: we do not know the statistical properties—until, of course, after the fact. Given a set of observations, plenty of statistical distributions can correspond to the exact same realizations—each would extrapolate differently outside the set of events from which it was derived. The inverse problem is more acute when more theories, more distributions can fit a set of data, particularly in the presence of nonlinearities or nonparsimonious distributions. Under nonlinearities, the families of possible models/parametrization explode in numbers.

But the problem gets more interesting in some domains. Recall the Casanova problem in Chapter 8. For environments that tend to produce negative Black Swans, but no positive Black Swans (these environments are called negatively skewed), the problem of small probabilities is worse. Why? Clearly, catastrophic events will be necessarily absent from the data, since the survivorship of the variable itself will depend on such effect. Thus such distributions will let the observer become prone to overestimation of stability and underestimation of potential volatility and risk.

This point—that things have a bias to appear more stable and less risky in the past, leading us to surprises—needs to be taken seriously, particularly in the medical field. The history of epidemics, narrowly studied, does not suggest the risks of the great plague to come that will dominate the planet. Also I am convinced that in doing what we are to the environment, we greatly underestimate the potential instability we will experience somewhere from the cumulative damage we have done to nature.

One illustration of this point is playing out just now. At the time of writing, the stock market has proved much, much riskier than innocent retirees were led to believe from historical discourses showing a hundred years of data. It is down close to 23 percent for the decade ending in 2010, while the retirees were told by finance charlatans that it was expected to rise by around 75 percent over that time span. This has bankrupted many pension plans (and the largest car company in the world), for they truly bought into that “empirical” story—and of course it has caused many disappointed people to delay their retirement. Consider that we are suckers and will gravitate toward those variables that are unstable but that appear stable.

Preasymptotics. Let us return to Platonicity with a discussion of preasymptotics, what happens in the short term. Theories are, of course, a bad thing to start with, but they can be worse in some situations when they were derived in idealized situations, the asymptote, but are used outside the asymptote (its limit, say infinity or the infinitesimal). Mandelbrot and I showed how some asymptotic properties do work well preasymptotically in Mediocristan, which is why casinos do well; matters are different in Extremistan.

Most statistical education is based on these asymptotic, Platonic properties, yet we live in the real world, which rarely resembles the asymptote. Statistical theorists know it, or claim to know it, but not your regular user of statistics who talks about “evidence” while writing papers. Furthermore, this compounds what I called the ludic fallacy: most of what students of mathematical statistics do is assume a structure similar to the closed structures of games, typically with a priori known probability. Yet the problem we have is not so much making computations once you know the probabilities, but finding the true distribution for the horizon concerned. Many of our knowledge problems come from this tension between a priori and a posteriori.