Naive Empiricism: The Fallacy of Confirmation from Past Data

Accidents Near Home

Suppose 20% of accidents happen within a twelve-mile radius of the driver’s home. Does that mean driving near home is especially dangerous? Not without knowing the exposure. If people spend 20% of their driving time within that radius, the accident share alone provides no evidence of greater risk per unit of driving time. Counting accidents without comparing how much people drive in each area is a typical example of naive empiricism.

But there is a more severe aspect of naive empiricism. For an unrestricted universal proposition, I can use data to find a counterexample, but a finite series of confirming observations cannot prove that it will always hold. For instance, the statement

The market never goes down 20% in a given three-month period

can be tested against past observations, but surviving those tests does not establish that it will hold in the future. I can quantitatively reject the proposition by finding counterexamples, but it is not possible for me to accept it simply because, in the past, the market never went down 20% in any three-month period (you cannot easily make the logical leap from "has never gone down" to "never goes down"). Samples can be greatly insufficient; markets may change; we may not know much about the market from historical information.

For such universal claims, finding a counterexample and accumulating confirming observations have different logical force. Why? Consider the following statements:

Statement A: No swan is black, because I looked at four thousand swans and found none.

Statement B: Not all swans are white.

I cannot logically make statement A, no matter how many successive white swans I may have observed in my life and may observe in the future (except, of course, if I am given the privilege of observing with certainty all available swans). It is, however, possible to make Statement B merely by finding one single counterexample. Indeed, the Australian Cygnus atratus, a swan variety that was jet black, supplied a counterexample to Statement A! The reader will see a hint of Popper's ideas, as there is a strong asymmetry between the two statements; and, furthermore, such asymmetry lies in the foundations of knowledge. It is also at the core of my operation as a decision maker under uncertainty.

People rarely test some of their testable statements; perhaps they would rather not face the consequences of the inference. The following inductive statement illustrates the problem of interpreting past data literally, without methodology or logic:

I have just completed a thorough statistical examination of the life of President Bush. For fifty-eight years, close to 21,000 observations, he did not die once. I can hence pronounce him as immortal, with a high degree of statistical significance.

Niederhoffer's publicized hiccup came from his selling naked options based on his testing and assuming that what he saw in the past was an exact generalization about what could happen in the future. He relied on the statement "The market has never done this before," so he sold puts that made a small income if the statement was true and lost hugely in the event of it turning out to be wrong. When he blew up, close to a couple of decades of performance were overshadowed by a single event that only lasted a few minutes.

Another logical flaw in this type of historical statement is that often when a large event takes place, you hear the "it never happened before," as if it needed to be absent from the event's past history for it to be a surprise. So why do we consider the worst case that took place in our own past as the worst possible case? If the past, by bringing surprises, did not resemble the past previous to it (what I call the past's past), then why should our future resemble our current past?

There is another lesson to his story, perhaps the greatest one: Niederhoffer appears to approach markets as a venue from which to derive pride, status, and wins against "opponents" (such as myself), as he would in a game with defined rules. He was a squash champion with a serious competitive streak; it is just that reality does not have the same closed and symmetric laws and regulations as games. This competitive nature got him into ferocious fighting to "win." Markets (and life) are not simple win/lose types of situations, as the cost of the losses can be markedly different from that of the wins. Maximizing the probability of winning does not lead to maximizing the expectation from the game when one's strategy may include skewness, i.e., a small chance of large loss and a large chance of a small win. If you engaged in a Russian roulette–type strategy with a low probability of large loss, one that bankrupts you every several years, you are likely to show up as the winner in almost all samples—except in the year when you are dead.

I remind myself never to fail to acknowledge the insights of the 1960s empiricist and his early contributions. Sadly, I learned quite a bit from Niederhoffer, mostly by contrast, and particularly from the last example: not to approach anything as a game to win, except, of course, if it is a game. Even then, I do not like the asphyxiating structure of competitive games and the diminishing aspect of deriving pride from a numerical performance. I also learned to stay away from people of a competitive nature, as they have a tendency to commoditize and reduce the world to categories, like how many papers they publish in a given year, or how they rank in the league tables. There is something nonphilosophical about investing one's pride and ego into a "my house/library/car is bigger than that of others in my category"—it is downright foolish to claim to be first in one's category all the while sitting on a time bomb.

To conclude, extreme empiricism, competitiveness, and an absence of logical structure to one's inference can be a quite explosive combination.