St. Petersburg and Ellsberg

ST. PETERSBURG AND ELLSBERG

In this extract from How Not to Be Wrong, Jordan Ellenberg explains the puzzles and their implications for decision theory.

Utils are useful when making decisions about items that don’t have well-defined dollar values, like wasted time or unpleasant meals. But you also need to talk about utility when dealing with items that do have well-defined dollar values—like dollars.

This realization arrived very early in the development of probability theory. Like many important ideas, it entered the conversation in the form of a puzzle. Daniel Bernoulli famously described the conundrum in his 1738 paper “Exposition on a New Theory of the Measurement of Risk”: “Peter tosses a coin and continues to do so until it should land ‘heads’ when it comes to the ground. He agrees to give Paul one ducat if he gets ‘heads’ on the very first throw, two ducats if he gets it on the second, four if on the third, eight if on the fourth, and so on, so that with each additional throw the number of ducats he must pay is doubled.”

This is obviously a rather attractive scenario for Paul, a game he should be willing to ante up some entrance fee to play. But how much? The natural answer, given our experience with lotteries, is to compute the expected value of the amount of money Paul gets from Peter. There’s a 50/50 chance that the first throw of the coin lands heads, in which case Paul gets one ducat. If the first throw is tails and the second is heads, an event which happens 1/4 of the time, Paul gets two ducats. To get four, the first three throws have to fall tails, tails, heads, which happens with probability 1/8. Carrying on and adding up, Paul’s expected profit is

(1/2) × 1 + (1/4) × 2 + (1/8) × 4 + (1/16) × 8 + (1/32) × 16 + . . .

or

1/2 + 1/2 + 1/2 + 1/2 + . . .

That sum is not a number. It’s divergent; the more terms you add, the bigger the sum gets, growing without bound past any finite threshold. This seems to suggest that Paul should be willing to spend any number of ducats for the right to play this game.

That sounds nuts. And it is! But when the math tells us something that sounds nuts, mathematicians don’t just shrug and walk away. We go hunting for the kink in the tracks where either the math or our intuition has gone off the rails. The conundrum, known as the St. Petersburg paradox, had been devised by Nicolas Bernoulli, Daniel’s cousin, some thirty years before, and many of the probabilists of the time had puzzled over it without coming to any satisfying conclusion. The younger Bernoulli’s beautiful untwisting of the paradox is a landmark result, and one that has formed the foundation of economic thinking about uncertain values ever since. The mistake, Bernoulli said, is to say that a ducat is a ducat is a ducat. A ducat in the hand of a rich man is not worth the same as a ducat in the hand of a peasant, as is plainly visible from the different levels of care with which the two men treat their cash. In particular, having two thousand ducats isn’t twice as good as having one thousand; it is less than twice as good, because a thousand ducats is worth less to a person who already has a thousand ducats than it is to the person who has none. Twice as many ducats doesn’t translate into twice as many utils; not all curves are lines, and the relation between money and utility is governed by one of those nonlinear curves.

Bernoulli thought that utility grew like the logarithm, so that the kth prize of 2^k ducats was worth just k utils. Remember, we can think of the logarithm as more or less the number of digits: so in dollar terms, Bernoulli’s theory is saying that rich people measure the value of their pile by the number of digits after the dollar sign—a billionaire is as much richer than a hundred-millionaire as the hundred-millionaire is richer than the ten-millionaire.

In Bernoulli’s formulation, the expected utility of the St. Petersburg game is the sum

(1/2) × 1 + (1/4) × 2 + (1/8) × 3 + (1/16) × 4 + . . .

This tames the paradox; this sum, it turns out, is no longer infinite, or even very large. In fact, there’s a beautiful trick that allows us to compute it exactly:

Imagine arranging terms in a triangle. The first row contains successive halves; each later row starts one column farther right and repeats the part of the first row from that point onward. The sum of the first row, (1/2) + (1/4) + (1/8) + . . . , is 1; this is the very infinite series that Zeno encountered in chapter 2. The second row is the same as the first, but with every entry divided by 2; so its sum must be half the sum of the first row, or 1/2. By the same reasoning, the third row, which is just the second row with each term halved, must have half the sum of the second row; so 1/4. Now the sum of all the numbers in the triangle is 1 + 1/2 + 1/4 + 1/8 + . . . ; just one more than Zeno’s sum, which is to say, 2.

But what if we sum down the columns first instead of the rows? Just as with the holes in my parents’ stereo set, it can’t matter whether we start counting vertically or horizontally; the sum is what the sum is. In the first column there is just a single 1/2; in the second, there are two copies of 1/4, making (1/4) × 2; in the third, three copies of 1/8, making (1/8) × 3, and so on. The series formed by the column sums is none other than the sum Bernoulli set up to study the St. Petersburg problem. And its sum is the sum of all the numbers in the infinite triangle, which is to say: 2. So the amount Paul should pay is the number of ducats his personal utility curve tells him 2 utils is worth.

The shape of the utility curve, beyond the bare fact that it tends to bend downward as the money increases, is impossible to pin down precisely, though contemporary economists and psychologists are constantly devising ever-more-intricate experiments to refine our understanding of its properties. (“Now just get your head settled comfortably at the center of the fMRI, if you don’t mind, and I’m going to ask you to rank the following six poker strategies in order from most enticing to least enticing, and after that, if you wouldn’t mind just holding still while my postdoc takes this cheek swab . . . ?”)

We know, at least, that there is no universal curve; different people in different contexts assign different utilities to money. This fact is important. It gives us pause, or it ought to, when we start making generalizations about economic behavior. Greg Mankiw, the Harvard economist, wrote a widely circulated blog post in 2008 explaining that increased income taxes proposed by presidential candidate Barack Obama would lead him to slack off at work. After all, Mankiw was already at an equilibrium, where the utility of the dollars he’d earn from another hour of work would be exactly canceled by the negative utility imposed by the loss of an hour with his kids. Diminish the number of dollars Mankiw makes per hour, and that trade stops being worth it; he cuts back on work until he drops to the income level where an hour with his kids is worth the same to him as an hour spent working for his Obama-diminished pay. He agrees with Reagan’s view of the economy as seen from the standpoint of a cowboy-movie star; when the tax rate goes up, you make fewer cowboy movies.

But not everybody is Greg Mankiw. In particular, not everybody has the same utility curve he has. The comic essayist Fran Lebowitz tells a story about her youth in Manhattan, driving a cab. She started driving at the beginning of the month, she said, and kept driving every day until she’d made enough money to pay for rent and food. Then she stopped driving and wrote for the rest of the month. For Lebowitz, all money above a certain threshold contributes essentially zero further utility; she has a different-looking curve than Mankiw does. Hers goes flat once her rent is paid. What happens to Fran Lebowitz if income taxes go up? She works more, not less, to bring herself back up to the threshold.

Bernoulli was not the only mathematician to arrive at the idea of utility and its nonlinear relation with money. He’d been anticipated by at least two other researchers. One was Gabriel Cramer of Geneva; the other was a young correspondent of Cramer’s, none other than the needle thrower Georges-Louis LeClerc, Comte de Buffon. Buffon’s interest in probability was not restricted to parlor games. Late in life, he reminisced about his encounter with the vexing St. Petersburg paradox: “I dreamed about this problem some time without finding the knot; I could not see that it was possible to make mathematical calculations agree with common sense without introducing some moral considerations; and having expressed my ideas to Mr. Cramer, he told me that I was right, and that he had also resolved this question by a similar approach.”

Buffon’s conclusion mirrored Bernoulli’s, and he perceives the nonlinearity especially clearly:

Money must not be estimated by its numerical quantity: if the metal, that is merely the sign of wealth, was wealth itself, that is, if the happiness or the benefits that result from wealth were proportional to the quantity of money, men would have reason to estimate it numerically and by its quantity, but it is barely necessary that the benefits that one derives from money are in just proportion with its quantity; a rich man of one hundred thousand ecus income is not ten times happier than the man of only ten thousand ecus; there is more than that what money is, as soon as one passes certain limits it has almost no real value, and cannot increase the well-being of its possessor; a man that discovered a mountain of gold would not be richer than the one that found only one cubic fathom.

The doctrine of expected utility is appealingly straightforward and simple: presented with a set of choices, pick the one with the highest expected utility. It is perhaps the closest thing we have to a simple mathematical theory of individual decision making. And it captures many features of the way humans make choices, which is why it remains a central part of the quantitative social scientist’s tool kit. Pierre-Simon Laplace, on the last page of his 1814 treatise A Philosophical Essay on Probabilities, writes, “We see, in this Essay, that the theory of probabilities is, in the end, only common sense boiled down to ‘calculus’; it points out in a precise way what rational minds understand by means of a sort of instinct, without necessarily being aware of it. It leaves nothing to doubt, in the choice of opinions and decisions; by its use one can always determine the most advantageous choice.”

Again we see it: mathematics is the extension of common sense by other means.

But expected utility doesn’t get at everything. Once again, the troubling complications enter in the form of a puzzle. This time, the puzzle-bearer was Daniel Ellsberg, who later became famous as the whistle-blower who leaked the Pentagon Papers to the civilian press. (In mathematical circles, which can be parochial at times, it would not be outlandish to hear it said of Ellsberg, “You know, before he got involved in politics, he did some really important work.”)

In 1961, a decade before his explosion into public view, Ellsberg was a brilliant young analyst at the RAND Corporation, consulting with the U.S. government on strategic matters surrounding nuclear war—how it could be prevented, or, barring that, effectively conducted. At the same time, he was working toward a Harvard PhD in economics. On both tracks, he was thinking deeply about the process by which human beings made decisions in the face of the unknown. At the time, the theory of expected utility held a supreme position in the mathematical analysis of decisions. Von Neumann and Morgenstern, in their foundational book The Theory of Games and Economic Behavior, had proven that all people who obeyed a certain set of behavior rules, or axioms, had to act as if their choices were governed by the drive to maximize some utility function. These axioms—later refined by Leonard Jimmie Savage, a member of the wartime Statistical Research Group with Abraham Wald—were the standard model of behavior under uncertainty at the time.

Game theory and expected utility theory still play a great role in the study of negotiations among people and states, but never more so than at RAND at the height of the cold war, where the writings of von Neumann and Morgenstern were the subject of Pentateuchal levels of reverence and analysis. The researchers at RAND were studying something fundamental to human life: the process of choice and competition. And the games they studied, like Pascal’s wager, were played for very high stakes.

Ellsberg, the young superstar, had a taste for crossing up established expectations. After graduating third in his class at Harvard, he had startled his intellectual comrades by enlisting in the Marine Corps, where he served for three years as an infantryman. In 1959, as a Harvard Junior Fellow, he delivered a lecture on strategy in foreign policy at the Boston Public Library, in which he famously contemplated the effectiveness of Adolf Hitler as a geopolitical tactician: “There is the artist to study, to learn what can be hoped for, what can be done with the threat of violence.” (Ellsberg always insisted that he didn’t recommend that the United States adopt Hitler-style strategies, but only wanted to make dispassionate study of their effectiveness—maybe so, but it’s hard to doubt he was trying to get a rise out of his audience.)

So it’s perhaps no surprise that Ellsberg was not content to accept the prevailing views. In fact, he’d been picking at the foundations of game theory since his undergraduate senior thesis. At RAND, he devised a famous experiment now known as Ellsberg’s paradox.

Suppose there’s an urn with ninety balls inside. You know that thirty of the balls are red; concerning the other sixty balls, you know only that some are black and some are yellow. The experimenter describes to you the following four bets.

• RED: You get $100 if the next ball pulled from the urn is red; otherwise, you get nothing.
• BLACK: You get $100 if the next ball is black, otherwise nothing.
• NOT-RED: You get $100 if the next ball is either black or yellow, otherwise nothing.
• NOT-BLACK: You get $100 if the next ball is either red or yellow, otherwise nothing.

Which bet do you prefer; RED or BLACK? What about NOT-RED versus NOT-BLACK?

Ellsberg quizzed his subjects to find out which of these bets they preferred, given the choice. What he found was that the people he polled tended to prefer RED to BLACK. With RED, you know where you stand: you’ve got a 1-in-3 chance of getting the money. With BLACK, you have no idea what odds to expect. As for NOT-RED and NOT-BLACK, the situation is just the same; Ellsberg’s subjects liked NOT-RED better, preferring the state of knowing that their chance of a payoff is exactly 2/3.

Now suppose you have a more complicated choice: you have to pick two of the bets. And not any two you like: you have to take either “RED and NOT-RED” or “BLACK and NOT-BLACK.” If you prefer RED to BLACK and NOT-RED to NOT-BLACK, it seems reasonable that you prefer “RED and NOT-RED” to “BLACK and NOT-BLACK.”

But now here’s the problem. Picking RED and NOT-RED is the same thing as giving yourself $100. But so is BLACK and NOT-BLACK! How can one be preferable to the other when they’re the same thing?

For a proponent of expected utility theory, Ellsberg’s results looked very strange. Each bet must be worth a certain number of utils, and if RED has more utility than BLACK, and NOT-RED more than NOT-BLACK, it just has to be the case that RED + NOT-RED is worth more utils than BLACK + NOT-BLACK; but the two are the same. If you want to believe in utils, you have to believe that the participants in Ellsberg’s study are just plain wrong in their preferences; they are bad at calculating, or they’re not paying close attention to the question, or they’re simply crazy. Since the people Ellsberg asked were in fact well-known economists and decision theorists, this conclusion presents its own problems for the status quo.

For Ellsberg, the answer to the paradox is simply that expected utility theory is incorrect. As Donald Rumsfeld would later put it, there are known unknowns and there are unknown unknowns, and the two are to be processed differently. The “known unknowns” are like RED—we don’t know which ball we’ll get, but we can quantify the probability that the ball will be the color we want. BLACK, on the other hand, subjects the player to an “unknown unknown”—not only are we not sure whether the ball will be black, we don’t have any knowledge of how likely it is to be black. In the decision-theory literature, the former kind of unknown is called risk, the latter uncertainty. Risky strategies can be analyzed numerically; uncertain strategies, Ellsberg suggested, were beyond the bounds of formal mathematical analysis, or at least beyond the bounds of the flavor of mathematical analysis beloved at RAND.

None of which is to deny the incredible utility of utility theory. There are many situations, lotteries being one, where the mystery we’re subject to is all risk, governed by well-defined probabilities; and there are many more circumstances where “unknown unknowns” are present but play only a small role. We see here the characteristic push and pull of the mathematical approach to science. Mathematicians like Bernoulli and von Neumann construct formalisms that apply a penetrating light to a sphere of inquiry only dimly understood before; mathematically fluent scientists like Ellsberg work to understand the limits of those formalisms, to refine and improve them where it’s possible to do so, and to post strongly worded warning signs where it’s not.

Ellsberg’s paper is written in a vivid, literary style uncharacteristic of technical economics. In his concluding paragraph, he writes of his experimental subjects that “the Bayesian or Savage approach gives wrong predictions and, by their lights, bad advice. They act in conflict with the axioms deliberately, without apology, because it seems to them the sensible way to behave. Are they clearly mistaken?”

In the world of cold war Washington and RAND, decision theory and game theory were held in the highest intellectual esteem, seen as the scientific tools that would win the next world war, as the atom bomb had won the last one. That those tools might actually be limited in their application, especially in contexts for which there was no precedent and thus no means of estimating probabilities—like, say, the instantaneous reduction of the human race to radioactive dust—must have been at least a little troubling for Ellsberg. Was it here, over a disagreement about math, that his doubts about the military establishment really began?