Think Probabilistically About the World

THINK PROBABILISTICALLY ABOUT THE WORLD

Let’s go back to the day Donald Trump was elected president of the United States. When people are asked why they were shocked about the results, they often cite FiveThirtyEight.com, the website founded by Nate Silver that is famous for its predictions of everything from political elections to basketball championships, as a source they used to reach the conclusion that Hillary Clinton would win the election for sure.

It is true that FiveThirtyEight.com predicted that Hilary Clinton was more likely to win than Donald Trump. But the website’s final forecast (published the night before the election) gave Donald Trump a 29% chance of winning. This was based on several factors that the site considered to indicate a high level of uncertainty: the way the electoral college map was playing out, the fact that lots of people were still undecided days before the election, and the fact that polls were very volatile in the weeks preceding the election. While a 29% chance for Trump still meant that Hillary Clinton was a strong favorite to win, this probability was high enough that people should not have been so surprised when Trump won.

Yet many people were shocked with the outcome, including people who were checking the FiveThirtyEight.com website constantly before the election. Jonathan Borck, who works for an economic consulting firm in Boston, was one of them, despite the fact that he is a former teaching assistant of Richard’s and now teaches statistics at the Harvard Kennedy School. “I was shocked by the outcome. But I should not have been, especially given my status as a card-carrying statistician. Never again will I dismiss an event that has a 29 percent chance of occurring.”

Underlying all of this was a fundamental failure either to understand or be willing to grapple with what that 29 percent chance actually means. Mathematically, it means that if you were to repeat the “experiment” of running the election 100 times, you should expect Trump to win 29 times. Of course, the election only happens once, so this idea is a bit abstract. In the case of election forecasting, Nate Silver and his team run thousands of simulations where the election is played out on their computers many times. For each of these simulations, a different result emerges based on the probabilities that each candidate will win a given state and other statistical assumptions. For example, in simulation #1, Trump wins 34 states including the critical swing states Michigan and Wisconsin, whereas in simulation #2, Trump wins 31 states, but this time neither Michigan nor Wisconsin goes to Trump. And so on. For each simulation, a winner is determined based on the states each candidate won and the Electoral College votes each state carries. On the night before the election, after running 10,000 simulations, the FiveThirtyEight team observed that Trump won in roughly 2,900 of these simulations, which is where the 29 percent number came from. As my former colleague Lant Pritchett said at the time, “if there is a 29 percent chance of rain, I would bring an umbrella!”

Note that the correct interpretation of this 29 percent chance of winning the election is different from saying that the polls indicated that 29 percent of people were planning to vote for Trump. While the polls involve a considerable degree of uncertainty, if only 29 percent of sampled voters said they would vote for Trump, he would have had a minuscule chance of winning the election, far less than one tenth of one percent. Even if the polls were extremely inaccurate, say underestimating Trump’s chances by 10 percent, Trump’s chances would have been effectively close to zero.

The 29 percent probability is an example of a subjective probability. This means it is based on judgment and cannot be calculated in the same objective way we can calculate the likelihood of drawing an ace, king, queen or jack, but not the ace of spades, if you pull a card at random from a deck. Subjective probabilities are the only tool available when assessing a very wide range of real-world events (for example, there is a 60 percent chance that the price of a barrel of oil will be above $50 on June 30 of next year). They also apply to personal matters, to beliefs, and to one-time-only events. As an example, you may think that there is a 30% chance you will be promoted in your job next year. Deciding effectively in most important uncertain situations requires an assessment that uses subjective probabilities.

Thinking probabilistically about the world involves three distinct elements:

1. Understanding what these subjective probabilities actually mean (as we saw above),
2. Assigning probabilities to many things (events, beliefs, etc.) in our lives, to help us better understand the world around us and make better decisions, and
3. Updating these probabilities appropriately when relevant new information comes in.

Key elements of thinking probabilistically about the world

We have already examined the first element. Let’s take each of the last two in turn.

---

Assigning probabilities

When we recognize uncertainty in the world and try to embrace it, we recognize the need to estimate probabilities of events in both the public arena (for example, a certain candidate will win the election, a certain pharmaceutical firm will develop a successful vaccine, the U.S. government will retaliate against the Chinese government for imposing tariffs on American exports, etc.) and one’s personal life (for example, the next bus will be late, I will like my new job in San Francisco, etc.). Assessing these probabilities can help us make better decisions, as we will see below and in the next chapter.

The idea of assigning probabilities to events may strike you as impossible. Many of Richard’s students, friends, and colleagues have made this same claim of impossibility. But after thinking about it and practicing, many think this is the most valuable lesson he ever imparted. Richard is fond of saying that assessing subjective probabilities is no different than hitting a baseball or learning long division; one improves with practice. In fact, Richard encourages his collaborators to practice constantly in everyday situations, such as in assessing the likelihood that Raul will be here on time, or the chances that Lisa will say that she liked the movie.

Erin St. Peter, a former student of Richard’s and now a research analyst at Wharton’s Real Estate Center, recalls a time when Richard invited his team of teaching assistants to lunch to discuss their research ideas. “As our group approached the checkout line of the cafeteria, he passed his credit card to me and asked, ‘What do you think the probability is that the cashier will question the name on the card if you use it to pay for our group?’ Because the world is more uncertain than you think, I knew the answer should not be 0. And because I am a different gender than the cardholder I thought the answer might be higher for me than it would be for a man. I think I said less than 2%. We passed through the line without question. Examples like these remind me of all of the opportunities you have in the day to be curious and to think probabilistically.”

Calibrating subjective probabilities can be particularly challenging when predicting what individuals will do, as Erin’s cashier example illustrates. Joe Newhouse, a faculty colleague who has known Richard from his undergraduate days, notes this challenge in contract bridge where great players are constantly inferring probabilities to decide on the best course of action. “Bridge is a game of drawing inferences from both what the other players do and do not do. It is no surprise that Richard is a past national champion.”

Victor Paci, a lawyer and one of Richard’s partners at a real estate private equity firm, is a subjective probabilities enthusiast, driven by decades of experience. He argues that the underwriting process on real estate transactions requires thinking probabilistically. “Our typical real estate modeling includes a number of underlying assumptions about future events. We make these assumptions based in part on subjective probabilities. For example, we might be contemplating whether to invest in a development company. Once we have made the investment, new opportunities could become available, such as the opportunity to purchase equity interests in the developer’s projects. Because the chance of this new opportunity is entirely speculative at the time that we decide to make the original investment, it would be inappropriate to include it in our base case underwriting and pricing. However, we will nevertheless model it using subjective probabilities, and take our conclusions about its value into account when assessing the attractiveness of the base case investment.”

You might think that it is possible to estimate the probability of some kinds of events, such as who will win an election or whether a firm can execute on a component of a real estate transaction, but hopeless for some other kinds of events. Many national security officials feel that it is a hopeless task to try to quantify things such as the likelihood that Iran will have operable nuclear weapons within two years or that a particular leader will still be in power three years from now. Yet Jeff Friedman, a former student of Richard’s and now a faculty member at Dartmouth College, coauthored six papers with Richard on the role that probabilistic reasoning can play in national security decision-making. Their analyses endorse quantitative estimates, as opposed to verbal terms such as “likely” or “very unlikely” that are the backbone of the security community’s assessment. Friedman lectures every year at military institutions and intelligence agencies. “These experiences are always rewarding, yet I often come away impressed by the sheer magnitude of skepticism that many practitioners direct toward a style of thinking that is so demonstrably useful.” He credits Richard with helping him see not only the usefulness of this way of thinking but also how enjoyable it can be. “The key to learning how to think probabilistically is not just to understand that this endeavor is practically useful, but to realize that it can also be so enjoyable. That is one of the most important contributions that Richard has made through his research, teaching, and mentorship.”

Estimating probabilities can also have big payoffs for personal decisions. Jennifer Lerner is a Harvard professor in the Department of Psychology and the Kennedy School and a coauthor of Richard’s. She and her husband used probabilistic thinking to save money on their wedding.

Thinking probabilistically for personal decisions Jennifer Lerner If you go to Martha Stewart’s web page for wedding planning (something I am not advising you do), she asks rhetorically, “Wouldn’t it be great if wedding planning were an exact science?” Unfortunately, the rhetorical question is just a teaser. According to Ms. Stewart’s website, “When it comes to predicting how many guests will attend a wedding, it’s nearly impossible to accurately estimate what percentage of the overall guest list will affirmatively respond to your invite.” When my (now) husband Brian and I were planning our wedding back in June 1998, my mother held the same view. Unless an invited guest specifically had a reason for not attending, and told us so, my mom believed that every person on the list would attend and we had to plan accordingly. Enter conflict: My husband and I did not share this view. We wanted to plan probabilistically. We researched average attendance rates based on how far people had to travel and plugged those rates into our algorithm for attendance. My mom was horrified. It seemed to her like a cold-hearted exercise to anticipate that some people would not attend our wedding, even if they hadn’t specifically told us so. And in my Jewish and Italian family where feeding people is a cherished, sacred event, the idea of not having an excess of food, let alone enough dinners for every single person invited, triggered more than mild disagreement. Rather than argue it out, my husband and I (poor doctoral students at the time who needed to save every penny) decided to keep our probabilistic estimates to ourselves. Only the two of us and the caterer knew that we expected some invitees would not attend. I’m happy to report that it was a wonderful wedding. Some invited guests were not able to attend at the last minute and we had not overpaid.

Notice that estimating subjective probabilities does not mean that your intuition or gut need to be ignored. In fact, they can help you form a fundamentally richer view of the world that recognizes the uncertainty around you. William Samuelson, a professor at Boston University, coauthor, and long-time tennis partner of Richard’s, illustrates this point nicely: “Over the years in both personal and professional conversations, I’ve had people tell me how they made important decisions. Across this sample, there are four or five frequently quoted reasons. Frequently, they say it was based on “their gut” or alternatively based on “their experience.” To which I ask the classic Zeckhauser follow-up question, “And what does your gut/experience tell you about the chances that you’ll be happy with the choice you made?” That usually elicits a “what do you mean” look and sparks a conversation about assessing probabilities and making decisions.”

Thinking probabilistically often involves comparing probabilities of two different events to determine the best path forward. Richard often tells fun stories involving risk and probabilities, and the following one recounted by Maryaline Catillon, one of Richard’s former students and now coauthor, illustrates this idea well: “I was in a national park with my family and cars were not allowed, so we would go around in a bus, and stop from time to time to hike, picnic, and the like. Once we stopped with a group, and went for a picnic. We could see a family of bears approaching from a distance and went back to the bus stop. The bus finally arrived. Ten people were waiting. There were only six available seats and we were the last four. The conductor said, “I am sorry, there are not enough seat belts for all of you.” What do you think was the probability that we would die in a bus accident? And the probability we would be the victim of bears?” Maryaline adds, “Richard sometimes challenges colleagues who think like the bus driver, or the folks who make rules for the bus driver. When facing problems involving competing risks, I often wonder which is the bear and which is the bus.”

---

Updating probabilities

The third key element of probabilistic thinking is the willingness to update your probabilities, and doing so appropriately in response to new information. The process used to update our probability assessments with new information, sometimes referred to as Bayesian updating, is incredibly useful in many walks of life. See the figure below.

Imagine that tonight your favorite professional basketball team will play against their arch-rivals. Based on their respective records thus far this season and some sports websites you consulted, you initially assess that the probability that your team wins tonight is 40 percent. Decision theorists call this a “prior” probability or simply a “prior.” Two hours before the game you learn that the opposing team’s superstar got injured in a freak accident that day and won’t be able to play tonight. Since the opposing team wins much more often when he plays than when he does not, you update your initial probability to 55 percent. Decision theorists call this a “posterior” probability. It will be your prior probability if you update again in response to additional new information.

The game begins, and your team racks up a 6-point lead at the end of the first half. Maybe now you would update your subjective probability of a victory to 65 percent. Maybe the other team now scores several times and your subjective probability drops back to 55 percent. As the game progresses, your assessments change depending on what’s happening (or more generally, on information you gather). If there are only thirty seconds left, your victory assessment might soar to 80 percent, even if your team is only up by two points and the other team has the ball. If you think this is overly academic, you can check websites such as espn.com, which do exactly this – they report the odds of each team winning as the game evolves.

Bayesian Updating

Some pieces of information are more important than others. Consider again the 2016 U.S. presidential election, and picture yourself watching the coverage on election night. Let’s say you started the day believing there was a 29 percent chance of Trump winning the election (as the FiveThirtyEight forecast indicated). Results start coming in. You soon learn that Florida, well known as a swing state, has been declared for Trump. At that moment, you should update the probability that Trump will win the election to be far more than 29 percent. This is not only because Florida’s 29 electoral votes (out of 270 needed for victory) were now in Trump’s basket, but also because Trump’s winning Florida increases your assessment of the likelihood that Trump will win some similar battleground states. This single snippet of information might update your probability that Trump wins the election to 40 percent. In contrast, suppose you learn that Trump lost New York state. Since New York votes heavily for the Democrats, you never thought that Trump had a chance there. Trump’s loss there conveys no information about his success in other states. These two results (Florida and New York) might be announced with the same fanfare and a big “Breaking News” sign by CNN, but in terms of helping you update your assessment of the probability that Trump will win the election, the Florida result is highly informative; the New York one tells you close to nothing.

The early days of COVID-19 provide an instructive lesson of the benefits of carefully assessing new information in determining how strongly to update your priors. Sometimes you will feel that the new information is potentially inaccurate or that you lack the level of expertise needed to assess it effectively. However, if you must make a decision and further information will not be available, you must still assess how strongly to adjust your prior beliefs. This is nicely illustrated by Stefan Trautmann, coauthor of Richard’s and professor of Behavioral Finance at the University of Heidelberg in Germany.

Updating subjective probabilities in the early days of COVID-19 Stefan Trautmann In mid-January 2020, a Chinese colleague told me about some novel virus in the city of Wuhan, which I had visited about a year earlier. Shortly afterwards the virus became more prominent also in the German news as the Chinese government put more than 10 million people under quarantine, and shortly after many more. How to think about a virus outbreak in a distant country in some city few Germans had ever heard about? Some virologists in the media suggested that the new virus may not be particularly dangerous, and that the current flu season might kill more people. Not being a virologist, epidemiologist, or otherwise having medical expertise, how to think about the public-health and economic risks deriving from the virus outbreak? Richard’s way of thinking through events where there is little to guide our risk assessment, suggests putting ourselves in the shoes of those better informed, especially if they have clear incentives. No matter what you think about the risk tolerance of Chinese officials, their approach to public health, or Chinese politics more generally, any government that strictly enforces a full shutdown of an important industrial center and a curfew on millions of people must have relevant information that there are some very severe risks around. Some simple Zeckhauser rules of thinking would have helped many governments to be better prepared when their countries were eventually hit by the COVID-19 outbreak.

Stefan was recommending that nations should have drawn inferences from the significant decision taken by another entity, in this instance China. Doing so represents a more general principle about decision making: When updating probabilistic beliefs, it is important to use all available information that is relevant, including information from the decisions of others.

Sometimes, asking others to give you their assessment of a particular probability can be helpful in refining your own assessment, or to gain useful information about someone else’s views. Suppose you want to know how your friend or daughter is feeling about getting admitted into the college of her dreams. If you ask “Do you think you will get in?” you will receive a yes or no answer (or maybe “I don’t know”). But if instead you ask, “What do you think are the chances you will get admitted?” you may gain more information and will be able to have a richer conversation. This idea is nicely illustrated by coauthor Maciej Kotowski, a professor of economic theory at the University of Notre Dame and former Harvard Kennedy School colleague.

Gaining information by asking about probabilities Maciej Kotowski There are many interesting things in my office – colorful stools to sit on, a tub of Lego bricks, and a wall clock of peculiar proportions. However, most visitors’ eyes are drawn to three one-dollar bills clipped to the corner of my whiteboard. These bills do not belong to me. More precisely, they once belonged to me. Now, they’re Richard’s, who despite countless opportunities has not claimed them. The bills’ story is simple and reveals how useful it can be to use subjective probabilities to guide actions. Much of my research involves pursuing problems with uncertain answers. Occasionally, one is lucky enough to whittle down a problem into a precise conjecture or formal proposition that is either true or not true. But then, what next? Spending days or weeks trying to prove an incorrect conjecture is hardly a good use of time. Equally wasteful is hunting for a counterexample that might not exist. The natural course of action is for a researcher to pursue his “hunch” as to which path is correct. That is, based on the researcher’s understanding of the problem, one path is deemed more likely to lead to a quick resolution. Typically, subjective probabilities stem from intuition alone. A slightly improved method, perhaps, is to solicit an expert’s opinion. One could ask the expert, “Do you think this theorem is true?” and an answer of “yes” or “no” will follow. Slightly more information about the expert’s opinion can be collected via a small bet. Would the expert be willing to bet a dollar that the theorem is true? Maybe at 2 to 1 or 3 to 1 odds? Such bets are not made to earn money – the sums are trivial. Instead, they are a small price to pay for more information to further refine one’s own assessment of the problem, which will determine which path to follow. The three dollars represent my debt to Richard’s intuition. He was right on a few bets, and I proved a few theorems.

In the professional world, assessing the probabilities that different courses of action will lead to success, and being willing to update these probabilities, is essential to effective decision making. It can help you choose investments, select collaborators, determine which projects to pursue, etc. It can also help you decide which projects to abandon, as a story from Nils Wernerfelt, a research scientist at Facebook and coauthor of Richard’s, illustrates.

Assessing probabilities to decide which projects to keep Nils Wernerfelt Richard has a story from years ago when a young researcher came to him asking for feedback on several promising early-stage projects. Richard reviewed the material and based on his feedback, the researcher made the tough decision to kill some of the projects. I have this image of Richard’s forever ingrained in my mind of him raising his hand and saying: “And that’s when I knew Steve Levitt would be successful!” As that line suggests, the side of this story that Richard emphasizes is that it’s critical to know when to kill projects. Time and energy are scarce resources, and even if a project of any nature – not just economics papers – starts out strong, we should constantly be thinking probabilistically and updating our expected values. This is a hard but essential skill. Another side of the story that’s always stayed with me is how important it is to identify sources of high-quality signals (i.e., information about the likelihood that a project will succeed). For Steve Levitt to update his assessments and kill his projects, he had to know where to go to acquire those signals in the first place. This, too, is an essential skill. Richard has been that expert source for many of us across a range of important subjects in our lives (research topics, career choices, Thai restaurants in Cambridge, etc.). I implore the readers of this book to not only take his maxims to heart, but also identify who the Richard Zeckhausers are in their own existences. Speaking from personal experience, finding such a person will be one of the most important moments in their lives.

Most of the time, a decision maker must decide both what information to gather, often at some cost, and when to act rather than wait to gather more information. Consider a student who has been admitted to several colleges. She might decide to call three graduates from her high school now attending the colleges she is seriously considering, or she could visit each college in person. The importance of a decision should affect her expenditures on information gathering. She will spend four years of her life at her chosen college. A trip would provide much better information than three phone calls. Contrast this with her decision of where to do a summer internship. In this case, the decision might be less consequential and hence she might decide not to travel to each of the potential locations.

Thinking probabilistically can be hard. As Howard Kunreuther, emeritus professor and codirector of Wharton Risk Management and Decision Processes Center at the Wharton School, University of Pennsylvania, puts it: “I now believe that even if people understand probability theory, it is challenging to learn how to apply these concepts when making decisions, particularly when dealing with extreme events, such as natural disasters, few people think probabilistically when deciding whether to protect themselves against future losses.”

---

In sum

Ultimately, thinking probabilistically starts by correctly viewing the world as a highly uncertain place. With that view in mind, the logical prescription is to assign probabilities to the various events that could occur, and then update those probabilities as new information arrives. These updated probabilities are used to make better decisions. This approach is beautifully summarized in the box below by Chris Robert, one of Richard’s former students and a technology entrepreneur.

Thinking probabilistically Chris Robert I left Richard’s course with an essentially different view of the world, one that is fundamentally more accurate and powerful, and this idea of viewing the world probabilistically is at the heart of it. Just as Morpheus helped Neo in The Matrix, Richard helped me to see a deeper reality, one in which uncertainty and probability distributions are behind everything that happens. None of us ever fully control outcomes – even our own thoughts and behavior! – but we act to influence probability distributions as best we can. Philosophically, I may view the world as deterministic, but in practice I have to accept that I am far from omniscient and even farther from omnipotent. Accepting that everything is effectively uncertain and that my thoughts, hopes, and actions can at best indirectly influence the world by influencing probability distributions: this has brought me a kind of peace through acceptance, and a greater mastery over the world around me. What can I do to control an outcome? That’s the wrong question. What can I do to influence the odds? Now that’s productive. That I can work with – in my life and in my job.

MAXIM 7