Everyday Analogue

WHEN TRYING TO UNDERSTAND A COMPLEX REAL-WORLD SITUATION, THINK OF AN EVERYDAY ANALOGUE

Negotiation between two countries might be similar in structure to simpler negotiations like those between a couple or between a parent and child, and the latter is easier to grasp at the outset. Richard Zeckhauser often uses an actual negotiation with a contractor fixing his home as an example to illustrate complex real-world negotiations like the Iran nuclear deal. As Alice Heath, a PhD student at the Harvard Kennedy School and a multi-year teaching fellow with Richard says: “Big policy problems are often similar in structure to “smaller” problems that we face every day. The main difference is the number of people impacted. But if we see the analogue to an everyday example, we will be able to really understand the problem and then apply our intuition to the bigger problem.”

The analogues we choose might be everyday situations, or ones that are memorable for other reasons. For example, imagine you are contemplating transporting 100 identical items, one by one, over the Niagara Falls using buckets. There are two types of buckets. The first type has been used 100 times and succeeded in 70 of them. The second type has been used 2 times and succeeded only once. What would you do? This is the kind of classic puzzle that Richard gives to his students and colleagues. Pause for a minute to think about it before you read the answer in the next paragraph.

Trying the second bucket a few more times may be worthwhile if what you learn can improve enough later choices to justify the cost of those trials. With the first one, you are reasonably sure that its success rate is close to 70 percent, whereas with the second one you are highly uncertain because it has only been used twice. Further trials can provide evidence that its success rate is below 70 percent, favoring a switch to the first bucket, or above 70 percent, favoring continued use. The decision also depends on prior expectations, the cost of failure, and how many choices remain.

Maryaline Catillon was on the receiving end of this puzzle. “This image of transporting buckets across the Niagara Falls always comes to mind when I am thinking about choosing between something I am used to (be it a project, idea, product, organization, restaurant, etc.) and an alternative that I have less experience with. The larger the uncertainty, with the same expected probability of success, the more I should try the new bucket. And the optimal length of experimentation depends on whether the old product or the new product does better. I should experiment longer even if the new product looks inferior, because I can always switch back to the old product. However, if I mistakenly conclude that the old product is superior, and discard the new product for good, I will never learn more about the new product. There is no switching back.” Maryaline’s professional work with pharmaceutical trials provides another setting for this trade-off between learning and using an established option.

Maryaline goes on, “The Niagara Falls analogy also has very important implications from the perspective of a project manager, a researcher, or anybody managing any sort of portfolio. When making decisions, such as whether to invest time and effort in a project, think of its option value. Projects with more uncertain outcomes are likely to be more interesting, and potentially high impact. Being the first person to explore an area (be it a research topic or a business idea), is potentially much more interesting, but also easier than entering widely explored areas. Higher-uncertainty projects can bring much higher gains.”[16]

Jonathan Borck has deployed this analogue approach effectively in his work as an economic consultant where he frequently needs to communicate with lawyers and judges who don’t speak his language, and in his second job teaching Kennedy School students. “In my work as a consultant and a statistics instructor, I find it valuable to illustrate complex statistical problems using the familiar example of political polling, which many people (at least in my social and professional circles) know well. For example, when trying to convey the concept of sampling error, say when sampling mortgages to evaluate the performance of mortgage-backed securities, I’ll analogize to the inherent uncertainties when polling for an upcoming presidential race.”

Analogues can be powerful when trying to communicate technical ideas to nontechnical people. For example, suppose you are interested in measuring the effect of a microfinance program on reducing poverty in a certain community. You could see if the poverty rate of the people who participated in the microfinance program changed after the program. If the poverty rate went down, you would conclude that the program was effective. If it didn’t go down, you would not. But this is not a good method because many other things could have caused the participants’ poverty rate to decrease. For example, maybe particularly good weather had allowed participants to grow more crops and earn more income. Maybe the general conditions of the economy got better.

So how can we measure the program’s effectiveness? Ideally, we would compare how people who participated in the microfinance program fared after the program with how these same individuals would have fared if they had not participated. The latter is referred to as the counterfactual scenario (what would have happened in the absence of the program), and of course we never get to observe it. In practice, researchers mimic the counterfactual by comparing the group of individuals who participated in the microfinance program with another group of individuals who are otherwise identical except that they didn’t receive the program. This latter group is often referred to as the control group.

So a key step in evaluating the impact of a microfinance program or many other interventions is to be able to form a control group that is truly comparable to the group that received the intervention. A simple analogue is to say that you want to compare apples to apples. I have used this analogue hundreds of times in training policy makers about evaluation of social programs by showing them a slide with two identical groups of apples, and it is incredible how such a simple image helps cement this abstract idea. They refer to this analogue frequently throughout the training when discussing whether a given evaluation method is likely to yield a credible result.

Coming back to our microfinance example, it would not be appropriate to estimate the impact of the program by comparing a group of people who applied and received the program with a group of people who did not even apply to the program, because the two groups are different; it appears that people self-selected into the microfinance group. We would not be able to know whether any difference in their outcomes was due to the program or to the fact that the two groups were different to begin with. This would be like comparing apples and oranges.[17] One way to get around this comparability problem is to conduct a randomized controlled trial (RCT), where we randomly assign some individuals to receive the intervention and others not. Random assignment makes treatment independent of participants’ initial characteristics in expectation. With appropriate design and follow-through, this supports a credible estimate of the program’s impact.[18]

Gary Orren, a colleague and friend of Richard’s for over half a century and a faculty member at the Harvard Kennedy School, provides another example of how analogues make ideas memorable.

AmeriCorps and the Swiss Army knife Gary Orren AmeriCorps is a government network that unites hundreds of nonprofit organizations throughout the United States (like City Year, Teach for America, and Habitat for Humanity) under a single umbrella of national service. Supported by public sector/private sector partnerships, AmeriCorps is funded by the federal government, corporations, foundations, and individual donors. The corps members, typically between the ages of seventeen and twenty-four, commit a year or two of full-time service to meet critical community needs such as increasing academic achievement, mentoring youth, fighting poverty, and sustaining national parks while also fostering civic engagement and developing their own leadership skills. Along with many others, I helped build the AmeriCorps program in the early 1990s. Over the years, I have remained a big fan and supporter of this program. Unfortunately, not everyone shares my affection for AmeriCorps. Some people—including some who hold powerful positions—not only oppose the program but have sought to cut its funding and even eliminate it. I have defended and promoted AmeriCorps on Capitol Hill, in corporate boardrooms, and with foundation officials. In this battle, I have been armed with statistical data, narrative examples, and historical information. To strengthen my argument, I decided to add a new arrow to my quiver: an everyday analogue to AmeriCorps. I thought that an analogy would help me better understand and convey the complex arguments for and against AmeriCorps. It would simplify AmeriCorps to its essential elements. It would clarify the pros and cons about AmeriCorps that my audiences didn’t yet comprehend by comparing it to something familiar that they did understand. I believed that an analogy would paint a picture that would make my message stickier and more lasting. I wanted an everyday analogue that would mirror the key features of AmeriCorps. AmeriCorps is a program where young people do many valuable things for their communities pretty well at a good price. What is like that? I considered several possibilities and suggestions. I settled on an analogue I saw in a published article that caught my eye: “AmeriCorps is America’s Swiss Army knife.” Like the AmeriCorps program, the Swiss Army knife does many valuable things pretty well (it’s not the best screwdriver, best scissors, or best knife) for a good price. When I spoke with legislative, corporate, and foundation leaders, I included this analogy in my pitch. When I returned to their offices weeks later, I reintroduced myself and said: “I spoke with you several weeks ago about the AmeriCorps program.” Many of my listeners replied with smiles of satisfaction, “Oh yeah, I remember you. Swiss Army knife.” In the office next door, I heard, “Swiss Army knife.” Up and down the hall I heard, “Swiss Army knife.” Notice that built into this everyday analogue is another maxim for good analytical thinking: an affordable concession. We should never become so enamored with the strong points in our thinking that we overlook the weakest links in the chain. Furthermore, acknowledging the weakest links is often persuasive when we communicate with others. Concessions convey honesty and objectivity and make what we say more credible. Although I am a strong proponent of AmeriCorps, I recognize and acknowledge that AmeriCorps has both pros and cons. In truth, many of the young corps members do only a pretty good job in their daily service, just as the Swiss Army knife does many valuable things only pretty well. Frankly, it’s not the best knife, screwdriver, or scissors. Similarly, the young AmeriCorps members are only seventeen, eighteen, nineteen years old and they are not professional school teachers trained to teach literacy, nor are they professional construction workers trained to rehab community centers. I concluded that my concession was affordable for two main reasons. Experts in the field of national service and people familiar with AmeriCorps were already well aware that many corps members only did a pretty good job. This awareness was not going to disappear from the discussion if I chose not to mention it. Furthermore, I didn’t only acknowledge that the corps members only did a pretty good job. I also emphasized that they did some things extraordinarily well. They served as unique role models for young school children; they were passionately committed to repairing the world; and they brought tireless energy to the enterprise. The analogy of the Swiss Army knife helped me better understand the strengths and weakness of AmeriCorps and enabled me to convey that understanding more effectively to others.