Recognizing You Were Wrong Makes You Better at Being Right
Superforecasters improve their accuracy by analyzing their mistakes, unlike most people who rationalize errors. Learning from failures is a key to better forecasting.
THE IDEA
RECOGNIZING YOU WERE WRONG MAKES YOU BETTER AT BEING RIGHT
The way most people react when the world violates their expectations is to ask themselves, “Can I believe I’m still right?” Most of the time, the answer is “Yes, easily.”
Since beginning his study of forecasting in the 1980s, Tetlock has heard hundreds of justifications for failed forecasts, which he has catalogued into seven belief system defenses. One of them he calls “I Was Almost Right.” After George W. Bush won the 2000 U.S. presidential election, many people who had confidently predicted a win for his opponent Al Gore insisted that they would have been right if conditions had been slightly different: If Gore had been a better debater. If the election had been held a few days later. If the third-party candidate had been less stubborn.
The superforecasters had a very different relationship with their mistakes. When their predictions missed the mark by a lot—if they predicted something was very likely and it didn’t happen or if they predicted something was very unlikely and it did happen—they would go back and reevaluate their process, asking, “What does this teach me about how to make better forecasts?” Here’s an example:
The Yasukuni Shrine in Japan is a controversial spot. On the one hand, it holds many of Japan’s military heroes; on the other hand, it also holds over a thousand of Japan’s war criminals. When a political figure visits Yasukuni, it’s considered a diplomatic faux pas, a slap in the face to other countries like China and Korea that have suffered at the hands of the Japanese armies in the past.
One of IARPA’s forecasting questions in 2013 was “Will Japan’s prime minister, Shinzo Abe, visit Yasukuni this year?” There had been a rumor that Abe was planning a visit, but a superforecaster named Bill Flack didn’t buy it. It just didn’t make sense for Abe to shoot himself in the foot, diplomatically speaking, for no real gain. But the rumor turned out to be true. Flack asked himself why he got that one wrong and realized: “I think that the question I was really answering wasn’t ‘Will Abe visit Yasukuni?’ but ‘If I were PM of Japan, would I visit Yasukuni?’”
This is another reason superforecasters are much happier to think about what they got wrong—they know that analyzing their errors is an opportunity to hone their technique. Lessons like “Don’t assume world leaders would react the same way as you” are like power-ups, upgrades to your mental arsenal that make you smarter going forward.
The superforecasters had started the tournament with greater accuracy than everyone else, and as the months went on, their lead increased. Each year, the superforecasters’ average accuracy improved by roughly 25 percent. Meanwhile, the other forecasters didn’t improve at all.
When to use it
A project manager reviews past project deadlines to understand why estimates were off, leading to more accurate scheduling.
An investor analyzes their past stock trades to identify recurring biases (e.g., overconfidence, recency) and adjusts their strategy accordingly.
A sports analyst reviews their preseason predictions to find gaps in their understanding of team dynamics or player performance.
A policy advisor re-evaluates forecasts about political events (e.g., election outcomes) by focusing on process rather than justifying the miss.
FROM THEORY TO EVERYDAY LIFE
In practice
The 2000 U.S. Presidential Election
After George W. Bush won, some forecasters who had predicted Al Gore’s victory defended their prediction by imagining slightly different conditions, such as better debating or a later election. Galef uses this as an example of protecting a belief instead of asking how the forecasting process could improve.
Shinzo Abe’s Visit to Yasukuni Shrine
Superforecaster Bill Flack predicted that Prime Minister Abe would not visit the Yasukuni Shrine in 2013 because it seemed diplomatically costly. When Abe did visit, Flack realized he had been answering the wrong question: instead of analyzing Abe's likely behavior, he had unconsciously assumed Abe would act like he would. This insight helped him adjust his forecasting approach.
A MOMENT TO REFLECT
Put it to work.
Think of a recent prediction or expectation you had that turned out to be wrong. What rationalization(s) did you use to preserve your belief? How could analyzing that failure improve your future predictions?
When you forecast outcomes, are you truly considering the other party's perspective, or are you projecting your own reasoning onto them? Give an example where this distinction matters.
Review a past decision where you overestimated the likelihood of an outcome. What specific cognitive processes led to that error, and how can you adjust your method next time?
Often confused with
CONTINUE THE CONVERSATION
Source texts
- The Scout Mindset (Julia Galef)