Outcome Bias
Outcome bias is judging a decision by how it turned out while giving too little weight to the information and reasoning available at the time. It can reward reckless choices that happen to succeed and discredit sound choices that happen to fail.
THE IDEA
Outcome bias is judging a decision by its result while giving too little weight to the reasoning and information available when it was made. The same sound process can sometimes fail, and a reckless gamble can sometimes succeed. Knowing the result makes it harder to assess the original choice on its merits.
Winning money in Vegas, for example, does not establish that a bet was worth taking. If a gambler treats a lucky win as evidence of a good strategy, the result gets in the way of learning.
Taleb’s story of the investors Nero and John shows the same temptation. John loses his wealth during a market downturn, while Nero, who stuck to his strategy, feels vindicated. Nero takes John’s failure and his own survival as proof that his methods were correct. Yet the contrast alone cannot establish how much either outcome owed to skill or luck.
In a bridge-monitor study, participants assessed a city’s decision about paying for a full-time monitor to prevent debris from blocking water flow. They judged the decision differently depending on whether they were told that a damaging flood later occurred. The information available to the city when it made the choice was the same; the evaluators’ knowledge of the result changed.
The Star Trek examples contrast a risky plan that fails with one that succeeds. In the failed attempt, Spock had said the chance of success was small. Failure alone does not show that his estimate was wrong or that trying was unreasonable. In the successful attempt, a happy ending does not show that the crew chose well. To assess either choice, we would need to consider its alternatives, risks, and stakes.
When reviewing a decision, first ask what was known at the time, what alternatives were available, and how their possible consequences were weighed. Then use the result as evidence for learning. A single outcome may reveal a missed risk, but it may also reflect chance. Looking across repeated decisions can help distinguish the two.
When to use it
Evaluating investment decisions after market results
Assessing medical treatments where patient outcomes vary
Reviewing strategic business moves in post-mortem meetings
Reviewing a diagnosis that proved wrong: assess the symptoms and evidence available to the doctor, the alternatives considered, and whether the reasoning was justified at the time.
A business leader approves a high-risk investment that fails. Investors may criticize the decision without considering that the expected return justified the risk.
Deciding whether to change a study method after a disappointing exam: consider how consistently it was used, the material covered, and results across more than one test.
A sports coach calls a risky play that doesn't score. Fans and analysts may blame the coach, ignoring that the play had a reasonable probability of success.
When evaluating past decisions in business or personal life, focus on the decision process rather than the result.
In medical diagnosis, avoid judging a doctor's decision solely by patient outcome if the decision was sound.
In investing, evaluate the reasoning behind a trade rather than whether it made money.
Evaluating investment performance: avoid praising only successful trades; review the reasoning behind all trades.
Performance reviews at work: assess employees' decision-making processes, not just project outcomes.
Sports coaching: analyze game decisions based on strategy, not just final scores.
Personal reflection: when making choices, focus on the process to reduce regret from random outcomes.
Evaluating investment performance: A risky investment that paid off might be seen as brilliant, but could have been poor at the time.
Hiring decisions: A new hire who turned out to be a star might lead you to overlook flaws in the hiring process.
Assessing a malpractice claim: distinguish a bad patient outcome from the question of whether the treatment decision was appropriate given what was known.
Sports coaching: A coach's risky play call that fails is criticized, but if it succeeds, it's celebrated, regardless of the strategic soundness.
FROM THEORY TO EVERYDAY LIFE
In practice
Star Trek Scenario
In Star Trek II: The Wrath of Khan, Spock tells Kirk, 'By coming after me, you may well have destroyed what slim chance you had for survival.' Despite the rational warning, the crew acts and everyone survives. Someone evaluating the decision solely by the happy ending might consider it a good choice, ignoring the high risk.
Star Trek: The Risky Plan
Kirk proposes reversing circuits on McCoy’s neuroanalyzer to counter a paralysis projector. Spock says the chance of success is small. They try it and fail. The failure alone does not show that the estimate was wrong or the attempt unreasonable; that judgment depends on the alternatives and stakes.
Surgeon decision scenario
In a study by Baron & Hershey (1988), participants were given a hypothetical scenario about a surgeon deciding to carry out a risky operation. The surgeon was shown to be well aware of the success rate. Participants were then told either the patient had died or lived. The surgeon’s decision to operate was considered a mistake when the patient had died and the right choice when the patient had lived.
Nero and John
After a market downturn, John (a rival) loses everything, while Nero, who stuck to his trading strategy, feels vindicated. Nero interprets the outcome as validation of his approach, ignoring that both successes and failures may be due to randomness.
Investment Performance
An investor who picks a stock that rises sharply is praised, while one who picks a stock that falls is criticized. Both choices may have been equally reasonable given the information at the time, but the outcome biases the evaluation.
Gambling Win
A casino bet with negative expected value can still pay off. Winning does not make the original odds more favorable, though it may tempt the gambler to repeat the bet.
Bridge Monitor Decision
In a study, students evaluated a city's decision to pay for a full-time bridge monitor. When told that debris caused a flood that the monitor could have prevented, they rated the decision to not hire the monitor as poor. But when told no flood occurred, they rated the same decision as acceptable. The decision process was identical; only the outcome changed.
A MOMENT TO REFLECT
Put it to work.
Think of a time you judged a decision as good simply because it turned out well. Could the same decision have been made with poor reasoning?
How can you separate the quality of a decision from its outcome when evaluating past choices?
Think of a recent decision you made that led to a bad outcome. How might you evaluate it differently if you ignore the outcome and focus only on the information available at the time?
When judging someone else's decision that failed, what questions can you ask to avoid outcome bias?
Recall a decision that had a good outcome but was actually reckless. How could outcome bias lead to overestimating the skill of the decision-maker?
What information would you record when making a decision so you could later assess your reasoning fairly?
Have you ever abandoned a sound approach after one failure? What would a longer record of results show?
What alternatives were available at the time, and how did their risks compare?
Would you judge the same reasoning differently if the outcome had gone the other way?
In your own work or investing, how often do you reward outcomes versus the quality of the decision-making process?
How could your team review the original reasoning before learning the result?
When does an unexpected result reveal a flaw in your assumptions, and when might it reflect chance?
When evaluating a leader's past decision, how can you separate the quality of the decision from its outcome?
In a business context, how might outcome bias lead to rewarding reckless behavior that happened to succeed?
Often confused with
References
- Outcome bias in decision evaluation — Baron, J., & Hershey, J. C. (1988)
CONTINUE THE CONVERSATION
Source texts
- 60 Cognitive Biases That Ruin Everything We Do by David Scott
- Fooled by Randomness The Hidden Role of Chance in the Markets and in Life (Nassim Nicholas Taleb)
- The 25 Cognitive Biases Uncovering The Myth Of Rational Thinking (Charles Holm)
- The Scout Mindset (Julia Galef)
- The Scout Mindset (Julia Galef)