Skip to main content
Decision Frameworks

Hypothesis Testing

Test competing explanations by asking what evidence would count against each one, especially the explanation you favor. This helps reveal weaknesses that a search for confirming evidence can miss. Jones uses Disney’s abandoned Virginia theme park to illustrate the cost of overlooking opposition.

THE IDEA

The Case of Disney’s America

Do you believe that UFOs are spacecraft from another solar system? A believer may collect sightings and testimonials while a skeptic focuses on natural explanations and hoaxes. Either can settle on a favored answer and look mainly for support. Jones calls this the “get-off-at-the-first-stop” approach to satisficing. The difficulty is that the same evidence may fit several explanations. My cat’s wet fur could mean she came in from the rain, fell into a sink, or was sprayed by the children. To distinguish those possibilities, I need evidence that could count against each one.

Jones describes a criminal investigation in which police arrested a young man for rape because he lived nearby, the victim identified him, and he had no alibi. They stopped investigating. Months later, a look-alike neighbor confessed. The police had “satisfied” on the first plausible suspect. Continuing to test the case against him might have exposed their mistake earlier.

Jones applies the same reasoning to the proposed “Disney’s America” theme park in Prince William County, Virginia. Disney announced it in late 1993 and canceled it ten months later after intense opposition. CEO Michael Eisner said, “I expected to be taken around on people’s shoulders.” Yet the surrounding estates and stables—the region held 15,000 horses—suggested an affluent population with strong attachments to the land. Planners focused on financial projections, jobs, and tourism while misunderstanding or downplaying the strength of opposition. They had “satisfied” on the prospect of approval.

The hypothesis testing exercise is to examine both “the plan will be approved” and “the plan will not be approved.” Alexander’s question asks, “What evidence would prove this false?” Applied early, that question could have drawn attention to local landowners and to the nearby Manassas Civil War battlefields, which gave the National Trust for Historic Preservation (NTHP) a reason to oppose the project.

Jones proposes that Disney might have involved historians and preservationists before choosing a site, giving them a role in shaping how history would be presented. That could have exposed objections earlier and perhaps changed the plan or its reception. It does not establish that the NTHP would have supported the park or that approval would have followed.

Before committing to a plan, ask, “What would show my favored hypothesis is wrong?” Then seek evidence that helps distinguish it from plausible alternatives. The purpose is to discover weaknesses while there is still time to respond.

When to use it

  • Evaluating a new product launch by formulating two hypotheses: 'The product will succeed' and 'The product will fail', then gathering evidence for both, including competitor reactions, market trends, and potential regulatory issues.

  • Assessing a potential investment by looking for indicators that the investment could underperform or be a bad fit, rather than only confirming its potential upside.

  • Planning a major organizational change by considering both successful acceptance and strong resistance, identifying key stakeholders, and developing strategies to address their concerns in advance.

  • Deciding on a career move by testing assumptions about job satisfaction, growth opportunities, and work-life balance, while also considering reasons why the move might not be beneficial.

  • Scientific research: Formulating and testing multiple hypotheses to determine the best explanation for observed phenomena.

  • Criminal investigations: Test plausible alternative suspects and scenarios before treating the initial case as settled.

  • Business strategy: Evaluating different market hypotheses before committing to a course of action.

  • Personal decision-making: Considering various reasons for a problem (e.g., car won't start) before choosing a fix.

FROM THEORY TO EVERYDAY LIFE

In practice

01

Disney's America Theme Park Failure

Walt Disney Company planned to build a historical theme park in Virginia but faced strong opposition from residents and the National Trust for Historic Preservation, leading to cancellation. The planners focused on evidence supporting the project's success and ignored signs of opposition. Hypothesis testing would have involved systematically considering evidence for and against project approval, potentially revealing the strength of opposition and allowing preemptive action.

02

UFO Beliefs

People who believe UFOs are extraterrestrial often seek out supporting evidence and ignore contradictory evidence, while skeptics may dismiss all reports without considering plausible alternatives.

03

Criminal Investigation

Police arrested a man for rape based on initial evidence (identification, proximity, no alibi) and stopped investigating. Later, the real rapist confessed, showing the danger of satisficing.

✳

A MOMENT TO REFLECT

Put it to work.

  1. Think of a recent important decision you made. What evidence did you ignore or downplay that contradicted your preferred outcome? How might hypothesis testing have changed your process?

  2. When planning a project, how can you deliberately seek out evidence that your plan might fail? What specific questions can you ask to uncover potential opposition or obstacles?

  3. Consider the Disney's America case. What other steps could the planners have taken, besides hypothesis testing, to better understand the risks? How does hypothesis testing complement other decision-making techniques?

  4. Can you recall a decision where you settled on a hypothesis too quickly and later found you missed a better explanation?

  5. When evaluating evidence, how can you ensure you are considering all plausible alternatives rather than just confirming your initial belief?

  6. What steps can you take to systematically test multiple hypotheses in a business or personal decision?

Connected ideas

Another lens can bring the same problem into focus.

Often confused with

CONTINUE THE CONVERSATION

Source texts

  • The Thinkers Toolkit by Morgan D Jones
  • The Thinkers Toolkit by Morgan D Jones
← More decision frameworks