The Scientific Method According to Feynman

The Scientific Method According to Feynman

Learning to “see” the world as a scientist sees it requires:

1. the willingness to question what you think you know, and
2. being able to look at what you really observe with a sense of childlike playfulness, joy, and deep curiosity.

Curiosity and openness are habits that support inquiry. A method adds discipline: state an idea, work out what it predicts, and compare that prediction with observations.

The scientific method structures inquiry through observation, data gathering, prediction, and testing. Its habits can also help with everyday problems.

Physicist Brian Cox once spoke to students at Manchester University about black holes, and showed them a video clip from a 1960s lecture by Richard Feynman. Cox introduced the clip by saying it was simply the best explanation he had yet encountered about the spirit of science and what the scientific method was all about. The clip easily laid it all out in under a minute.

According to Feynman,

“Now I’m going to discuss how we would look for a new law. In general, we look for a new law by the following process. First, we guess it.

Then we compute the consequences of the guess, to see . . . if this law we guess is right, what it would imply, and then we compare the computation results to nature, or we compare to experiment or experience, or compare it directly with observations to see if it works.

If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.

It doesn’t make any difference how beautiful your guess is, it doesn’t matter how smart you are who made the guess, or what his name is . . . If it disagrees with experiment, it’s wrong. That’s all there is to it.”

Feynman’s explanation gives a compact sequence:

Step 1: To look for a new law, first guess it. Step 2: Compute the consequences of the guess. Step 3: Compare the computation results to nature. Step 4: If the results disagree with nature, then our guess is wrong. Step 5: Revise the hypothesis or test, and repeat as needed.

The sequence compares a prediction with what happens. A mismatch gives us a reason to examine the hypothesis, the test, and the assumptions connecting them. Agreement supplies support, but other explanations may also fit the observations. Further tests help distinguish them.

In more formal scientific terms, the “guess” Feynman is talking about is a hypothesis. It can be based on what we already know or what we have observed, it can emerge because there are obvious gaps in what we know, or it can be driven by nothing more than curiosity. A hypothesis turns a question into a claim whose implications can be checked. That discipline is useful whether the question is original research or an everyday uncertainty.

Imagine that reality is a little like one of those Magic 8 Balls that can be shaken to reveal an answer to a question. Let’s say that this Magic 8 Ball only has three possible responses to any questions you ask: it can say YES, NO, or TRY AGAIN. The TRY AGAIN response could mean a few things—your question is phrased incorrectly, there’s been an error along the way, or you need to, well, try again. A claim may be supported, contradicted, or too imprecise to assess in its present form.

When you ask a question of this great universal Magic 8 Ball, then, you cannot ask it “Why do some people not die of this virus that is killing everyone?” or “What medicine will help people infected with this virus?” The ball will only tell you that any one guess is correct, incorrect, or undecided. You could ask “Do the people who survive this virus have higher serum bilirubin levels than those who don’t?”

In this hypothetical example, the “asking” takes the form of an observational comparison. You compare three groups—people who have died from the virus, people who have the virus but have not died from it, and people who have not had the virus. You then look at the blood work of all three groups and ask what their serum bilirubin levels are. Since you have thousands of people to measure and observe, you use statistics to help you crunch the numbers. Suppose you notice a pattern: the survivors have higher bilirubin levels than both other groups. Differences in measurement timing and other characteristics would also need examination.

This result is essentially the Magic 8 Ball saying, “Yes, this is a pretty good guess.” But that doesn’t mean that that’s the end of the story and we have conclusively proven something forever. It just means we have discovered some evidence and support for our guess. A useful empirical hypothesis needs implications that observations could challenge. For example, they make a claim (“bilirubin protects people from the virus”) and then set up an experiment so they can observe if this guess matches up with what really happens in the world. The group comparison alone cannot establish protection: illness could change bilirubin levels, or another factor could affect both. A stronger test would distinguish these explanations.

So, we seek to find out whether a hypothesis (guess) can be rejected or not. We improve an explanation by testing where its predictions fail and how well alternatives account for the evidence. We begin with speculation, and putting that speculation out into the real world, we see how it performs. Then, based on our results, we can make another experiment, and our next guesses and speculations can be a little smarter than the previous one.

Naturally, real experimental science has strict protocols to follow and its own logical rules that help formalize this process. Scientists have a whole range of activities that count as “observation,” and there is a difference between theoretical and applied sciences, between different kinds of observations (for example, in the social sciences versus in medicine), and between different approaches that use tools like mathematics, statistics, software, or technology to extend and expand the normal powers of human perception. But in the end, it all comes down to Feynman’s basic recipe: make a guess, observe what reality is actually doing, compare the two, rinse and repeat.

Another important thing to remember, and Feynman does approach this in his simplified explanation, is that a scientist’s reputation or preferences cannot settle whether a prediction matches the evidence. If we make a guess that turns out to not explain reality in the way we thought it might, this isn’t a problem. We haven’t made a mistake, and that doesn’t mean we’re bad or wrong or stupid. Making “wrong” guesses is Doing Science in the exact same way that making “right” guesses is. Either can be informative if the test is capable of distinguishing plausible explanations. In the same way, a guess is right because that’s what it is—not because it’s been made by someone who is good or intelligent or has been right before. A guess can be right and still be something we don’t like, and vice versa.

Scientific inquiry requires being willing to accept an unwelcome answer. They don’t ask it because they want to confirm that what they already believe is true. They are not looking for “proof” that will permit them to continue holding on to what they suspected was the case all along. Their first duty is to find something that is true and correct. If the process that leads them there entails dozens of wrong guesses, then so be it.

If human beings were doing science in this way, you would expect them to regularly publish findings in which their hypotheses were shown to be rejected by their observations and experiments. In reality, though, academic publishing contains an improbably high level of papers showing that the researchers’ guess just so happened to be correct. In 2005, John Ioannidis argued that many published research findings could be false under plausible combinations of low prior odds, limited statistical power, and bias. This was an argument about the reliability of research under specified conditions, not a count showing that most medical papers had failed replication.

Selective publication and flexible analysis can make positive findings appear more reliable than they are. Pressure to publish can reinforce those practices. Replication failures have several possible causes, so they should not all be attributed to researchers seeking confirmation.

Questionable research practices, “publish or perish” culture in universities, political pressure, and the rampant misuse of statistics are all impediments to what Feynman would have considered truly scientific—but that doesn’t mean the principle itself is not sound. On a smaller scale, we are all at risk as flawed human beings of succumbing to our blind spots, biases, prejudices, and pet theories. We can convince ourselves we’re encountering truth when we’re really just creating something we like and naming it truth because it’s convenient.

The same habit of testing an explanation can help with everyday troubleshooting.

Let’s say your car is acting up. You don’t know much about cars, but it’s been making a weird noise, and one day you see a light on the dashboard—one you don’t recognize, with some weird circles and a cross through them. You look in the car’s manual to see what the symbol means: It says “brake lights.” But what does that mean? Your hypothesis: the brake light is broken. How do you test this hypothesis? You could step outside of the car and look with your own two eyes. But then you see something curious—both brake lights appear to be working. Okay, time for a new hypothesis: The dashboard indicator is malfunctioning! Now, how are you going to test that? And how can you determine whether this phenomenon is connected with the noise you’re hearing?

The next day, you notice two things: The brake light is off, and the noise has stopped. Now, this does offer some support for the idea that the light and noise are somehow connected. But you haven’t proven it yet! If you’re not a mechanic and can’t get to one, you might simply conduct an “experiment” that consists in you noticing whether the two phenomena ever occur together. Every time they do, you gather more evidence for the idea that something is going on, and these two events have something to do with it. When one day the noise occurs without the light being on, you reconsider whether they always share a cause. A failure to coincide does not establish that they are independent . . .

Sometimes, the spirit of the scientific method simply reveals itself in our ability to stop ourselves from saying, “It can’t be done,” so we can instead actively ask, “Can it be done? How can it be done?” or even better, “What is being done right now? What do I observe and why is it happening?” The scientifically minded person has an attitude of experiment—they don’t make pronouncements and leave it at that. They frequently think, “I wonder what will happen if I . . .” and follow through. If something is unanswered, mysterious, or unknown, they don’t shrug their shoulders or choose the “answer” they like best and cling to it. They say to themselves, “Let’s find out!”

While such examples of trial and error and logical common sense seem almost too obvious, the useful habit is to compare an explanation with evidence that could challenge it. The same applies to things outside the five senses. In CBT (cognitive behavioral therapy), counselors often advise people to test their theories of reality, rather than just assume that their knee-jerk interpretation of events is always correct. If someone frowns at you on public transport, you could quickly conclude that you’re weird and unlovable and everyone knows it. You could also consider that the person was unhappy for a reason unrelated to you. A frown alone does not distinguish these explanations.

But you don’t necessarily have to go on faith. You can test your “unlovable” hypothesis the next time you go on public transport. Is what you observe in line with your theory? In other words, is everyone you encounter frowning at you or behaving as though they dislike you? If people are just minding their own business and being perfectly neutral about your presence, that challenges the prediction that everyone will respond negatively. A global judgment about your worth is not something a passing stranger’s expression can establish. Remember what Feynman said: “It doesn’t make any difference how beautiful your guess is, it doesn’t matter how smart you are who made the guess, or what his name is . . . If it disagrees with experiment, it’s wrong. That’s all there is to it.”

In our example, it doesn’t matter how right your hypothesis feels, or how much your guess seems like it fits your pre-existing worldview (that is, the one in which you are pretty sure you’re unlovable). It doesn’t matter how right you want to be—if observations conflict with the prediction, the explanation needs reconsideration.

Summary

• Feynman emphasized deriving predictions and comparing them with observation. These habits can help us examine familiar assumptions. • To see the world anew and without stale old misconceptions, try to look at it as though you were a Martian arriving on Earth for the very first time. What do you see? Which features might you notice if you questioned your usual interpretation? Prior knowledge remains useful, but it should be open to revision. • Knowing the arbitrary symbols assigned to a thing is not knowing it. Look beyond language. • Relaxation, daydreaming, creativity, and fun are not impediments to serious intellectual activity, but an important part of it. Your mind is naturally curious about the world. Curiosity and playfulness is a big part of how it survives and evolves. Work hard, let go, then work hard again. “Serious play” still requires domain knowledge and is focused and purposeful. Play may make an inquiry more engaging, without guaranteeing relief from burnout. • The scientific method is a way to structure our thinking and our approach to observation, gathering data, making predictions and theories, and inching our way closer to truth and understanding using reason and empiricism. • First make a guess about a new law. Then compute the consequences of the guess, then compare the computation results to nature. If the results disagree with nature, then your guess is wrong, if they agree, you have support for your hypothesis. What you want to be true is irrelevant; a scientist asks a question because they want to know the answer, not because they want to confirm what they already believe is true.