Cognitive Bias in Problem Solving

Problematic Proclivities

Morgan D. Jones examines how emotion, mental shortcuts, pattern recognition, and hidden assumptions can affect problem solving. These habits often help us act quickly, but they can also lead us to accept an explanation before checking it.

1. There is an emotional dimension to almost every thought we have and every decision we make.

Perhaps the mental trait with the greatest influence over our thinking is emotion. We are inarguably emotional creatures. Sometimes, as Daniel Goleman says in his thought-provoking book, Emotional Intelligence: Why It Can Matter More Than IQ, emotion is so strong that it hijacks—overwhelms—our power to reason. This is not news, of course. Alexander Pope wrote three hundred years ago that “What reason weaves, by passion is undone.” Still, the role that emotions, even subtle ones, play in our decision making is greatly underrated, if not dismissed, but it shouldn’t be. As Nancy Gibbs, writing in Time magazine in 1995 about Goleman’s book, said, “Whether we experience a somatic response—a gut feeling of dread or a giddy sense of elation —emotions are helping to limit the field in any choice we have to make.”

Check your emotional state before making an important decision. When strong feelings are narrowing your attention, a delay may help. If the decision cannot wait, even a simple structure—writing down the options, evidence, and uncertainties—can make the reasoning easier to examine.

2. Mental shortcuts our unconscious minds continuously take influence our conscious thinking.

We humans tend to believe that, by consciously focusing our mind on a problem, we are controlling our mental faculties and are fully aware of what’s going on in our gray matter. Unfortunately, that simply isn’t true. Most of what goes on in the mind involves “mental shortcuts” that occur without our knowledge and beyond our conscious control. These subroutines—which are comparable to computer “subroutines”—whir away constantly every second we are awake and, for all anyone knows, while we’re asleep. We normally aren’t aware they exist until we think about the lengthy, elaborate sequence of mental steps involved in an unconscious act such as catching a lightbulb someone playfully tosses at us. “Here, catch!” the tosser says, and without thinking we spontaneously dive to catch the bulb. This is a practiced, largely automatic action. Jones uses it to illustrate a mental shortcut—a subroutine.

Imagine the lengthy, intricate sequence of perception, association, recognition, coordination, and reaction the mind must go through to activate the muscles that move the arm, hand, and fingers to a position that will precisely intersect the lightbulb at a predicted point in its downward trajectory to catch the bulb and close the fingers gently around it. In a fraction of a second, the mind issues all of these complex motor commands and, what is more important, in the correct sequence, without our precognition or conscious participation. The mind accomplishes this remarkable feat by recalling from memory the commands, made familiar by lifelong practice, for catching an object with our favored hand. Obviously, the brain does not make a decision on each individual step of the entire sequence before sending the commands to the muscles. If it did, the bulb would smash on the floor long before the sequence was completed. Instead, the mind retrieves from memory the entire ready-made sequence and executes it.

Automatic associations also influence how we analyze a problem. The intricately interwoven neural networks of our brain react in concert to everything the brain senses: every sight, every sound, every bit of information. As Lawrence Shainberg wrote in Memories of Amnesia, “The name rang a bell. And once the bell was rung, others rang as well. That’s the miracle of the brain. No bell rings alone.” Each perception triggers hidden reactions whose silent tolling secretly influences our thoughts. For example, we hear someone say he or she is on a diet. We instantly stereotype that person according to our views concerning dieting. This stereotyping is an unconscious reaction of the mind. We do not consciously decide to stereotype; the mind does it for us automatically without our asking. It is a connection—a mental shortcut—that the mind instantly makes through its neural networks and that influences our perception of that person.

Mental shortcuts express themselves in a variety of forms with which we are all familiar: personal bias, prejudice, jumping to a conclusion, a hunch, an intuition. Two examples:

Your spouse—or a great and good friend who will accompany you—asks where the two of you should go on your next vacation. You think about it for a moment (the subroutines are whirring) and decide you’d like to go hiking in Vermont. Your spouse or friend suggests two alternatives, but they don’t appeal to you (whir, whir) as much as your idea does. And the more you think about your idea (whir, whir), the more appeal it has and the more reasons (whir, whir) you can think of why it’s a good idea. And in two minutes you’re in a heated argument.

You are briefed by a subordinate on a problem in your office. You immediately recognize (whir, whir) what the problem is, the source of the problem (whir, whir), and how to resolve it (whir, whir). You confidently instruct the subordinate what corrective action to take. Ten days later the corrective action has failed to be corrective, the problem has worsened, your subordinate mistrusts your judgment, and you look bad.

We often notice the effects of a shortcut without being aware of how it arose. For example, if someone says “gun control,” you may immediately recall images, thoughts, and feelings. Those associations can be useful, but they can also steer judgment before we have examined the question.

These shortcuts are part and parcel of what we call “intuition,” which is really a euphemism for the unconscious mind. That we humans are prone to place great store in intuition is understandable inasmuch as intuition was the mechanism that guided our evolution into the planet’s dominant—and smartest—species. Without our constant intuitive insights we humans probably would have become extinct eons ago. Albert Einstein said, “The really valuable thing is intuition.” I found this quote in The Intuitive Edge: Understanding and Developing Intuition by Philip Goldberg. Goldberg explains that intuition can be understood as the mind turning in on itself and apprehending the result of processes that have taken place outside awareness. But he notes that intuition cannot be ordered, commanded, implored, or contrived. We simply have to be ready for it.

What if I ask, “Who was the twenty-second president of the United States?” Your eyes close as you press a finger to your lips. “Let me think a minute,” you say. Think a minute? You’re not thinking, you’re waiting! Like you do when put on hold by a telephone operator, except in this case you’re waiting for your unconscious to come on the line, that is, to make its shortcuts deep down in your memory where Grover Cleveland’s name, face, and what-all are stored, probably, as cognitive and neural scientists speculate, in bits and pieces throughout your brain. If your unconscious fails to make those connections, you won’t remember who the twenty-second president was. It’s as simple and as complicated as that.

Mental shortcuts can help experts recognize familiar situations. Imagine giving a set of medical symptoms to an intern fresh out of medical school and ask him to diagnose the problem and recommend a treatment. Then give the same symptoms to a doctor with twenty years experience in that field. It takes the intern fifteen minutes to come up with a diagnosis and treatment; it takes the experienced doctor fifteen seconds. The doctor’s mind makes mental shortcuts based on accumulated knowledge from long experience. The example illustrates how experience can speed recognition; the times are illustrative, and speed alone does not establish accuracy.

We cannot “teach” the mind to stop every automatic response. We can, however, examine a response after it appears: what assumption produced it, what evidence supports it, and what alternative might fit?

3. We are driven to view the world around us in terms of patterns.

The human mind instinctively views the world in terms of patterns, which it recognizes based on memories of past experiences. We see a face (a pattern). Our mind searches its memory (the bells are ringing), finds a matching face (pattern), and delivers to our consciousness the name and other information stored with that pattern. This is not a conscious process. We don’t decide to remember the name associated with the face. The name just pops into our head. The unconscious mind, unbidden, does it for us.

We see patterns in situations. Let’s say the lights go out. We are not flustered or frightened. We light a candle or turn on a flashlight and wait for the electric company to restore power. How do we know power will be restored? Because such outages have occurred before; we know the pattern. Or let’s say we are watching a football game and see a referee give a hand signal. We know what the signal means; we’ve seen it before.

We see patterns in sequences of events. We know when to put mail into the mailbox because the postman arrives at the same time each day. We encounter a traffic jam, watch a police car and ambulance pass ahead of us with sirens wailing, and deduce that a traffic accident is blocking the road. It’s a familiar pattern. When we arrive home from work, our spouse does not greet us with a kiss and a smile; something has happened to upset the pattern. Because our brain is so thoroughly optimized as a pattern-recognition mechanism, we need only a fragment of a pattern to retrieve the whole from memory. And that retrieval is unconscious.

While this impressive capability is beneficial to our ability to function effectively, our compulsion to see patterns can easily mislead us when we analyze problems.

Patterns can capture attention. Consider the sequence 40-50-60-__. The next number may seem to be 70. That is a reasonable prediction under a simple increasing rule, but the displayed terms do not determine a unique continuation. The problem arises if we treat 70 as certain without considering the rule or context.

We can also perceive a pattern in disconnected observations. The illustration cited as Figure 1-2 uses six dots to suggest shapes such as a hexagon or circle. Perceiving a shape does not establish that the dots were placed to form it. The same distinction matters when a pattern in data suggests an explanation.

A striking example of mistaken patterning was the initial reaction to the attack on Nancy Kerrigan prior to the U.S. National Figure Skating Championships in January 1994. Several newspaper columnists rashly cited the incident as further evidence of the rising risk to professional athletes of wanton violence by deranged fans. In short order, however, it became clear that the attacker was neither deranged nor a fan but a hit man hired by the ex-husband of Kerrigan’s chief competitor. The columnists had jumped to the wrong conclusion; they had mistaken the pattern.

But, what is worse, when we want to see a particular pattern or expect to see it, or have become accustomed to seeing it, not only do we fill in missing information but our mind edits out features that don’t fit the desired or familiar pattern.

We hear that a trusted colleague with whom we have worked for years is accused of sexually molesting children. We can’t believe it. We won’t believe it. It must be a mistake.

Historian Walter Lord, in Day of Infamy, a historical account of the Japanese attack on Pearl Harbor, cites a perfect example:

People all over Oahu couldn’t believe that what they were seeing was an enemy attack: unidentified planes, falling bombs, rising smoke. It was just the Army fooling around, it was just this, it was just that. Anything but the unbelievable truth.

The ease with which the human mind can misidentify a pattern is one of the things that makes us laugh at jokes. Take this story:

A man, walking down the street, passes a house and notices a child trying to reach a doorbell. No matter how much the little guy stretches, he can’t reach it. The man calls out, “Let me get that for you,” and he bounds onto the porch and rings the bell. “Thanks, mister,” says the kid. “Now let’s run.”

What makes the anecdote funny is that our mind has quickly conjured an image—a pattern—of a nice man helping a sweet little child, when suddenly we discover we are fooled. It’s a prank. We misidentified the pattern.

Stereotyping is a form of patterning: perceiving a similarity between two events or things because of superficial features and then, based on that perceived similarity, unconsciously ascribing to one of the events additional attributes of the other. We live by stereotyping; it underlies a great deal of our everyday behavior; it is a principal mechanism of the mind.

Stereotyping, of course, governs racial, ethnic, and every other type of bigotry. Let’s say that I don’t like a particular law firm in my city. I meet someone and ask where he or she works. The person says he or she works for that firm. Instantly, in a thousandth of a second, before I can reason my thinking, my prejudice against the law firm dominates my reasoning and I get a negative impression of this person. The insidious part is that I have to struggle against this impression to see this person as he or she really is; I have to struggle against my own mind. As I said, the mind is not always our best friend.

ABC-TV’s Prime Time Live conducted a fascinating experiment that demonstrated the hidden power of stereotyping. The results, presented on the program on June 9, 1994, revealed blatant age discrimination in hiring by a number of different companies. In the experiment the interviewee (an actor) played the role of two men: one in his twenties, the other disguised to appear in his fifties. In a number of interviews, despite the fact that the older man had more relevant job experience, the younger man received the job offers; the older man received none. Even when two of the job interviewers were told of the double role playing, they insisted their decision to hire the younger man had not been influenced by age bias. However, the hidden video-audio tapes of the interviews showed, without any question, that the interviewers were instantly put off by the older man’s age.

Jones uses the ABC segment to illustrate how quickly stereotyping can affect judgment and how little insight people may have into its influence. The interviewers’ denials do not establish that their choices were unbiased. The example supports checking decisions against observable evidence rather than relying on confidence in our own fairness.

Another kind of patterning is the tendency of the human mind to look for cause-and-effect relationships. Regarding this tendency, Morton Hunt said in The Universe Within that attributing causality to an event which is regularly followed by another is so much a part of our everyday thinking that we take it for granted and see nothing extraordinary about it. We seem to view the world in terms of cause and effect, and we somehow instinctively know the difference between the two concepts. This has been demonstrated unequivocally in experiments, even with infants. So we by nature strive to know why something has happened, is happening, or will happen and what the result was, is, or will be. Paleontologist Stephen Jay Gould said of patterning: “Humans are pattern-seeking animals. We must find cause and meaning in all events… everything must fit, must have a purpose and, in the strongest version, must be for the best.”

This compulsion, while obviously helpful in dealing with the world around us, can deceive us when we analyze problems, because we often perceive cause and effect when no such relationship exists.

A man awoke one morning to find a puddle of water in the middle of his king-size water bed. To fix the puncture, he rolled the mattress outdoors and filled it with more water so he could locate the leak more easily. But the enormous mattress, bloated with water and impossible to control on his steeply inclined lawn, rolled downhill, smashing into a clump of thorny bushes that poked holes in the mattress’s rubbery fabric. Disgusted, he disposed of the mattress and its frame and moved a standard bed into his room. The next morning he awoke to find a puddle of water in the middle of the new bed. The upstairs bathroom had a leaky drain. Mistaken cause and effect. It happens all the time.

It is the false perception of a cause-and-effect relationship that enables magicians to trick us—or more correctly, to trick our minds—so easily with illusions. I am ever intrigued by the reaction of audiences to magicians’ performances. The spectators are amused and surprised, apparently little realizing that the illusions they see on the stage are no different than those that occur daily in their lives but pass undetected. Each of us would be amazed at the number of times each day we perceive patterns of all types that are not what they appear to be. It is false perceptions, of course, that enable con artists to fleece people out of huge sums of money on pretexts that, in the clear light of retrospect, are flatly implausible and make the victims wonder how they were so easily fooled.

Another example. Retail sales drop during a period of heavy snowstorms. A sales manager has observed this pattern before and figures sales will rebound when the weather clears. Perhaps they will, but he should consider alternative interpretations in case he has misread the cause-and-effect relationship. But will he consider alternatives? Probably not. His unconscious mind is captivated by the presumed connection (the cause-and-effect pattern) between the weather and the loss of sales.

4. We instinctively rely on, and are susceptible to, biases and assumptions.

Bias—an unconscious belief that conditions, governs, and compels our behavior—is not a new concept. Eighteenth-century philosopher Immanuel Kant theorized that the mind is not designed to give us uninterpreted knowledge of the world but must always approach it from a special point of view … with a certain bias.

Almost everything we do is driven by biases. For instance, when we walk into a dark room (and recognize the dark room “pattern”), we instinctively reach for the light switch on the wall. What prompts this action is the unconscious belief (the bias) that flipping the switch will bring light to the room. We know this because we have experienced the “flip-the-switch-make-the-light-go-on” cause-and-effect pattern before. The depth of our bias becomes apparent whenever we flip the switch and the light doesn’t come on. We’re surprised; our belief has been invalidated; suddenly we are conscious of the bias. We get the same reaction when we turn the spigot on a faucet and no water comes out—something we have always assumed was true (a cause-and-effect pattern) turns out to be false. I will keep reaching for the wall switch until I determine through experience that doing so won’t produce light. Then I’ll have a new bias: the bulb has burned out, and switching it on won’t bring light. I’ll then operate on the basis of the new bias until the bulb is replaced, at which point I’ll revert to the original bias.

Many assumptions become familiar without deliberate examination. We may not notice them until an expectation fails. Making an assumption explicit does not remove every bias, but it gives us something we can test.

Although customarily viewed as a pejorative term, bias is generally a good thing. As with our other human mental traits, we would be dysfunctional without it. It is bias that enables us to repeat an action we have taken before without going through all of the mental steps that led to the original act. In this sense, bias is a preconditioned response. Thus, I am inclined (biased) to hold a golf club in a certain way because I have held it that way before. I shave my face every day as I have shaved it before. I brew coffee today as I did yesterday.

Experience can produce useful expectations, and the assumptions those expectations support deserve examination.* Jones uses experienced drivers to illustrate how recognizing a dangerous situation can help someone respond before a novice would. That does not establish that every assumption becomes more accurate with age. The useful question is whether the experience matches the situation now being judged.

While biases enable us to process new information extremely rapidly by taking mental shortcuts, the rapidity of this process and the fact that it is unconscious—and often unnoticed—have the unfortunate effect of strengthening and validating our biases at the expense of truth. The reason, explained in Section 6 below, is that we tend to give high value to new information that is consistent with our biases, thus reinforcing them, while giving low value to, and even rejecting, new information that is inconsistent with our biases thus preserving them. New information that is ambiguous either is construed as consistent with our biases or is dismissed as irrelevant. In this respect, biases are, like deadly viruses, unseen killers of objective truth.

Hidden biases are the prime movers of the analytic process. They spring to life spontaneously whenever we confront a problem. They are imbedded like computer chips in our brain, ready to carry out their stealthy work at the drop of a problem. By “stealthy” I don’t mean to indicate that all of our hidden biases are detrimental or sinister; as I said, they enable us to function effectively in the world. But biases, like those other instinctive human mental traits, can lead us astray. Because it is uncharacteristic of Homo sapiens to identify, much less evaluate or—heaven forbid, challenge—the hidden biases underlying one’s perspective of a problem, we are unwitting slaves to them. What is more, because many biases operate unnoticed, we can miss their effects on our analysis, conclusions, and recommendations.

Descriptions of reproduction offer an example of how metaphor can shape interpretation. In a June 1992 Discover article, David Freedman challenged accounts of heroic sperm and a passive egg. He described sperm as “tethered” while emphasizing the egg’s active role in binding. Jones uses the account to question the language of the explanation; these metaphors should not be mistaken for literal motives or a complete account of fertilization.

Jones argues that the familiar image of a macho sperm shaped how researchers described their observations.

A research team at Johns Hopkins, even after having revealed the sperm to be an escape artist and the egg to be a chemically active sperm catcher, continued for three years to describe the sperm’s role as actively “penetrating” the egg.

Researchers at the University of Wisconsin described the way sea urchin sperm first make contact with an egg by quickly stringing together protein molecules into a filament that extends out until it reaches the egg. But instead of describing this as an innocuous process of assembly and attachment, the group wrote that the sperm’s filament “shoots out and harpoons” the egg.

When a researcher at the Roche Institute of Molecular Biology wrote of his discovery that mouse eggs carry a molecular structure on their coating that fits inside a complementary structure on the sperm, helping bind the two together, he described the two structures, naturally enough, as a lock and key, but he called the egg’s protruding structure the lock and the sperm’s engulfing structure the key.

An unexpected finding can prompt us to reconsider an assumption. We can also look deliberately for evidence that would challenge a stereotype before a contradiction forces the issue.

The trouble with biases is that they impose artificial constraints and boundaries on what we think. The insidious part is that we aren’t even aware that our thinking is restricted. We go about our daily problem solving believing we are exercising freedom of thought, when in fact our thinking is as tightly bound by our biases as an Egyptian mummy is by its wrappings.

A store manager discovers that money is missing from a cash box, but she dismisses the idea that Glenn, the cashier, stole the money. If he was responsible for the cash being missing, he made an honest mistake. Her instantaneous conclusion (it pops into her head before she can say Abraham Lincoln) is “Of course he didn’t steal the money. He’s as honest as the day is long.” Then those other troublesome mental traits take over and finish what the unconscious shortcut began. Having rationalized away the possibility that he was a thief, she defends her instant conclusion from challenges from those who aren’t acquainted with the cashier and who, like their Missouri relatives, need to be shown that the cashier is as Lincolnesque as she purports. She casts aside their doubts and dismisses their arguments, placing greater stock in her own assessment of his integrity. That assessment is based, of course, on the assumption (the bias) that the cashier is an honest person. If it turns out he is dishonest, her whole line of reasoning collapses like a windblown house of cards.

Alexander Pope warned that “A little learning is a dangerous thing,” and the warning raises a problem for analysis. Knowledge can sharpen understanding while increasing confidence in familiar interpretations. Calling a view “clearer” does not establish that it is more accurate. Expertise helps most when its assumptions remain open to examination.