The Interview Illusion

The Interview Illusion

In the spring of 1999, Dan Heath interviewed a guy named Rob Crum, who was applying for a job as a graphic designer at Thinkwell, the textbook-publishing firm Dan cofounded. Here’s how he remembers the interview process:

Crum was a young man with close-cropped hair, glasses, and clothes that were awfully hip for an interview. He had earrings and a big nose ring that was shaped roughly like the ones you see on bulls. During the interview, he answered questions haltingly, as if deciding how much he should share, and some of his comments seemed a little sarcastic. I didn’t click with him. Over a few weeks, about 10 candidates interviewed for two designer positions, and Rob was toward the bottom of my list.

As a separate part of the interview process, the candidates were asked to complete a work sample—a timed test, conducted in our office, that simulated the kind of work they’d be doing for us (e.g., creating a clean-looking graph for a calculus textbook or illustrating the concept of Bernoulli’s principle). A colleague coded these samples with numbers, rather than names, so that we could score them without knowing which candidate had submitted them. When my cofounder and I compared our scores, we were excited to discover that we’d ranked the same sample as number one. Then we asked our colleague whose sample it was. It was Rob Crum’s.

We debated for a long time whether to hire Rob. I was skeptical; he didn’t seem like he was a “culture fit.” (Wasn’t that crucial?) My first impression had not been very positive. (Aren’t you supposed to trust your instincts?) In the end, though, I agreed to trust the sample and hire him.

Thank goodness I caved. From the beginning, Rob was one of our best people and, after two promotions, he became the art director, overseeing a department of about a dozen artists. He was a gifted designer with a knack for clean and simple visuals, and beyond that, he was a hardworking and conscientious manager. Most embarrassing for me, my first impressions of him had been dead wrong. Ridiculously wrong. Rob turned out to be kind, humble, and sincere. He became a good friend as well as a colleague.

I cringe at how much I struggled with the decision to hire Rob and how much weight I gave to my own flawed first impressions. In retrospect, I wonder why I bothered to interview him at all. I was trying to size him up—to peer into his soul and assess him as a potential colleague. I was trying to predict how good an employee he’d be. But I didn’t need to predict that! The work sample told me everything I needed to know.

By way of comparison, imagine if the U.S. Olympic track coach used two tests in selecting the men who’d run on the 4×100 relay team. Test 1: Get the man on the track to see how fast he runs. And test 2: Meet him in a conference room and see if he answers questions like a fast runner would.

Note that in most of Corporate America, our hiring process looks more like test 2 than test 1. Let’s all slap our foreheads in unison.

Research has found that interviews are less predictive of job performance than work samples, job-knowledge tests, and peer ratings of past job performance. Even a simple intelligence test is substantially more predictive than an interview.

In one study, reported by the psychologist Robyn Dawes, a unique situation emerged that allowed the value of interviews to be assessed. In 1979, the University of Texas Medical School system interviewed the top 800 applicants and scored them on a seven-point scale. These ratings played a key role in the admissions decision, in addition to the students’ grades and the quality of their undergraduate schools. UT admitted only students who ranked higher than 350 (out of 800) on the interview.

Then, unexpectedly, the Texas legislature required the medical school to accept 50 more students. Unfortunately, by the time the school was told to admit more students, the only ones still available were the dregs of the interviewees. So the school admitted 50 of these bottom dwellers, who’d ranked between 700 and 800.

Fortunately, no one at the medical school was aware who were the 700s and who were the 100s, so fate had created a perfectly designed horse race between the good interviewees and the lousy ones. The performance difference? Nada. Both groups graduated and received honors at the same rate.

Well, sure, you scoff, the dregs might do fine in the course work, but a good interviewer picks up on social skills! So once the dregs started working in a real hospital, where relationships are critical, it would become easy to sort the socially skilled from the socially skewed.

Nope, didn’t happen. Both groups performed equally well in the first year of residency. The interviews seemed to correlate with nothing other than, well, the ability to interview.

With so little proof that interviews work, why do we rely on them so much? Because we all think we’re good at interviewing. We are Barbara Walters or Mike Wallace. We leave the interview confident that we’ve taken the measure of the person. The psychologist Richard Nisbett calls this the “interview illusion”: our certainty that we’re learning more in an interview than we really are. He points out that, in grad-school admissions, interviews are often taken as seriously as GPA. The absurdity, he says, is that “you and I, looking at a folder or interviewing someone for a half hour, are supposed to be able to form a better impression than one based on three-and-a-half years of the cumulative evaluation of 20 to 40 different professors.”

HopeLab, the nonprofit mentioned earlier that uses technology to improve kids’ health, has tried to evolve away from interviews. “Often our best interviewees turn out to be our worst performers,” said Steve Cole of HopeLab. In response, HopeLab has begun to give potential employees a three-week consulting contract.

Cole said, “It’s unbelievably effective. No more fear. How are we going to make our hiring decisions? We make our decisions based on the empirical performance of the employee in our community, on the kinds of jobs that we do. The job market totally prevents you from getting this kind of useful information. So collect your own personal performance data in your own personal context. In some ways it really doesn’t matter how well they did in their last job.”

Next time you’ve got a job opening to fill, consider Steve Cole’s advice. What’s the best way you could give your potential hires a trial run?

TO OOCH IS TO ask, Why predict something we can test? Why guess when we can know? Those questions bring us to the end of this section, in which we’ve been studying strategies for fighting the confirmation bias. The basic problem we face, in analyzing our options, is this: We will usually have an inkling of the one that we want to be the winner, and even the faintest inkling will propel us to gather supportive information—and sometimes nothing but supportive information. We cook the books to support our gut instincts.

To avoid that trap, we’ve got to Reality-Test Our Assumptions. We’ve seen three strategies for doing that. First, we’ve got to be diligent about the way we collect information, asking disconfirming questions and considering the opposite. Second, we’ve got to go looking for the right kinds of information: zooming out to find base rates, which summarize the experiences of others, and zooming in to get a more nuanced impression of reality. And finally, the ultimate reality-testing is to ooch: to take our options for a spin before we commit.

Where does this leave us? Armed with better information to make a good choice. In making that choice, which is where we’re headed next, we face an unlikely obstacle. If you’ve ever carefully plotted out a budget, using your best information and analysis, and then promptly ditched it when you came across the perfect pair of shoes—or if you’ve impulsively bought stocks or fearfully dodged a critical relationship conversation—then you’ve already encountered the person who is often the foremost enemy of a wise decision: you.

Next up: what to do about you.