Caveats and Contingencies

SOME CAVEATS AND CONTINGENCIES

If you want to get better at conveying complexity, it’s worth taking a close look at how scientists communicate. One key step is to include caveats. It’s rare that a single study or even a series of studies is conclusive. Researchers typically feature multiple paragraphs about the limitations of each study in their articles. We see them less as holes in our work and more as portholes to future discoveries. When we share the findings with nonscientists, though, we sometimes gloss over these caveats.

Research on science communication suggests that this is a mistake. In a series of experiments, psychologists demonstrated that when news reports about science included caveats, they succeeded in capturing readers’ interest and keeping their minds open. Take a study suggesting that a poor diet accelerates aging. Readers were just as engaged in the story—but more flexible in their beliefs—when it mentioned that scientists remained hesitant to draw strong causal conclusions given the number of factors that can affect aging. It even helped just to note that scientists believed more work needed to be done in this area.

We can also convey complexity by highlighting contingencies. Every empirical finding raises unanswered questions about when and where results will be replicated, nullified, or reversed. Contingencies are all the places and populations where an effect may change.

Consider diversity: although headlines often say “Diversity is good,” the evidence is full of contingencies. Although diversity of background and thought has the potential to help groups think more broadly and process information more deeply, that potential is realized in some situations but not others. Research discussed here found that people were more likely to promote diversity and inclusion when the message is more nuanced (and more accurate): “Diversity is good, but it isn’t easy.”* In these studies, acknowledging complexity made messages more credible while maintaining engagement and encouraging curiosity.

In social science, rather than cherry-picking information to fit our existing narratives, we’re trained to ask whether we should rethink and revise those narratives. When we find evidence that doesn’t fit neatly into our belief systems, we’re expected to share it anyway.* In some of my past writing for the public, though, I regret not having done enough to emphasize areas where evidence was incomplete or conflicting. I sometimes shied away from discussing mixed results because I didn’t want to leave readers confused. Research suggests that many writers fall into the same trap, caught up in trying to “ maintain a consistent narrative rather than an accurate record.”

A fascinating example is the divide around emotional intelligence. On one extreme is Daniel Goleman, who popularized the concept. He preaches that emotional intelligence matters more for performance than cognitive ability (IQ) and accounts for “nearly 90 percent” of success in leadership jobs. At the other extreme is Jordan Peterson, writing that “There is NO SUCH THING AS EQ” and prosecuting emotional intelligence as “a fraudulent concept, a fad, a convenient band-wagon, a corporate marketing scheme.”

Both men hold doctorates in psychology, but neither seems particularly interested in creating an accurate record. If Peterson had bothered to read the comprehensive meta-analyses of studies spanning nearly two hundred jobs, he’d have discovered that—contrary to his claims—emotional intelligence is real and it does matter. Emotional intelligence tests predict performance even after controlling for IQ and personality. If Goleman hadn’t ignored those same data, he’d have learned that if you want to predict performance across jobs, IQ is more than twice as important as emotional intelligence (which accounts for only 3 to 8 percent of performance).

I think they’re both missing the point. Instead of arguing about whether emotional intelligence is meaningful, we should be focusing on the contingencies that explain when it’s more and less consequential. It turns out that emotional intelligence is beneficial in jobs that involve dealing with emotions, but less relevant—and maybe even detrimental—in work where emotions are less central. If you’re a real estate agent, a customer service representative, or a counselor, being skilled at perceiving, understanding, and managing emotions can help you support your clients and address their problems. If you’re a mechanic or an accountant, being an emotional genius is less useful and could even become a distraction. If you’re fixing my car or doing my taxes, I’d rather you didn’t pay too much attention to my emotions.

In an effort to set the record straight, I wrote a short LinkedIn post arguing that emotional intelligence is overrated. I did my best to follow my own guidelines for complexity:

Nuance: This isn’t to say that emotional intelligence is useless.

Caveats: As better tests of emotional intelligence are designed, our knowledge may change.

Contingencies: For now, the best available evidence suggests that emotional intelligence is not a panacea. Let’s recognize it for what it is: a set of skills that can be beneficial in situations where emotional information is rich or vital.

Over a thousand comments poured in, and I was pleasantly surprised that many reacted enthusiastically to the complexified message. Some mentioned that nothing is either/or and that data can help us reexamine even our closely held beliefs. Others were downright hostile. They turned a blind eye to the evidence and insisted that emotional intelligence was the sine qua non of success. It was as if they belonged to an emotional intelligence cult.

From time to time I’ve run into idea cults—groups that stir up a batch of oversimplified intellectual Kool-Aid and recruit followers to serve it widely. They preach the merits of their pet concept and prosecute anyone who calls for nuance or complexity. In the area of health, idea cults defend detox diets and cleanses long after they’ve been exposed as snake oil. In education, there are idea cults around learning styles—the notion that instruction should be tailored to each student’s preference for learning through auditory, visual, or kinesthetic modes. Some teachers are determined to tailor their instruction accordingly despite decades of evidence that although students might enjoy listening, reading, or doing, matching instruction to a preferred mode does not reliably improve learning. In psychology, I’ve inadvertently offended members of idea cults when I’ve shared evidence that meditation isn’t the only way to prevent stress or promote mindfulness; that when it comes to reliability and validity, the Myers-Briggs personality tool falls somewhere between a horoscope and a heart monitor; and that being more authentic can sometimes make us less successful. If you find yourself saying ____ is always good or ____ is never bad, you may be a member of an idea cult. Appreciating complexity reminds us that no behavior is always effective and that all cures have unintended consequences.

xkcd.com

In the moral philosophy of John Rawls, the veil of ignorance asks us to judge the justice of a society by whether we’d join it without knowing our place in it. I think the scientist’s veil of ignorance is to ask whether we’d accept the results of a study based on the methods involved, without knowing what the conclusion will be.