Synthesizing Empirical Research
Research synthesis must fit the decision’s deadline as well as its evidence needs. Banasiewicz’s 3E framework proposes that organizations build on existing research summaries, then assess how well those findings apply to the specific decision.
THE IDEA
Synthesizing Empirical Research
Information can arrive too late to help a decision. If a competitor suddenly cuts its price, an organization may need a quick assessment of likely customer responses before choosing whether to match the offer. A detailed demand analysis delivered after the decision may be less useful. The appropriate balance between speed and precision depends on the stakes, the uncertainty, and the time available.
Academic research often seeks generalizable findings, while a manager needs evidence relevant to a particular choice. Finding studies, judging their methods, and synthesizing their results take time and require human judgment. Evidence-based management therefore needs to account for the difference between a decision deadline and the time needed for a reliable review. Speed alone does not make information useful; relevance and quality still matter.
A synthesis may draw on observational studies, experiments, and existing research summaries. These sources differ in design, samples, measures, and analysis. Independent researchers reaching similar conclusions can strengthen a finding, but differences between studies can also make their results difficult to compare. Reviewers must decide which findings address the same question and which differences limit their application.
A business may lack the time or expertise to conduct a comprehensive research review for each decision. Ignoring research would discard potentially useful evidence. One option is to build on existing syntheses and assess their relevance to the decision, as outlined in Figure 9.1.
The Empirical & Experiential Evidence (3E) framework proposes that commercial organizations build on foundational synthesis carried out by universities and other research organizations. The organization can then focus its effort on how the findings apply to its own circumstances.
FIGURE 9.1 Organizational Synthesis of Theoretical Research.
Meta-analysis uses statistical methods to combine results from sufficiently comparable studies. Its estimates of effect size can help decision-makers judge the magnitude of a relationship or intervention’s effect. It is used in fields such as medicine, education, and criminal justice, but the usefulness of a pooled estimate depends on the quality and comparability of the underlying studies.
A systematic review uses explicit, transparent methods to find, select, appraise, and synthesize research (see Chapter 7). It can include a meta-analysis when statistical pooling is appropriate, or synthesize findings without pooling them. The distinction is therefore not simply quantitative versus qualitative: systematic review describes the review process, while meta-analysis describes a statistical method that may be part of it.
Multiple reviewers 1, explicit selection criteria, and records of review decisions help make a synthesis open to scrutiny, but require time and effort. Those costs can be justified when the decision warrants them; a short deadline may instead favor using a review that already exists. General findings also need assessment against local conditions. In contrast to the emphasis on academic research in Chapters 4 and 5, the 3E proposal emphasizes practical usability when choosing how to summarize and apply evidence.
When to use it
For a time-sensitive competitive decision, look for existing research reviews and assess their relevance before deciding whether new synthesis is feasible.
When assessing evidence about a drug, examine how a research synthesis selected studies, compared their quality, and handled differences in results.
A management team with a short decision horizon can examine existing reviews and meta-analyses, checking their quality and relevance to its situation.
An organization deciding on a new marketing strategy can benefit from rough patterns identified in systematic reviews of previous campaigns, even if those reviews are not yet fully updated.
FROM THEORY TO EVERYDAY LIFE
In practice
Competitor Price Drop Response
When a competitor cuts its price, an organization may need to respond before a detailed analysis is ready. A quick assessment of demand and likely customer responses can be useful, provided its uncertainty and limits are understood.
Pharmaceutical Meta-Analysis for Drug Approval
A pharmaceutical company can use meta-analysis to compare and combine results from relevant drug studies, contributing to an assessment based on more than a single study.
A MOMENT TO REFLECT
Put it to work.
How can you balance the need for timeliness and accuracy when synthesizing research for a pressing business decision?
What research question do you need a review to answer, and when would combining study results statistically be appropriate?
In what situations might rough insights provided sooner be more valuable than precise insights delivered later?
Often confused with
CONTINUE THE CONVERSATION
Source texts
- Evidence-Based Decision-Making How to Leverage Available Data and Avoid Cognitive Biases (Andrew D. Banasiewicz)