Evidence Pooling
Evidence pooling combines information from several sources while examining its quality, overlap, and consistency. This approach first reviews evidence within each type, then considers how the different types bear on the decision together.
THE IDEA
Evidence pooling brings together information relevant to a decision. Convergent validity concerns how far different indicators point toward a similar conclusion. Disagreement also matters: it may reveal differences in quality, context, or what the sources measure. This relates to the concepts of weight and preponderance of evidence discussed in Chapter 5.
For likelihood estimates, the aim is a pooled probability supported by the combined evidence. Its trustworthiness depends on the aggregation rules, including how they handle source quality and overlapping information. The approach groups evidence as anecdotal, research, and statistical, then proceeds in two stages: (1) review and reconcile evidence within each type and (2) combine the types. Figure 6.1 depicts those stages.
FIGURE 6.1 Two Stages of Evidence Pooling.
When to use it
In business strategy, combining market trends, competitor analysis, and internal capabilities to decide on a new product.
In medical diagnosis, pooling patient history, lab results, and imaging studies for a more accurate diagnosis.
In legal judgment, weighing witness testimony, forensic evidence, and expert opinions to reach a verdict.
In data science, merging multiple models or data sources for a more robust predictive model.
FROM THEORY TO EVERYDAY LIFE
In practice
Business Product Launch Decision
Before a product launch, a company considers customer interview accounts, research reports, and sales data from similar products. The team checks whether these sources support the same demand estimate and whether the reports reuse the same underlying data.
Medical Diagnosis
A doctor considers patient history, relevant research, and test results together. Each source’s contribution depends on its relevance and reliability, including whether several observations reflect the same underlying information.
A MOMENT TO REFLECT
Put it to work.
Where do your sources agree or disagree, and what might explain the differences?
What could go wrong if apparently independent pieces of evidence come from the same underlying source?
When should you prioritize one type of evidence over another in the pooling process?
CONTINUE THE CONVERSATION
Source texts
- Evidence-Based Decision-Making How to Leverage Available Data and Avoid Cognitive Biases (Andrew D. Banasiewicz)