Theory-Laden vs Data-Laden Knowledge Creation
Banasiewicz distinguishes theory-guided hypothesis testing from exploratory analysis that seeks patterns without a predetermined explanatory theory. He favors using established knowledge to guide and check new findings. Both approaches depend on data quality, and exploratory findings need further testing before they support confident decisions.
THE IDEA
How do we know that a finding deserves to count as knowledge? Evidence must be gathered, but it must also be interpreted. Galileo’s telescopic observations supported Copernican ideas and challenged established astronomy. Accepting those observations depended in part on trusting the instrument and understanding its optics. Observation and the beliefs used to interpret it were intertwined.
Unfamiliar ideas can still face this difficulty. Quantum theory, for example, makes assumptions that many people find counterintuitive.3 Familiarity is not a test of truth, so we need ways to compare explanations with evidence.
Banasiewicz distinguishes theory-laden and data-laden approaches to creating decision-guiding knowledge. In the theory-laden approach, an existing framework generates hypotheses that empirical analysis can test. Results are interpreted in relation to that framework. A theory supplies a reason to expect a pattern and helps explain what the pattern would mean if found.
The source connects this approach to research on how background beliefs shape information processing, discussed in Chapter 1. Its broader point is that learning builds on what we already believe. However, a handful of loosely stated assumptions or findings from an earlier analysis does not necessarily amount to an explanatory theory. Several competing theories may also fit parts of the evidence, particularly in the social sciences.
The data-laden approach begins with exploration rather than a predetermined explanatory theory. Analysts look for useful patterns in available data and develop possible explanations from what they find. This can reveal relationships that an established framework overlooks. The analysis still involves choices about what to measure, which records to include, and how to interpret the results.
Banasiewicz emphasizes the risks of this dependence on available data. Errors, omissions, and noisy measurements can produce misleading patterns. Automated mining of large event databases can also generate statistically significant findings with little practical value.4 A pattern’s statistical significance does not establish that it is useful, causal, or likely to recur in new data.
Theory-guided analysis offers checks that isolated exploration may lack. Related studies can test whether a finding recurs, and different predictions from the same theory can be compared. New results can then strengthen, challenge, or revise a cumulative explanation. These checks help only when they are actually used: an established theory does not repair unreliable measurements or protect an analysis from bias.
The practical choice need not be exclusive. Exploration can suggest a hypothesis; theory can help specify it; further evidence can test it. Before using either kind of finding for a business decision, examine the data’s limitations and ask how the conclusion has been checked.
When to use it
A scientist testing a hypothesis derived from an established theory.
A business analyst exploring customer data to find new patterns without predetermined questions.
A policy maker evaluating the impact of a new regulation using both theory and data.
A researcher in social sciences dealing with multiple competing theoretical frameworks.
FROM THEORY TO EVERYDAY LIFE
In practice
The Galileo Affair
Galileo’s telescopic observations supported Copernican astronomy, while disputes about interpreting those observations included questions about the instrument and optics. The example shows why accepting evidence also involves assessing how it was obtained.
Automated Data Mining in Business
An analyst mines a large event database and finds many statistically significant relationships. Some may be too small to matter or may fail to recur. The next task is to assess their practical importance and test them with further evidence.
A MOMENT TO REFLECT
Put it to work.
In your own decision-making, do you tend to rely more on existing theories or on exploring data? What are the potential pitfalls of each approach?
How can you ensure that data exploration in business leads to valid insights rather than spurious correlations?
Consider a situation where you had to choose between a theory-driven and a data-driven approach. What factors influenced your choice?
Often confused with
CONTINUE THE CONVERSATION
Source texts
- Evidence-Based Decision-Making How to Leverage Available Data and Avoid Cognitive Biases (Andrew D. Banasiewicz)