Data Aggregation
Data aggregation makes information easier to obtain and analyze, but group averages conceal differences within groups. An area’s average income, for example, does not describe the income of every resident.
THE IDEA
Data Aggregation
Data aggregators collect information from different sources and make it available together. They may also remove duplicates, impute missing values, handle outliers and combine files. These services reduce the work of obtaining usable data, but their transformations can remove information as well as noise.
Geodemographic data illustrate the trade-off. They combine demographic attributes, such as income, age and education, with geographic areas. Individual or household observations are summarized for groups—typically 20–30 nearby households in the example described here 16. The resulting averages are useful for comparing areas, but they conceal variation among the people within each area.
This loss of detail through averaging is distinct from regression to the mean, in which unusually extreme observations tend to be followed by less extreme observations when measured again. Treating an area average as if it described every resident can produce misleading conclusions about individuals.
Privacy restrictions can prevent data providers from releasing individual records while allowing grouped statistics. When using those statistics, check how the groups were defined, how missing values were handled and which differences the aggregation hides.
When to use it
Market research using census block group data for targeted advertising
Healthcare studies that use zip-code level averages to assess disease prevalence
Economic forecasting based on aggregated GDP or unemployment indices
Educational policy analysis using school-district averages for test scores
FROM THEORY TO EVERYDAY LIFE
In practice
Geodemographic Marketing
A marketing analyst buys income data summarized for groups of 20-30 nearby households. Each group average conceals differences among its households. A campaign based on that average may therefore miss customers whose incomes differ substantially from their neighbors’ incomes.
A MOMENT TO REFLECT
Put it to work.
How might relying on aggregated data lead to misleading conclusions in your own field of work or study?
What are the trade-offs between data usability and data fidelity when using aggregated data?
Where might a group average conceal individual differences that matter to a decision you are making?
CONTINUE THE CONVERSATION
Source texts
- Evidence-Based Decision-Making How to Leverage Available Data and Avoid Cognitive Biases (Andrew D. Banasiewicz)