Informational Quality Assessment
Assess data quality by checking what existing measures mean, whether their values are dependable, and how they vary. New indicators, combinations, and aggregates may answer questions the raw fields cannot answer directly. Visualizations can help reveal gaps, unusual values, and limited variation.
THE IDEA
Phase 2: Assessment of Informational Quality
Before concluding that available data are insufficient, consider whether existing measures can be transformed into useful indicators or combined to answer the question.
The source uses indexing to mean expressing raw values as more usable markers. A retailer might define high-value customers using a $1,000+ annual-spend threshold. A customer whose recorded spending is $1,000+ receives an indicator of 1; otherwise the indicator is 0. The threshold is a business definition, so it needs to suit the decision.
A similar operation can create a modeling target. Historical insurance claims may contain a litigation date. After checking what a missing date means, an analyst can derive a litigation indicator (0–1 or no-yes).15 The field used to construct the target should not be mistaken for information available before litigation occurs.
Other transformations combine existing fields. A current high-value customer flag requires both current status and the spending classification. Aggregation creates measures at a different level, such as annual sales from individual purchases. These are ways to derive variables; combining fields does not automatically establish a statistical interaction effect.
Review the original and derived variables for accuracy, meaning, and variation. Are values recorded correctly? Does the measure capture the concept needed for the decision? How much do values differ across records? These questions are related, but a correctly recorded value does not by itself establish that the measure is appropriate.
A variable with little variation may distinguish few records. If nearly all members of a frequent-flyer program are college graduates, a graduate-status flag will not separate most members. That limits some uses of the variable, but does not prove that it is useless: an uncommon category may still matter for a particular outcome.
Visualizations help inspect these properties. Distinguish categorical variables, such as membership status, from quantitative variables, such as spending or age.16 Quantitative variables can be discrete counts or modeled as continuous measures; categorical variables may also be stored as numeric codes.
Bar charts can show category frequencies. Histograms and density plots can show the distribution of quantitative values. Look for missing categories, implausible values, concentrations, and differences across relevant groups. A chart can reveal a problem worth investigating, but it cannot establish the meaning or validity of a measure on its own.
When to use it
A business analyst segments customer data by creating a high-value indicator to target marketing efforts.
A data scientist builds a predictive model for insurance claims by deriving a litigation target from historical data.
A market researcher evaluates survey data for variability to ensure that variables can explain differences between groups.
A manager reviews sales data using histograms and bar charts to understand the distribution of continuous and categorical metrics before making strategic decisions.
FROM THEORY TO EVERYDAY LIFE
In practice
Creating a High-Value Customer Indicator
A retailer defines high-value customers as those with annual spend of $1,000 or more. By indexing the 'annual spend' field, a new binary indicator 'high-value' is created, set to 1 for customers meeting the threshold and 0 otherwise.
Deriving a Litigation Target for Insurance Claims
In insurance claim analysis, an existing 'litigation date' field is repurposed into a binary 'litigation' indicator (0 for no, 1 for yes) to serve as a target variable for predictive modeling of claim litigation threats.
Assessing Variability in Education Levels
If nearly all frequent-flyer members are college graduates, graduate status will distinguish few members. It may be less useful for some comparisons, though its value still depends on the question and outcome being studied.
A MOMENT TO REFLECT
Put it to work.
How can you create new variables from existing data to better assess informational quality in your own work?
What steps would you take to evaluate the validity and variability of your data before making a decision?
In what ways can data visualization help you quickly assess the informational value of your variables?
CONTINUE THE CONVERSATION
Source texts
- Evidence-Based Decision-Making How to Leverage Available Data and Avoid Cognitive Biases (Andrew D. Banasiewicz)