Sanity Testing
After using any analytic structuring technique, perform a sanity check by asking whether the result makes sense. If not, reexamine or redo the analysis.
THE IDEA
Sanity Testing
After using any analytic structuring technique, do a “sanity check” by asking, “Does this make sense?” If the analytic result is reasonable, we’re probably okay. But if our intuition tells us something doesn’t seem right, we should go back and reexamine or completely redo our analysis.
When to use it
After completing a cost-benefit analysis, verify that the numbers are plausible and consistent with known benchmarks.
When conducting a statistical test, check that the results align with existing domain knowledge or common sense.
After creating a project plan, review timelines and resource estimates for realism before committing.
When debugging code, test the logic with a simple, predictable input to ensure the algorithm works as intended.
FROM THEORY TO EVERYDAY LIFE
In practice
Regression Output
A data analyst runs a regression model and gets a negative R-squared. Instead of accepting it, they perform a sanity check and realize they made a coding error in the formula.
Financial Forecast
A business manager creates a financial forecast showing huge profits with no associated costs. A sanity check flags the unrealistic assumption, prompting a review of the input data.
A MOMENT TO REFLECT
Put it to work.
When was the last time you performed a sanity check on your analysis? What did you discover?
How can you incorporate a sanity check into your routine decision-making process?
What are some common red flags that might indicate your analysis is off?
CONTINUE THE CONVERSATION
Source texts
- The Thinkers Toolkit by Morgan D Jones