Wittgenstein's Ruler
Taleb compares evaluating a judgment with measuring a table: an unreliable ruler may tell you more about the ruler than the table. A review can likewise reveal the reviewer’s assumptions and competence, so assess the source as well as the claim.
THE IDEA
WITTGENSTEIN’S RULER
What is the mechanism that should convince authors to avoid reading comments on their work, except for those they solicit from specified persons for whom they have intellectual respect? The mechanism is a probabilistic method called conditional information: Unless the source of the statement has extremely high qualifications, the statement will be more revealing of the author than the information intended by him. This applies, of course, to matters of judgment. A book review, good or bad, can be far more descriptive of the reviewer than informational about the book itself. This mechanism I also call Wittgenstein’s ruler: Unless you have confidence in the ruler’s reliability, if you use a ruler to measure a table you may also be using the table to measure the ruler. The less you trust the ruler‘s reliability (in probability called the prior), the more information you are getting about the ruler and the less about the table. The point extends way beyond information and probability. This conditionality of information is central in epistemology, probability, even in studies of consciousness. We will see later extensions with “ten sigma” problems.
The point carries practical implications: The information from an anonymous reader on Amazon.com is all about the person, while that of a qualified person, is going to be all about the book. This plays equally in court: Take the O. J. Simpson trial once again. One of the jurors said, “There was not enough blood,” meaning to assess the statistical evidence of what was offered: Such statement reveals very little about the statistical evidence as compared with what it shows about the author of the statement’s ability to make a valid inference. Had the juror been a forensic expert, the ratio of information would have tilted the other way.
The problem is that while such reasoning is central to my thinking, my brain knows it though not my heart: My emotional system does not understand Wittgenstein’s ruler. I can offer the following evidence: A compliment is always pleasant, regardless of its authorship—something manipulators know rather well. Likewise with book reviews or comments on my risk-management strategy.
When to use it
Evaluating online reviews for products or services to determine their credibility.
Assessing expert testimony in legal proceedings by considering the witness's qualifications.
Interpreting feedback on your work from different sources (e.g., anonymous comments vs. trusted colleagues).
Deciding which book reviews to take seriously when choosing what to read.
FROM THEORY TO EVERYDAY LIFE
In practice
Amazon Book Review
A review that misdescribes a book may reveal more about the reviewer’s assumptions than the book. Expertise can make a review more informative, but the reasons and evidence still need examination.
OJ Simpson Trial Juror
Taleb interprets a juror’s remark about insufficient blood as evidence about the juror’s statistical reasoning. His example illustrates how confidence in the speaker changes what a remark tells us; it does not independently establish the adequacy of the trial evidence.
A MOMENT TO REFLECT
Put it to work.
When reading a review or critique, ask yourself: What does this statement reveal about the author's qualifications and biases?
How can you evaluate the reliability of information based on the source, using the concept of conditional information?
Think of a time you were influenced by an opinion from an unqualified source. How does Wittgenstein's ruler apply to that situation?
CONTINUE THE CONVERSATION
Source texts
- Fooled by Randomness The Hidden Role of Chance in the Markets and in Life (Nassim Nicholas Taleb)