Herding Like Cattle
Taleb describes studies in which security analysts showed excessive confidence and forecasts performed little better than simple extrapolation. In one dataset, forecasts clustered more tightly together than their errors warranted, suggesting herding.
THE IDEA
Herding Like Cattle
A few researchers have examined the work and attitude of security analysts, with amazing results, particularly when one considers the epistemic arrogance of these operators. In a study comparing them with weather forecasters, Tadeusz Tyszka and Piotr Zielonka document that the analysts are worse at predicting, while having a greater faith in their own skills. Somehow, the analysts’ self-evaluation did not decrease their error margin after their failures to forecast.
During a visit to Jean-Philippe Bouchaud in Paris, I bemoaned the dearth of such published studies. He is a boyish man who looks half my age though he is only slightly younger than I, a matter that I half jokingly attribute to the beauty of physics. Actually he is not exactly a physicist but one of those quantitative scientists who apply methods of statistical physics to economic variables, a field that was started by Benoît Mandelbrot in the late 1950s. This community does not use Mediocristan mathematics, so they seem to care about the truth. They are completely outside the economics and business-school finance establishment, and survive in physics and mathematics departments or, very often, in trading houses (traders rarely hire economists for their own consumption, but rather to provide stories for their less sophisticated clients). Some of them also operate in sociology with the same hostility on the part of the “natives.” Unlike economists who wear suits and spin theories, they use empirical methods to observe the data and do not use the bell curve.
He surprised me with a research paper that a summer intern had just finished under his supervision and that had just been accepted for publication; it scrutinized two thousand predictions by security analysts. What it showed was that these brokerage-house analysts predicted nothing—a naïve forecast made by someone who takes the figures from one period as predictors of the next would not do markedly worse. Yet analysts are informed about companies’ orders, forthcoming contracts, and planned expenditures, so this advanced knowledge should help them do considerably better than a naïve forecaster looking at the past data without further information. Worse yet, the forecasters’ errors were significantly larger than the average difference between individual forecasts, which indicates herding. Normally, forecasts should be as far from one another as they are from the predicted number. But to understand how they manage to stay in business, and why they don’t develop severe nervous breakdowns (with weight loss, erratic behavior, or acute alcoholism), we need to understand what sustains their confidence, a question explored by psychologist Philip Tetlock.
When to use it
Evaluating the reliability of financial analyst recommendations for investment decisions.
Assessing group decision-making in corporate strategy where consensus is mistaken for accuracy.
Understanding policy forecasts where experts may converge to a shared view without independent verification.
Recognizing the danger of relying on consensus in any field where past predictions have been poor, such as economic forecasting.
FROM THEORY TO EVERYDAY LIFE
In practice
Tyszka and Zielonka: Analysts vs. Weather Forecasters
Taleb describes a study by Tyszka and Zielonka in which security analysts predicted less accurately than weather forecasters while expressing greater confidence. Their confidence did not adjust adequately after forecasting failures.
Bouchaud's Intern Study: 2000 Analyst Predictions
A research paper by Jean-Philippe Bouchaud's intern analyzed 2000 predictions from security analysts. It found that analysts' forecasts were no better than a naïve model using past data, despite their access to information about companies’ plans. The error was larger than the average spread among forecasts, indicating herding behavior.
A MOMENT TO REFLECT
Put it to work.
In what ways does herding influence your own decisions, especially when you have access to private information?
Why do security analysts maintain high self-confidence despite consistently poor forecasts? How might this apply to other professionals?
Can you identify a recent collective prediction (e.g., election polls, product sales) where herding may have inflated confidence and reduced accuracy?
Often confused with
References
- Study comparing security analysts and weather forecasters — Tadeusz Tyszka and Piotr Zielonka0
- Research paper on 2000 analyst predictions — Jean-Philippe Bouchaud and intern0
- Work on forecasting and expert judgment — Philip Tetlock0
CONTINUE THE CONVERSATION
Source texts
- The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)