The Triumph of Mediocrity (Regression to the Mean)

THE TRIUMPH OF MEDIOCRITY

The early 1930s were a time of soul-searching for the American business community. Something had gone wrong, that much was plain. But what kind of thing? Was the great crash of 1929, and the subsequent depression, an unpredictable catastrophe? Or was the American economy systemically flawed?

Horace Secrist was as well placed as anyone could be to answer this question. Secrist was a professor of statistics and director of the Bureau for Business Research at Northwestern, an expert in the application of quantitative methods to business, and the author of a widely used statistics textbook for students and business executives. Since 1920, years before the crash, he had been compiling meticulously detailed statistics on hundreds of business concerns, from hardware stores to railroads to banks. Secrist tabulated expenses, total sales, outlays on wages and rent, and every other piece of data he could get, trying to locate and taxonomize the mysterious variations which made some businesses thrive and others falter.

So in 1933, when Secrist was ready to reveal the results of his analysis, people in both academia and business were inclined to listen. All the more so when he revealed the striking nature of his results in a 468-page volume, thickly marbled with tables and graphs. Secrist pulled no punches: he called his book The Triumph of Mediocrity in Business.

“Mediocrity tends to prevail in the conduct of competitive business,” Secrist wrote. “This is the conclusion to which this study of the costs (expenses) and profits of thousands of firms unmistakably points. Such is the price which industrial (trade) freedom brings.”

How did Secrist arrive at such a doomy conclusion? First, he stratified the businesses in each sector, carefully segregating the winners (high income, low expenses) from the inefficient duds. The 120 clothing stores Secrist studied, for instance, were first ranked by ratio of sales to expenses in 1916, then divided into six groups, or “sextiles,” of twenty shops each. Secrist expected to see the shops in the top sextile consolidate their gains over time, growing ever more superior as they honed their already top-of-market skills. What he found was precisely the opposite. By 1922, the clothing stores in the highest sextile had lost most of their advantage over the typical shop; they were still better-than-average stores, but by and large they were no longer the standouts. What’s more, the lowest sextile—the worst stores—experienced the same effect in the opposite direction, improving their performance toward the average. Whatever genius had propelled the top-sextile stores to excel had mostly exhausted itself in a mere six years. Mediocrity had triumphed.

Secrist found the same phenomenon in every kind of business. Hardware stores regressed to mediocrity; so did grocery stores. And it didn’t matter what metric you used. Secrist tried measuring his companies by the ratio of wages to sales, the ratio of rent to sales, and whatever other economic stat he could put his hands on. It didn’t matter. With time, the top performers started to look and behave just like the members of the common mass.

Secrist’s book arrived as a bucket of cold water to the face of an already uncomfortable business elite. Many reviewers saw in Secrist’s graphs and tables a numerical disproof of the mythology that sustained entrepreneurship. Robert Riegel of the University of Buffalo wrote, “The results confront the business man and the economist with an insistent and to some degree tragic problem. While there are exceptions to the general rule, the conception of an early struggle, crowned with success for the able and efficient, followed by a long period of harvesting the rewards, is thoroughly dissipated.”

Secrist looked for a force pushing business outliers toward the middle. His temperature comparison seemed to suggest a contrast with the natural world. Secrist, with characteristic thoroughness, had carried out a similar test on the average July temperature for 191 U.S. cities. Here there was no regression. The cities that were hottest in 1922 were just as hot in 1931.

After decades of recording statistics and studying the operation of American business, Secrist thought he knew the answer. It was built into the nature of competition itself to push down successful businesses and promote their incompetent rivals. Secrist wrote:

"Complete freedom to enter trade and the continuance of competition mean the perpetuation of mediocrity. New firms are recruited from the relatively 'unfit'—at least from the inexperienced. If some succeed, they must meet the competitive practices of the class, the market, to which they belong. Superior judgment, merchandising sense, and honesty, however, are always at the mercy of the unscrupulous, the unwise, the misinformed and the injudicious. The results are that retail trade is over-crowded, shops are small and inefficient, volume of business inadequate, expenses relatively high, and profits small. So long as the field of activity is freely entered, and it is; and so long as competition is 'free,' and within the limits suggested above, it is; neither superiority nor inferiority will tend to persist. Rather, mediocrity tends to become the rule. The average level of the intelligence of those conducting business holds sway, and the practices common to such trade mentality become the rule."

Secrist’s view contrasts with the familiar claim that free-market competition is the cleansing blade that cuts down the incompetent and the 10%-less-than-maximally-competent alike. Inferior firms are at the mercy of their betters, not the other way around.

But Secrist saw the free market, with its firms of different sizes and skill levels jostling against one another, as something like the one-room schoolhouse that was, by 1933, already well on its way to disuse. As Secrist describes it: “Pupils of all ages, of different mentality, and of training, grouped together in a single room, were to be educated. Pandemonium, discouragement, and inefficiency, of course, resulted. Common sense later pointed to the desirability of classification, grading, special treatment—corrections which opened the way for native ability to assert itself, and for superiority to withstand being watered down and diluted by inferiority.”

That last part sounds a little—well, how should I put it—can you think of anybody else in 1933 who was talking about the importance of superior beings withstanding dilution by inferior ones?

Given the flavor of Secrist’s views on education, it may come as no surprise that his ideas about regression to mediocrity descend from those of the nineteenth-century British scientist and pioneering eugenicist Francis Galton. Galton was the youngest of seven children and a sort of child prodigy. Galton’s bedridden older sister Adèle took on his education as her chief amusement; he could sign his name at two, and by four was writing her letters like this: “I can cast up any sum in addition and can multiply by 2, 3, 4, 5, 6, 7, 8, 10. I can also say the pence table. I read French a little and I know the Clock.” Galton had a peripatetic education; he tried studying mathematics at Cambridge but was defeated by the brutal Tripos exam, and devoted himself intermittently to the study of medicine, the line of work his parents had planned for him. But after his father died in 1844, leaving him a substantial fortune, he found himself suddenly less motivated to pursue a traditional career. For a while Galton was an explorer, leading expeditions into the African interior. But the epochal publication of The Origin of Species in 1859 catalyzed a drastic shift in his interests: Galton recalls that he “devoured its contents and assimilated them as fast as they were devoured,” and from then on, the greater share of Galton’s work was devoted to the heredity of human characteristics, both physical and mental. This work led him to a suite of policy preferences that are decidedly unsavory from a modern point of view. The opening of his 1869 book Hereditary Genius gives the flavor:

"I propose to show in this book that a man’s natural abilities are derived by inheritance, under exactly the same limitations as are the form and physical features of the whole organic world. Consequently, as it is easy, notwithstanding those limitations, to obtain by careful selection a permanent breed of dogs or horses gifted with peculiar powers of running, or of doing anything else, so it would be quite practicable to produce a highly-gifted race of men by judicious marriages during several consecutive generations."

Galton made his case by means of a detailed study of British men of achievement, from clerics to wrestlers, arguing that notable Englishmen tend to have disproportionately notable relatives. Hereditary Genius met with a great deal of resistance, particularly from the clergy; Galton’s purely naturalistic view of worldly success left little room for a more traditional view of Providence. Especially irksome was Galton’s claim that success in ecclesiastical pursuits was itself subject to hereditary influence: that, as one reviewer complained, “a pious man owes his piety not so much (as we had ever believed) to the direct action of the Holy Ghost on his soul, blowing like the wind where it listeth, but rather to his earthly father’s physical bequest of a constitution adapted to the religious emotions.” Whatever friends Galton had among the religious establishment were surely lost three years later, when he published a short article titled “Statistical Inquiries into the Efficacy of Prayer.” (Executive summary: prayer not so efficacious.)

By contrast, Galton’s book was received with great excitement, if not uncritical acceptance, by the Victorian scientific community. Charles Darwin wrote Galton in a kind of intellectual frenzy, not even waiting until he’d finished the book:

DOWN, BECKENHAM, KENT, S.E. December 23rd

The letter itself is not included in this excerpt.

What Regression Explains

The business results do not, by themselves, show that competition destroys excellence. An unusually good period can reflect both a firm’s lasting strengths and favorable temporary conditions. Selecting firms for extreme performance also selects for unusually favorable luck. When that luck changes, the group’s later performance can be closer to average even if its underlying strengths remain. The reverse can happen to firms selected for unusually poor results.

This tendency does not require every business to converge to the same level, nor is it confined to business. Persistent differences among cities can dominate a temperature comparison, while temporary variation may matter more in another set of measurements. The size of regression depends on how strongly the measurements are related; it is not an independent force pushing everything toward mediocrity.