Kolmogorov Complexity (Andrey Nikolayevich's Rule)

Andrey Nikolayevich's Rule

There is another, even deeper reason for our inclination to narrate, and it is not psychological. It has to do with the effect of order on information storage and retrieval in any system, and it's worth explaining here because of what I consider the central problems of probability and information theory.

The first problem is that information is costly to obtain.

The second problem is that information is also costly to store—like real estate in New York. The more orderly, less random, patterned, and narratized a series of words or symbols, the easier it is to store that series in one's mind or jot it down in a book so your grandchildren can read it someday.

Finally, information is costly to manipulate and retrieve.

With so many brain cells—one hundred billion (and counting)—the attic is quite large, so the difficulties probably do not arise from storage-capacity limitations, but may be just indexing problems. Your conscious, or working, memory, the one you are using to read these lines and make sense of their meaning, is considerably smaller than the attic. Consider that your working memory has difficulty holding a mere phone number longer than seven digits. Change metaphors slightly and imagine that your consciousness is a desk in the Library of Congress: no matter how many books the library holds, and makes available for retrieval, the size of your desk sets some processing limitations. Compression is vital to the performance of conscious work.

Consider a collection of words glued together to constitute a 500-page book. A book of randomly selected words will generally lack the repeated structure needed for a much shorter exact description. A random 100,000-word sequence is therefore much harder to describe compactly than a patterned sequence of 100,000 words. Now consider the opposite: a book filled with the repetition of the following sentence: "The chairman of [insert here your company name] is a lucky fellow who happened to be in the right place at the right time and claims credit for the company's success, without making a single allowance for luck," running ten times per page for 500 pages. The example describes this as compression into 34 words (out of 100,000). The point is that a repeated passage plus its repetition rule can describe the whole book; these illustrative word counts are not a calculation of its exact complexity. By finding the pattern, the logic of the series, you no longer need to memorize it all. You just store the pattern. And, as we can see here, a pattern is obviously more compact than raw information. You looked into the book and found a rule. Kolmogorov formalized descriptive complexity in terms of a shortest program that reproduces a string. The measure is called "Kolmogorov complexity."

We, members of the human variety of primates, have a hunger for rules because we need to reduce the dimension of matters so they can get into our heads. Or, rather, sadly, so we can squeeze them into our heads. The more random information is, the greater the dimensionality, and thus the more difficult to summarize. The more you summarize, the more order you put in, the less randomness. Hence the same condition that makes us simplify pushes us to think that the world is less random than it actually is.

And the Black Swan is what we leave out of simplification.

Both the artistic and scientific enterprises are the product of our need to reduce dimensions and inflict some order on things. Think of the world around you, laden with trillions of details. Try to describe it and you will find yourself tempted to weave a thread into what you are saying. A novel, a story, a myth, or a tale, all have the same function: they spare us from the complexity of the world and shield us from its randomness. Myths impart order to the disorder of human perception and the perceived "chaos of human experience."

Indeed, many severe psychological disorders accompany the feeling of loss of control of—being able to "make sense" of—one's environment.

Platonicity affects us here once again. The very same desire for order, interestingly, applies to scientific pursuits—it is just that, unlike art, the (stated) purpose of science is to get to the truth, not to give you a feeling of organization or make you feel better. We tend to use knowledge as therapy.