Research Synthesis Challenge
Synthesizing findings from a large body of empirical research faces challenges similar to big data: high volume, high variety, and the need for subjective human evaluation, leading to issues of inter-rater reliability.
THE REFERENCE LIBRARY
A growing collection of ideas for understanding the world — and your own thinking.
Synthesizing findings from a large body of empirical research faces challenges similar to big data: high volume, high variety, and the need for subjective human evaluation, leading to issues of inter-rater reliability.
Uses the habit loop (cue, routine, reward) to replace self-serving bias with truth-seeking, as exemplified by Phil Ivey's post-win self-critique. By substituting the routine while keeping the cue and reward, we can improve decision-making and learning.
A practice of selecting an intellectual trait (e.g., humility) and focusing on it for a month, noticing negative behaviors and consciously reshaping one's character.
Resilience is the capacity to maintain or recover important functions after a disturbance. Depending on the system, it may involve absorbing a shock, adapting to changed conditions, or reorganizing. Diversity, backup capacity, useful feedback, and access to support can contribute to it.
Dawson distinguishes recovering from setbacks, or resilience, from maintaining purposeful effort under pressure, or mental toughness. These abilities overlap and can develop through experience, practice, and support.
Systems have limits to resilience and must be managed for resilience as well as productivity. They exhibit self-organization and hierarchical structures that evolve from the bottom up, with upper layers serving lower layers.
Resistance to personal mastery in organizations stems from its perceived softness and unquantifiability, cynicism from disappointed idealists, and fear that empowering people without alignment increases organizational stress.
Temptation and self-control problems arise from a conflict between a farsighted 'Planner' and a myopic 'Doer,' leading to dynamic inconsistency. Commitment strategies, like removing temptation or pre-committing, help overcome the hot-cold empathy gap where people underestimate arousal's effect on behavior.
Taleb favors practices with long records of survival when short observation periods can hide rare failures. He argues that those proposing to disturb a complex system should bear the burden of proof; longevity is a heuristic rather than proof of safety.
Two studies tested whether reported cross-cultural differences in overconfidence merely reflected different uses of probability scales. Selling prices for bets and judgments made under proper scoring incentives did not support that explanation.
Anticipated responsibility and regret can make people especially reluctant to exchange health or safety for money. Kahneman examines how this affects risk tradeoffs, including the cost of protecting against one danger at the expense of another, while recognizing that regret itself matters to people.
The restaurant problem asks which customers could violate a conditional drinking-age rule. Its familiar setting makes the relevant checks easier to see than in an abstract card task: check drinkers’ ages and whether underage customers are drinking.
The tendency to overestimate one's ability to control impulses, leading to overexposure to temptation and higher likelihood of relapse.
Before clicking a search result, examine the URL, title, and snippet to quickly assess the source's authority and relevance. This practice, called result mining, saves time and improves search efficiency.
Resulting is the cognitive bias of judging a decision's quality based solely on its outcome, rather than the process or information available at the time. It leads to overlearning from luck and underlearning from good processes that happen to yield bad results.
The mental model of rethinking emphasizes the need to constantly update our beliefs and knowledge, especially in a rapidly changing world. Mike Lazaridis's BlackBerry story illustrates how failure to rethink can lead to dramatic decline.
Rethinking means revisiting beliefs and commitments when experience gives us reason to change them. Adam Grant applies it to careers and identity: interests can develop through practice, goals can have several possible paths, and even a small change in a role may make work more meaningful. Humility and curiosity help keep those options open.
This label describes trying to relieve a problem through the same source that sustains it. The error lies in treating temporary relief as a solution while overlooking whether the underlying problem becomes worse.
Assuming that because something happened, it necessarily had to happen, i.e. that it was the only possible outcome.
When experts explain history as inevitable, a plausible alternative history can threaten their account. Tetlock found that experts with strong theoretical commitments and a preference for explanatory closure were more likely to reject such alternatives, sometimes arguing that another event would have produced the same outcome.
Reversal is a lateral thinking technique that involves deliberately reversing a situation's assumptions or relationships to provoke new perspectives and creative solutions. It is not about finding a single correct reversal but about generating alternative arrangements of information.
Reversal is a creative thinking technique where you reverse a situation or its aspects to generate new perspectives and ideas, even if the reversal seems ridiculous.
A lateral thinking technique that involves deliberately reversing a conventional assumption or viewpoint to escape standard patterns of thought, often leading to new insights or solutions.
A creative thinking exercise where individuals generate alternative perspectives by reversing a given situation in as many ways as possible, then categorizing the types of reversals.
Judgments and choices can reverse dramatically depending on whether options are evaluated jointly (side-by-side) or singly (one at a time), revealing inconsistencies in our moral intuitions and emotional reactions.
Reverse causation means mistaking the direction of a causal relationship. An association alone does not tell you whether one variable affects the other, both directions operate, or another factor explains the pattern.
Reverse engineering works backward from an observed output to possible inputs or causes. Algebra illustrates the difficulty: computing a result can be straightforward, while recovering what produced it may require a different method and yield more than one answer.
A skilled person can fail despite favorable odds. Taleb calls this a reverse survivor and links short-term career pressure to strategies that usually succeed but carry a risk of catastrophic loss.
Taleb’s satirical reversal of the Turing test asks whether random phrase assembly can imitate supposedly substantial writing. His postmodern and corporate examples challenge readers to identify the claims and evidence beneath fluent prose; resemblance alone is not a test of a person’s intelligence.
The principle that behavior is shaped by its consequences: actions that are rewarded tend to be repeated, while actions that are punished tend to stop. This concept explains how incentives, reinforcement, and punishment influence human behavior in various domains, from personal habits to organizational systems.
Rewards and punishments can strongly influence behavior and judgment. Munger argues that people often underestimate this influence, rationalize actions that serve their incentives, and design systems that reward the wrong results.
Rhythm is the pattern and pace of events that provides structure and direction in music, language, and daily life. It synchronizes us with our environment and influences our emotions, attention, and social connections.
Rhythm organizes events in time through duration, repetition, pauses, and accents. It shapes musical phrases and speech, while recurring routines give daily activities a recognizable pattern.
The ecological fallacy occurs when conclusions about individuals are drawn from aggregate group data, leading to incorrect inferences, often seen in political, economic, and sociological analysis.
A righteous tone is a rhetorical device where speakers use emphatic, absolutist language—such as 'Period. Full Stop.' or breaking sentences with periods after each word—to convey moral superiority and shut down discussion, signaling that their position is ethically unassailable.
A self-deception can encourage further distorted beliefs as we try to explain other experiences consistently with it. Galef argues that these delayed, unpredictable effects are easy to overlook when weighing the apparent benefit of a comforting belief.
Risk is the effect of uncertainty on objectives and can involve opportunities as well as threats. Analytical tools help identify and manage some risks, while complex systems leave important interactions and outcomes uncertain.
People are risk-averse when facing potential gains and risk-seeking when facing potential losses, a pattern explained by a psychological value function that is concave for gains, convex for losses, and steeper for losses (loss aversion). This challenges classical expected utility theory and forms the basis of prospect theory.
Risk compensation describes taking more risks when protection makes a situation feel safer. Such behavior may offset some intended safety benefits; it does not establish that every protective measure is ineffective.
An approach to relationships based on ISO 31000 risk management separates possible harms, their likelihood, and their consequences. This account also illustrates the limits of that approach: unsupported population claims and assumptions about gender cannot establish an individual partner’s risk, and fixation on one person can distort judgment.
Taleb describes a conflict facing institutional risk managers: stopping a profitable trade can attract blame for lost gains, while allowing it can attract blame for a later collapse. He argues that these incentives can encourage vague warnings and self-protection.
A risk policy is a broad frame that embeds risky choices in a set of similar choices, helping to mitigate narrow framing and loss aversion by applying a consistent rule across similar situations.
A classic demonstration of how framing concurrent decisions in terms of gains versus losses leads to inconsistent risk preferences, where the logical rule of dominance is violated due to narrow bracketing. Based on Kahneman and Tversky's prospect theory experiment.
Failure mode and effects analysis (FMEA) examines how a process could fail and which failures deserve attention. Detailed risks, useful indicators, early milestones, and candid reporting help turn that analysis into action.
An experiment demonstrating that people commit the conjunction fallacy even when betting with real money, by choosing a less probable but more representative sequence of dice rolls over a more probable one.
Rituals are structured practices that shift collective attention and patterns of thinking during transitions, such as responding to emergencies or exploring strategic opportunities. Simulation exercises serve as rituals that prepare organizations for the unexpected.
Rivalries fuel intense negative stereotypes and prejudice toward out-groups, leading people to harm rivals even at the expense of their own interests. This bias is driven by identity, competition, and a sense of threat.
Taleb treats long survival as evidence that a system may withstand disturbances its designers cannot fully explain. He contrasts concentrated, fragile systems with the accumulated experience embodied in older practices and nature, while acknowledging that age is no guarantee.
The Romans developed their political system through experiential tinkering and learning from disasters, rather than through pure reasoning, contrasting with Lycurgus' rational construction of Sparta's laws.
Root cause analysis investigates the contributing causes of a problem and the conditions that allowed it to recur. It combines evidence with causal reasoning to identify and assess corrective actions.
Root cause analysis investigates why a problem occurs. A fishbone, or Ishikawa, diagram organizes possible causes into categories so they can be investigated; the diagram alone does not establish which causes are responsible.
A structured technique for identifying the root cause of a problem by repeatedly asking 'why' until the underlying issue surfaces, preventing superficial solutions.
Root cause analysis investigates why events happened and why their consequences were severe. The Titanic inquiries illustrate how the questions, evidence, time available, and investigators’ responsibilities shape what an inquiry can establish.
Root Cause Analysis (RCA) is a systematic process for identifying the underlying reasons why an event deviated from a plan, distinguishing between immediate, contributing, and root causes. It emphasizes looking beyond individual actions to understand systemic factors, using tools like the '5 whys', bowtie diagrams, and fault trees to prevent recurrence.
Apply root cause analysis to uncover deeper cultural and systemic issues behind common internal audit team problems, such as delays in information provision, management tick-box attitudes, and scope conflicts.
Root cause analysis (RCA) involves gathering relevant facts and evidence, establishing a timeline of events, and verifying causality to identify the true root causes of a problem or disaster.
Good root cause analysis takes time, detailed evidence, and a culture where people can speak openly. The managers quoted here recommend stabilizing a crisis first, returning to the investigation afterward, and examining how work is actually done before choosing improvements.
Root cause analysis can be useful for near misses and recurring minor defects as well as major failures. In high-volume processes, a small defect can accumulate into substantial harm or cost.
Karl Popper argued that genuine philosophical problems arise from outside philosophy, and philosophy degenerates when studied in isolation. This problem-first approach is valuable for practitioners in science, trading, and decision-making.
A heuristic that involves deliberately shifting attention to different features of a situation to gain new insights, especially when natural focus may be misleading, by using methods like reversal, listing features, and systematic rotation.