Skip to main content
Mental Models

Finding Quality Inputs

A repeatable testing process lets you compare candidate inputs and refine them from the results. It can organize a search even when you do not know which inputs will work.

THE IDEA

Finding Quality Inputs

An algorithm takes inputs through a defined process to produce outputs. When you do not know which inputs will give the result you want, a repeatable process can help you compare candidates and refine the search. You can consider it “algorithmic thinking.” The whole system need not be automated for consistent testing to be useful.

In the late 1920s, one company developed a repeatable process to try to create the world’s first broad-spectrum antibiotic. After World War I, scientists had a good understanding of bacterial infection. They were able to identify some of the primary bacteria, such as streptococcus, that caused incurable infections. They also understood how and why bacterial infection often occurred, such as from exposure to contaminated tools and instruments. Treating an infection inside the body remained a major challenge. What was missing was an understanding of bacteria—how it worked and where it was vulnerable.

Bayer, a giant German pharmaceutical company whose origins lay in dye-making, decided there was money to be made if they could find a cure for bacterial infections inside the body. There was some indication that a substance with bacteria-fighting properties could be created; earlier research had produced a treatment for syphilis called salvarsan, but nothing else had been found in the subsequent fifteen years.[24]

In charge of pharmaceutical research for Bayer was Heinrich Hörlein. He thought the research to find bacteria-killing drugs was lacking scale and therefore too much was dependent on individual scientists. So at Bayer, he created an industrial system to identify possible antibacterial compounds and hired dozens of people to put each antibiotic candidate through the same algorithmic-like process.

In The Demon under the Microscope, Thomas Hager explains that Hörlein knew the search would take years but also knew that success would result in enormous profits. Thus he aimed “to expand drug research from the lab of a single scientist to an efficiently organized industrial process with carefully chosen specialists guided by a coordinated strategy.” Hörlein hired Gerhard Domagk to run the “recipe,” putting each compound created by the chemists through an identical testing and evaluation process to see if the result would be an antibiotic that was safe for humans.[25]

Domagk and his team tested the chemicals given to them by Bayer’s chemists. One of the most prolific chemists was Josef Klarer. He produced hundreds of new chemicals that were systematically tested by Domagk and his assistants. Each chemical compound was tested against a panel of “the most common and deadly bacteria: tuberculosis, pneumonia, Staphylococcus, E. coli, and Streptococcus pyogenes.” After a bit of initial refining, Hörlein and Domagk created “a smooth-functioning, reliable machine for discovery.” The chemicals were tested both in test tubes and in living animals. In the animals, each chemical was “delivered three different ways (intravenously, subcutaneously, and by mouth).” Every chemical was tested the same way in mice, and meticulous records were kept of each test.[26]

Time went by. Thousands of mice died. But the researchers did not give up on their process. As the years went on, “despite the repeated negative results, Domagk changed neither his methods nor his approach.”[27] The team continued using the standardized protocol, allowing comparisons across candidates despite the negative results.

In the fall of 1932, their methodology and patience paid off. Klarer decided to attach sulfur to an azo compound. Chemical KL 695 was put through the testing process that thousands of other chemicals had been put through in the previous years. For the first time, the process produced the desired result: mice that recovered from bacterial infection with no apparent toxicity. Domagk didn’t yet know how it worked, only that it did. “Strangely, it did not kill strep in a test tube, only in living animals. And it worked only on strep, none of the other disease-causing bacteria. But, given the number and deadliness of strep diseases, it worked where it counted.” Funny enough, Domagk was on vacation during the first round of testing of KL 695 and so missed witnessing the initial breakthrough.[28] But the process by then was so entrenched, any one of the dozens of people on the team could run it.

The discovery of chemical KL 695 allowed the team at Bayer to refine the inputs they used for their testing algorithm. “Klarer now made variations on KL 695, finding that as long as sulfa was attached to the azo-dye frame in the correct position, the drug worked against strep. Attaching sulfa to an azo dye—any azo dye—somehow transformed it from an erratic, ineffective chemical into an efficient anti-strep medication.”[29] They kept refining their inputs so that more effective azo-sulfa compounds were discovered, including KL 730.

Later research identified sulfanilamide as the active antibacterial component. It resembles PABA, which bacteria use in folate synthesis, and interferes with that pathway, inhibiting bacterial growth. Sulfa is cheap and widely available, so once Bayer’s sulfa antibiotic was on the market, many companies began to make their own.[30]

Bayer’s algorithmic-like approach that led to the discovery of the antibiotic properties of sulfa had far-reaching effects. “Sulfa also changed the way drug research was done. Before sulfa, small laboratories followed investigators’ hunches and patent-medicine makers cobbled together remedies without testing the results. After sulfa, industrial-scale chemical investigation guided by specific therapeutic goals—the system for finding new medicines pioneered by Hörlein and his Bayer team—became the standard. Successful drugmakers were those who followed the Bayer model.”[31] Bayer continued to discover many useful antibiotics using a system that codified the process as much as possible.

A repeatable process makes candidates easier to compare and gives the next round of testing something to build on. Its value depends on whether the tests measure what matters and whether the results guide a useful refinement of the search.

When to use it

  • A researcher screening compound libraries to identify a new drug lead

  • A product manager running usability tests to find the most intuitive interface layout

  • A chef experimenting with ingredient substitutions to perfect a recipe

  • A recruitment team standardizing interview rubrics to identify the best candidates

FROM THEORY TO EVERYDAY LIFE

In practice

01

Bayer's Systematic Search for Antibiotics

In the late 1920s, Bayer created an industrial-scale process to test thousands of chemical compounds for antibiotic activity. Each compound was evaluated using an identical protocol—tested against a standard panel of deadly bacteria both in vitro and in mice via three routes of administration. After years of negative results, chemical KL 695 (a sulfa-containing azo dye) cured mice of streptococcal infections. This success allowed the team to refine their inputs, leading to the discovery of potent sulfa drugs. The approach revolutionized pharmaceutical R&D, becoming the standard for drug discovery.

02

A/B Testing Marketing Campaigns

A marketing team tests email subject lines by assigning audience members randomly to different versions while keeping the rest of each email the same. It compares open rates and tests promising variations in later rounds. The process provides evidence about subject lines for that audience, with each result informing the next comparison.

✳

A MOMENT TO REFLECT

Put it to work.

  1. In what area of your life or work could you design a repeatable testing process to discover the best inputs, even if you don't know them now?

  2. How can you ensure that your process for evaluating options is consistent enough to yield reliable comparisons?

  3. When have you persisted with a systematic approach despite repeated failures, and what did that teach you about the value of a good algorithm over immediate results?

References

  1. The Demon under the Microscope — Thomas Hager (2006)

CONTINUE THE CONVERSATION

Source texts

  • The Great Mental Models Volume 3 Systems and Mathematics
← More mental models