Outcomes are feedback

Outcomes are feedback

We can’t just “absorb” experiences and expect to learn. As novelist and philosopher Aldous Huxley recognized, “Experience is not what happens to a man; it is what a man does with what happens to him.” There is a big difference between getting experience and becoming an expert. That difference lies in the ability to identify when the outcomes of our decisions have something to teach us and what that lesson might be.

Any decision, whether it’s putting $2 on Count de Change at the racetrack or telling your kids they can eat whatever they want, is a bet on what will likely create the most favorable future for us. The future we have bet on unfolds as a series of outcomes. We bet on staying up late to watch the end of a football game and we sleep through our alarm, wake up tired, get to work late, and get reprimanded by the boss. Or we stay up late and any of the myriad other outcomes follows, including waking up perfectly on time and making it to work early. Whichever future actually unfolds, when we decide to stay up late to see the end of the game, we are making a bet that we will be happier in the future for having seen the final play. We bet on moving to Des Moines and we find our dream job, meet the love of our life, and take up yoga. Or, like poker player John Hennigan, we move there, hate it within two days, and have to buy our way home for $15,000. We bet on firing a division president or calling a pass play, and the future unfolds as it does. We can represent this like so:

As the future unfolds into a set of outcomes, we are faced with another decision: Why did something happen the way it did?

How we figure out what—if anything—we should learn from an outcome becomes another bet. As outcomes come our way, figuring out whether those outcomes were caused mainly by luck or whether they were the predictable result of particular decisions we made is a bet of great consequence. If we determine our decisions drove the outcome, we can feed the data we get following those decisions back into belief formation and updating, creating a learning loop:

We have the opportunity to learn from the way the future unfolds to improve our beliefs and decisions going forward. Interpreting relevant evidence from experience can reduce uncertainty about our beliefs and choices. Using outcomes to examine our beliefs and bets closes the feedback loop. This is the heavy lifting of how we learn.

Ideally, our beliefs and our bets improve with time as we learn from experience. Ideally, the more information we have, the better we get at making decisions about which possible future to bet on. Ideally, as we learn from experience we get better at assessing the likelihood of a particular outcome given any decision, making our predictions about the future more accurate. As you may have guessed, when it comes to how we process experience, “ideally” doesn’t always apply.

Learning might proceed in a more ideal way if life were more like chess than poker. The connection between outcome quality and decision quality would be clearer because there would be less uncertainty. The challenge is that any single outcome can happen for multiple reasons. The unfolding future is a big data dump that we have to sort and interpret. And the world doesn’t connect the dots for us between outcomes and causes.

If a patient comes into a doctor’s office with a cough, the doctor must work backward from that one symptom, that one outcome of a possible disease process, to decide among the multiple reasons the patient might have that cough. Is it because of a virus? Bacteria? Cancer? A neurological disorder? Because a cough looks roughly the same whether it is from cancer or a virus, working backward from the symptom to the cause is difficult. The stakes are high. Misdiagnose the cause, and the patient might die. That is why doctors require years of training to properly diagnose patients.

When the future coughs on us, it is hard to tell why.

Imagine calls to a customer by two salespeople from the same company. In January, Joe pitches the company’s products and gets $1,000 in orders. In August, Jane calls on the same customer and gets $10,000 in orders. What gives? Was it because Jane is a better salesperson than Joe? Or was it because the company updated its product line in February? Did a low-cost competitor go out of business in April? Or is the difference in their success due to any of a variety of other unconsidered reasons? It’s hard to know why because we can’t go back in time and run the controlled experiment where Joe and Jane switch places. And the way the company sorts this outcome can affect decisions on training, pricing, and product development.

This problem is top of mind for poker players. Most poker hands end in a cloud of incomplete information: one player bets, no one calls the bet, the bettor is awarded the pot, and no one is required to reveal their hidden cards. After those hands, the players are left guessing why they won or lost the hand. Did the winner have a superior hand? Did the loser fold the best hand? Could the player who won the hand have made more money if they chose a different line of play? Could the player who lost have made the winner forfeit if they chose to play the hand differently? In answering these questions, none of the players knows what cards their opponents actually held, or how the players would have reacted to a different sequence of betting decisions. How poker players adjust their play from experience determines their future results. How they fill in all those blanks is a vitally important bet on whether they get better at the game.

We are good at identifying the “-ER” goals we want to pursue (better, smarter, richer, healthier, whatever). But we fall short in achieving our “-ER” because of the difficulty in executing all the little decisions along the way to our goals. The bets we make on when and how to close the feedback loop are part of the execution, all those in-the-moment decisions about whether something is a learning opportunity. To reach our long-term goals, we have to improve at sorting out when the unfolding future has something to teach us, when to close the feedback loop.

And the first step to doing this well is in recognizing that things sometimes happen because of the other form of uncertainty: luck.