The Success Equation. Michael J. Mauboussin

before in similar situations. More formally, Kahneman and Tversky argue that three types of information are relevant to statistical prediction. The first is prior information, or the base rate. For example, if 85 percent of the taxicabs in a city are green, then 85 percent is the base rate. Absent any other information, you can assume that whenever you see a taxicab there's an 85 percent chance that it will be green. The second type of information is the specific evidence about an individual case. The third type of information is the expected accuracy of the prediction, or how precise you expect it to be given the information you have.⁵

      I had a conversation with a doctor that illustrates these three types of information. He mentioned that he had a treatment for improving a specific ailment that succeeded about 50 percent of the time (the base rate). But he added that he could induce almost any patient to undergo the treatment if he simply told them, “The last patient who was treated this way is doing great!” (specific evidence about an individual case). For the patients who were evaluating the treatment, the story of success swamped the statistics.

      The key to statistical prediction is to figure out how much weight you should assign to the base rate and specific case. If the expected accuracy of the prediction is low, you should place most of the weight on the base rate. If the expected accuracy is high, you can rely more on the specific case. In this example, the doctor gave the patient no reason to believe that the procedure had better than a 50/50 chance of working for him. So the patient should place almost no weight on the specific evidence that it worked for one patient, and should rely instead on the base rate in making his decision.

      Here's how the weighting of the base rate and the specific case relate to skill and luck. When skill plays the prime role in determining what happens, you can rely on specific evidence. If you're playing checkers against Marion Tinsley, you can easily predict the winner on the basis of your knowledge of Tinsley's deadly skill. In activities where luck is more important, the base rate should guide your prediction. If you see someone win a million dollars, that doesn't change the odds of winning the lottery. Just because someone wins at roulette, it doesn't help you to guess where the ball will end up on the next spin.

      Unfortunately, we don't usually think this way. When we make predictions, we often fail to recognize the existence of luck, and as a consequence we dwell too much on the specific evidence, especially recent evidence. This also makes it tougher to judge performance. Once something has happened, our natural inclination is to come up with a cause to explain the effect. The problem is that we commonly twist, distort, or ignore the role that luck plays in our successes and failures. Thinking explicitly about how luck influences our lives can help offset that cognitive bias.

      Quantifying Luck's Role in the Success Equation

      The starting place for this book is to go beyond grasping the general idea that luck is important. Then we can begin to figure out the extent to which luck contributes to our achievements, successes, and failures. The ultimate goal is to determine how to deal with luck in making decisions.

      This book has three parts:

       Chapters 1 through 3 set up the foundation. I start with some working definitions of skill and luck, examining the types of interactions where luck is relevant and noting where our methods to sort skill and luck may not work. I then turn to why we have such a difficult time comprehending the influence that luck exerts. The basic challenge is that we love stories and have a yearning to understand the relationship between cause and effect. As a result, statistical reasoning is hard, and we start to view the past as something that was inevitable. The section finishes by looking at the continuum from all-luck to all-skill. I examine a basic model to help guide intuition. These ideas include the paradox of skill and what determines the rate of reversion to the mean.

       Chapters 4 through 7 develop the analytical tools necessary to understand luck and skill. I open with methods for placing activities on the luck-skill continuum. Where an activity falls on that continuum provides a great deal of insight into how to deal with it. I then look at how skill changes over time. Simply put, skill tends to follow an arc: it improves for some time, peaks, and then glides lower. Next, I turn attention to the distributions—or the range of values—of luck. In activities where the results are independent of one another, simple models effectively explain what we see. But when a past result affects a future result, predicting winners becomes very difficult. The most skillful don't always win. I close this part by showing the difference between a useless statistic and a useful one. Useful statistics are persistent (the past correlates highly with the present) and predictive (doing well or poorly correlates strongly with the desired goal). As we will see, many statistics fail this basic test.

       Chapters 8 through 11 offer concrete suggestions about how to take the findings from the first two parts of this book and put them to work. I begin by outlining ways to improve skill. Where little luck is involved, deliberate practice is essential to developing skill. Where luck is rampant, we must think of skill in terms of a process, because the results don't provide clear feedback. Checklists can also be of great value because they improve execution and can guide behavior under stressful circumstances. I then look at how to cope with luck. When you are the favorite, for example, you want to simplify the game so that you can overwhelm your opponent. If you are the underdog, you want to inject luck by making the game more complex. Because luck is in part what remains unexplained, controlled tests allow for a more accurate reading on causality. If you want to know if an advertisement worked, for example, you need to consider the purchasing behavior of those who saw the ad versus those who didn't. This part also includes an in-depth discussion of reversion to the mean, an idea that most people believe they understand, even though their behavior shows that they don't. The book finishes with ten concrete tips on how to overcome the psychological, analytical, and procedural barriers in untangling skill and luck.

      This analysis of skill and luck will focus on business, sports, and investing because these are the areas I know best. Naturally, these realms are quite different. Sports are the easiest activities to analyze because the rules are relatively stable over time and there is lots of data. Other social processes, including business, have fewer rules and boundaries than sports and therefore tend to be more complex. Still, many of the same analytical methods are valid.⁶ Markets in general are the most difficult to analyze because prices are established through the interaction of a large number of individuals. Here again, the nature of the problem may be somewhat different from sports, but many of the tools for sorting out the relative influence of skill and luck still apply.

      Part of the fun and challenge of analyzing skill and luck is that it's a multidisciplinary endeavor. Statisticians, philosophers, psychologists, sociologists, corporate strategists, professors of finance, economists, and sabermetricians (those who apply statistical methods to the study of sports) all have something to contribute to the discussion.⁷ Unfortunately, the people within these disciplines don't always reach outside their fields. You will see ideas from each of these disciplines, and I'm hopeful that bringing them together will lead to a sounder and more balanced approach to analyzing decisions and interpreting the results.

      Untangling skill and luck is an inherently tricky exercise, and there are plenty of limitations, including the quality of the data, the sizes of samples, and the fluidity of the activities under study. The argument here is not that you can precisely measure the contributions of skill and luck to any success or failure. But if you take concrete steps toward attempting to measure those relative contributions, you will make better decisions than people who think improperly about those issues or who don't think about them at all. That will give you an enormous advantage over them. Some statisticians, especially in the world of sports, come across as know-it-alls who are out of touch with the human side of things. This characterization is unfair. Statisticians who are serious about their craft are acutely aware of the limitations of analysis. Knowing what you can know and knowing what you can't know are both essential ingredients of deciding well. Not everything that matters can be measured, and not everything that can be measured matters.

      While there are wide swaths of human activity where the ideas in this book are hard to apply, the ideas have concrete application in some important areas and should serve as a template for thinking about decisions beyond the scope of this book. Luck may explain that you met your future wife after your buddy lured you