Judgment Calls. Thomas H. Davenport

headed.

      Part One

      Stories About the Participative Problem–Solving Process

      THE FIRST PART OF THE BOOK begins with a few stories of organizations that showcase the first two themes of the new pattern discussed in the introduction: framing decisions as an iterative problem-solving process, and engaging in a more self-consciously participative approach to getting to a good answer. The segment begins with the tale of NASA, and a difficult launch decision its personnel had to make one time about the (recently retired) space shuttle. We'll see how NASA—an organization known for some bad decisions with tragic consequences in the past—learned from its mistakes and embraced a better approach to organizational judgment. After that we'll focus on an interesting and innovative small business whose leader turned a home-building business challenge into a value-added problem-solving process. And we'll conclude the segment with a case study of the global consulting firm McKinsey & Company—and a major decision its partners made about acquiring and developing their most precious of assets: their people.

      1

      NASA STS-119

      Should We Launch?

      IN FEBRUARY 2009, the engineers and scientists of NASA were wrestling with a grave, potentially life-or-death decision: whether to green-light the launch of mission STS-119, the next flight of the space shuttle Discovery. Every launch of a NASA manned spacecraft puts astronaut lives and millions of dollars of equipment on the line; reputation, political capital, and scientific standing also ride on a successful launch, but catastrophe can undo them all. For every mission, NASA would like the maximum possible certainty, but there are project pressures against endless debate and analysis to unravel every possible concern. Operational schedules are tightly wound and project milestones are critical. The issue here was whether STS-119 might have a faulty valve in the systems supplying fuel to the engines, integral to maintaining pressure in the all-important hydrogen tank. The previous mission (STS-126) had experienced such a problem, which had happily not affected the success of that flight. But NASA engineers do not bet on good luck—and the risk of possible disaster with this next mission was very real. The piece of equipment in question, no longer manufactured, could not be easily replaced—but it was buttressed by some system redundancy with other valves. Should STS-119 be launched? Could the flight readiness review team get to the right “go or no-go decision” with the appropriate level of confidence?¹

      Looming over these critical questions was the history of NASA itself, some fifty years of pioneering scientific triumphs punctuated by a few, but heartbreaking, accidents—where an occasional bad decision led to historic tragedy. How to be sure that this decision didn't become another tragedy? The real story of the launch of STS-119 is not about what finally happened, but about the power of how NASA personnel finally decided what to do—a process of disciplined and iterative decision making, buttressed by a strong but pragmatic culture of inquiry, things NASA developed in the morning-after clarity and learning following some historic and very public errors in judgment.

      Learning from History

      As the whole world knows, the first of those errors in judgment resulted in the fireball in the sky on January 28, 1986, when the space shuttle Challenger exploded in its second minute of flight, killing its entire crew. Despite concerns that cold weather could reduce the effectiveness of the O-ring pressure seals at the joints of the space shuttle's solid rocket motors, NASA managers had approved the launch of Challenger on that day, when the temperature at the Kennedy Space Center was barely above freezing. The spacecraft was destroyed as the failure of an O-ring to seal its joint allowed a jet of hot flame to escape and breach the shuttle's external fuel tank, causing a fatal ignition of the liquid hydrogen and liquid oxygen it contained.

      Like every shuttle launch, the January 1986 mission was preceded by a flight readiness review (an FRR, in NASA acronymspeak), whose purpose was to evaluate issues that might threaten mission success and to withhold launch permission until those issues are resolved. Two weeks earlier, an FRR had certified Challenger ready for flight. Of course, participants in that meeting could not foresee how cold it would be two weeks hence. The day before the launch, NASA personnel became concerned about the weather; the solid rocket motor manager at Marshall Space Flight Center asked Morton Thiokol, the manufacturer of those motors, to review their safety in cold weather. In a series of teleconferences that evening, Thiokol engineers initially recommended against a low-temperature launch. But after their view was challenged by NASA shuttle managers, an offline “caucus” among engineers and managers at Thiokol reversed that recommendation. Challenger lifted off the next morning and was destroyed seventy-three seconds later.

      The presidential commission set up to examine the Challenger disaster found that pressure to maintain the shuttle program's launch schedule led managers to minimize the seriousness of engineers' concerns about the O-rings.² The perceived need for shuttle “productivity” certainly contributed to the error in judgment. Sociologist Diane Vaughn's detailed study of the launch-approval process in The Challenger Launch Decision offers a fuller and more nuanced explanation. Vaughn points to what she calls “the normalization of deviance” as a key factor. Because earlier cold or cool weather flights that suffered O-ring problems did not result in disaster, that initially unexpected damage was gradually accepted as normal. FRR participants had come to view it as an acceptable risk. In other words, the success of nearly two dozen previous missions led to complacency that failed to take the danger seriously enough. The result, says Vaughn, was “an incremental descent into poor judgment.”³

      She also points to problems in the deliberative process during the January 27 teleconferences. NASA managers at Kennedy and Marshall Space Flight Center could not see the engineers at Morton Thiokol who had concerns about the O-rings. They missed the “body language” that could have helped expressed the engineers' unease; they were unaware of the local conversations between calls that might have given them a better grasp of the technical issue. As a result, the level of the engineers' concern was not clearly communicated. In addition, NASA's technical culture tended to discount the engineers' partly intuitive argument about the dangers.

      So the multiple causes of bad judgment in this disastrous case include reluctance to credit “bad news” that would thwart schedule and productivity goals, complacency resulting from a history of success, and ineffective communication.

      Seventeen years later, the only other fatal shuttle accident—the Columbia—occurred. It is unclear whether the crew could have been saved if NASA had understood the damage to Columbia while it was in orbit, but the board that investigated the disaster attributed the agency's failure to try to assess possible damage to many of the same factors behind the Challenger decision, especially the complacency born of many successes and communication failures.

      This second accident and the criticisms in the Columbia Accident Investigation Board report strengthened NASA's resolve to address the cultural as well as the procedural flaws responsible for those fatal errors, and multiple changes to both process and culture were instituted over time.⁴ Looking now at the flight readiness review—for STS-119, the Discovery mission originally scheduled for launch on February 19, 2009, and finally launched nearly a month later, on March 15—will illustrate what the space agency has done to ensure the soundness of its judgments about flight viability and safety.

      The Flight Readiness Review

      Today, the flight readiness reviews held at Kennedy Space Center before a scheduled launch date bring technical teams and managers together in one room, including representatives from three domains: program, engineering, and safety—about 150 people in the case of the STS-119 review. The importance of gathering them in one place, face-to-face, is clear when you contrast such a meeting with the Challenger teleconference, or any teleconference for that matter, where inattention, misunderstanding, and incomplete communication are common.

      The FRR is preceded by a series of smaller team meetings and technical reviews to discuss and analyze issues that will come up in the formal FRR. There are likely to be fifty teams working on specific