Remembering the Master of All Trades

Exiting graduate school in ‘91, I interviewed with the Thinking Machines Corporation (TMC). It was a job for which I was woefully unqualified. No surprise, I didn’t get the chance to find out. And anyway, in two years, the company had gone under, victimized by being ahead of its time. But for the ten years it was in existence, TMC burned a spectacular trail through applied computation science. Inarguably its brightest light was the physicist Richard Feynman. (TMC founder Daniel Hillis affectionately recounted their collaboration in Physics Today, and you can read it here.)

Feynman’s legend as a Javert-like pursuer of enlightenment is well-established. (See, for instance, the Feynman tribute site.) Sadly, set next to his spectacular Physics work, Feynman’s contributions to analytic decision-making are less well-known. Hillis’ article described some of Feynman’s pioneering work on the numerics of the Connection Machine. But more interesting to me is Feynman’s work on the Challenger disaster commission.
Alone among the “wise men” on the commission, Feynman went to the source – interviewing the shuttle engineers whose 1 in 100 failure estimate magically inflated to 1 in 100,000 in the hands of NASA management! His appendix to the final report is worth reading as a primer on (desirable) structured design and (self-deluding) risk management. For instance, here is Feynman talking about how shuttle uniquely engines were built:

The usual way that such engines are designed (for military or civilian aircraft) may be called the component system, or bottom-up design. First it is necessary to thoroughly understand the properties and limitations of the materials to be used (for turbine blades, for example), and tests are begun in experimental rigs to determine those. With this knowledge larger component parts (such as bearings) are designed and tested individually. As deficiencies and design errors are noted they are corrected and verified with further testing. Since one tests only parts at a time these tests and modifications are not overly expensive. Finally one works up to the final design of the entire engine, to the necessary specifications. There is a good chance, by this time that the engine will generally succeed, or that any failures are easily isolated and analyzed because the failure modes, limitations of materials, etc., are so well understood. There is a very good chance that the modifications to the engine to get around the final difficulties are not very hard to make, for most of the serious problems have already been discovered and dealt with in the earlier, less expensive, stages of the process.

The Space Shuttle Main Engine was handled in a different manner, top down, we might say. The engine was designed and put together all at once with relatively little detailed preliminary study of the material and components. Then when troubles are found in the bearings, turbine blades, coolant pipes, etc., it is more expensive and difficult to discover the causes and make changes. For example, cracks have been found in the turbine blades of the high pressure oxygen turbopump. Are they caused by flaws in the material, the effect of the oxygen atmosphere on the properties of the material, the thermal stresses of startup or shutdown, the vibration and stresses of steady running, or mainly at some resonance at certain speeds, etc.? How long can we run from crack initiation to crack failure, and how does this depend on power level? Using the completed engine as a test bed to resolve such questions is extremely expensive. One does not wish to lose an entire engine in order to find out where and how failure occurs. Yet, an accurate knowledge of this information is essential to acquire a confidence in the engine reliability in use. Without detailed understanding, confidence can not be attained.

A bottom-up approach is equally essential for building robust and sustainable Analytics-driven decision-support systems. Without the benefit of well-tested atomic models (e.g., the peculiar demand for fundamentally different types of products), building a large-scale model ab initio (in this instance, a manufacturing company’s demand forecast) invites shelfware-hood. I have seen exactly this sort of failure occur at a multi-billion dollar world-wide ERP roll-out, where the demand planning model template built by HQ analysts was so inapplicable to specific business units that local planners took to over-writing official forecasts produced by the ERP with their own hand-computed numbers!

As I’ve previously described (e.g., here), the practice of Analytics inevitably occurs in the shadow of business imperatives. Business goals give shape to the analysis. But they also have the power to corrupt the process. Feynman pungently captures that corruption at NASA, and its consequences in his final words:

For a successful technology, reality must take precedence over public relations, for nature cannot be fooled.

One wonders about Feynman’s analysis of the current financial crises, which appears at least as much a product of self-delusion on high as the Challenger disaster!

This entry was posted on 4 February 2009 at 19:13 and is filed under Methodology, Operations Research, Risk. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response or trackback from your own site.