16 January 2009 by Sanjay Saigal.

In an article on how business schools are responding to globalization, Robert Dolan, the profitability management guru, recounts his introduction to the chasm between theory and practice:

[Dolan…] tells a funny story about when he, a freshly minted PhD in Operations Research, was assigned to teach a course in marketing at the Chicago School of Business.

He didn’t know much about marketing , so he decided he would visit the offices of Proctor & Gamble to find out what it was all about: “I put on my suit and was making my way out of the campus when the dean spotted me and asked me where I was going, all dressed up.

When I told him I was going to Proctor & Gamble to learn about marketing, he asked me, ‘Do they have any Nobel Laureates in Proctor & Gamble?’ When I said ‘I don’t think so,’ he said, ‘Well, we’ve got dozens of Nobel Laureates. So you just stay right here’”.

I thought of Dolan’s (apocryphal?) dean while chatting about possibility of professional certification in OR with colleagues. Though certification is often viewed as a mechanism to help consumers distinguish quality practitioners from the riff-raff, in my view it could meet entirely different goals:

1. Bridge the yawning gap between technique-centered OR programs and the impact-oriented dictates of practice: As I recently observed while bemoaning the lack of good Management Science texts, the academic bent that favors rigor over utility inevitably creeps into pedagogy. Irrespective of the mainstreaming of OR, this sub-optimality is unlikely to change.

2. Position OR to “own” the strengthening brand of Advanced Analytics: An analyst certification that allows (among other things) applicants from other backgrounds to enter the profession could make OR the de facto credentialing authority in a crowded and contested field.

3. Certification could revitalize the OR ecosystem: A new practice-oriented market for course materials, coaching apparatus and examination aids would be needed. The primary beneficiary of such a development would most likely be forward-looking academics.

Posted in Operations Research, Vision | 1 Comment »

9 January 2009 by Sanjay Saigal.

Joe Nocera opens his recent NYT Magazine article on the failure of Financial Risk Management by referring to “a persistent tension between those who assert that the best decisions are based on quantification and numbers, determined by the patterns of the past, and those who base their decisions on more subjective degrees of belief about the uncertain future.” Like many of the statements in the long (7600 word), this will give pause to anyone who makes a living in Analytics. While it is true that in Finance, quantitative vs. “soft” approaches inspire quasi-religious passions, mainstream enterprises have long incorporated quantitative analysis as an indispensible tool. Note that word – tool. Analytic methods typically help along decision-making, they don’t replace human judgment.

Consider forecasting. It’s an inevitable function in a manufacturing or service enterprise. Forecasting systems typically include a statistical core – time-series trend analysis, regression-based “causal” models, or combinations thereof. But even in the CPG sector, where large volumes allow statistical forecasting to shine, demand planners inevitably perfect the forecast by injecting intelligence that the most advanced technology cannot replicate. For instance, most forecasts systems do not automatically incorporate adverse weather, or holiday-induced long weekends, or competitor actions. In fact, in my observation the more sophisticated the statistical forecasting system, the more sophisticated the planning staff’s analytic role.

But I digress. My point is simple – with experience, the conflict between Quants and traditional analysts (Quals?) becomes more notional than actual. I bring it up, however, to point out the serious lag between actual practice, and its reporting. But perhaps the schism has a more insidious side-effect.

Nocera’s article fingers the risk measure known as Value at Risk (VAR) as “what led to the financial meltdown”. (A good technical introduction to VAR is available here.) In a nutshell, for any financial instrument, VAR measures the maximum expected loss over a given time horizon. Since loss is uncertain, a probability or comfort factor is also involved. Institutions typically look for 99% or 95% certainty, depending on application. A number of “experts” are quoted denouncing or defending the role of VAR (including  innocent electrons sacrificed to Nassim Taleb’s predictable fire-bombing of practically all quantitative methods; this slow-unfolding but inevitable crisis is hardly a Black Swan, but to a man with a nuclear finger, all battles look like WWII).

Nocera’s indictment is unveiled as he recounts the gradual institutionalization of VAR:

• Instead of one of many risk indicators, VAR was treated as the risk measure. Speaking of investors and CEOs of financial institutions, Nocera says: ”In the bubble, with easy profits being made and risk having been transformed into mathematical conceit, the real meaning of risk had been forgotten. Instead of scrutinizing VaR for signs of impending trouble, they took comfort in a number and doubled down, putting more money at risk in the expectation of bigger gains. “It has to do with the human condition,” said one former risk manager. “People like to have one number they can believe in.”

  The tendency to rely on point estimates is a frequent cause of the so- called Flaw of Averages, which Probability Management is designed to combat. That the estimate is produced by sophisticated analytical computation doesn’t suffice. Understanding risk requires one to look at multiple alternative futures. That is best captured by a probability distribution (or a histogram, which is a granulated distribution). Or, minimally, by a ranking of scenarios.

• VAR can be non-additive over time: VAR computations are often based on the market at calculation time. But they aren’t updated as market conditions change, possibly dramatically in volatile times. When it comes to making new investments, that’s not a problem. But added to existing investments evaluated under different regimes, VAR may seriously misstate portfolio risk.
  
  
• VAR is super-additive over the market: It’s hard to understand how letting each institution rely on its own VAR measure could have sounded like a bright idea to Clinton-era regulators: ”The Securities and Exchange Commission, for instance, worried about the amount of risk that derivatives posed to the system, mandated that financial firms would have to disclose that risk to investors, and VaR became the de facto measure. If the VaR number increased from year to year in a company’s annual report, it meant the firm was taking more risk. Rather than doing anything to limit the growth of derivatives, the agency concluded that disclosure, via VaR, was sufficient.”

  Even if each market participant accurately managed its own risk, it’s stunningly obvious that given the proliferation of incestuous instruments such as credit swaps, the risk to the market was many orders of magnitude higher than any summation of individual risks.

Rereading the article, I am astounded by how many basic principles of quantitative analysis were violated by supposedly sophisticated actors. Perhaps it does come down to the Quant/non-Quant disconnect that opens the article: executives and regulators usually come through the latter door.

Posted in Intechné | 1 Comment »

2 January 2009 by Sanjay Saigal.

Weighing the opportunity to teach Analytic Decision-Making to executive MBA students, I realized that the value in such a project is essentially internal. The financial compensation, to not put too fine a point on it, was low. But the chance to explore the tools of my profession with highly motivated and skeptical, yet captive, mid-career professionals was too attractive to pass up.

I’m teaching something called a “breadth” course. Not quite a part of the core curriculum, but necessary to fulfill requirements of the specialization, e.g., Technology Management or Business Analytics & Technologies. The catalog describes breadth courses as specially focused on relevance. That led to my first surprise: the course description includes Optimization and Decision Analysis (DA), but no mention of Simulation or Queuing! (Forecasting is covered in another breadth course, and further in an elective. Data Mining and Business Intelligence are covered in an entirely separate data management elective, as if they were not part of the same Analytics continuum.)

The absence of Simulation in the syllabus initially struck me as astonishing. In the ill-structured fuzz of real-world business decision-making, Monte Carlo Simulation is the inescapable Swiss Army knife. However, as I began evaluating alternative texts, I had an epiphany! Textbooks purporting to teach quantitative decision-making to business students are typically adaptations of texts originally designed to teach Operations Research (OR) to engineering and other technical majors. For instance, a widely-used text has an entire chapter dedicated to Minimum Spanning Trees, an artifact I have not had to explain to a non-technical client in seventeen years! Authors also persist in continuing to sacrifice thickets, if not entire forests, explaining theoretical notions such as duality and sensitivity to students who would much rather be off discussing consumer product strategies!

After rapidly filtering through a number of OR-centric treatments, my choice reduced to two. The text I did not choose, though only by a hair, is Data, Models, and Decisions, by Bertsemas and Freund (both at MIT). I was nearly swayed by the authors’ smarts in simplifying very complex material, and a near-perfect pedagogic progression – starting with DA, proceeding through probability and statistics to Simulation and Regression, and finally to Optimization. The exposition is case-oriented and jargon-avoiding. However, its immersive use of mathematics raises the possibility that a treatment designed for the very technical Sloan School students might not work as well for more diverse audiences.

The text I chose is Decision Making with Insight, by Sam Savage. For business school use, this text has three key advantages: First, its DNA is that of practical business decision-making, not OR technology. Its spare style makes it practically useful as a cookbook (after the class is over) in the vein of the Numerical Recipes series. Second, Savage – a thought leader in spreadsheet-based Analytics – almost exclusively uses Excel as his computational sandbox. He reasons that while spreadsheets have easily-reached limitations in analytic use, they are the business analyst’s tool of choice, which guarantees a faster uptake than an unnatural specialized environment like, say, Lindo. Finally, the exposition is playful and exploratory. It elevates insight derived by computational experimentation over traditional mathematical development. The business school classroom requires business engineering, not business mathematics.

The main weakness of Savage’s book relates to its Excel-centrism. The book is designed to turn the reader into a self-motivated desktop analyst. But Savage doesn’t even hint at the vast and valuable world of specialized analytical tools and applications that power large-scale systems such as airline revenue management to power plant operations. Since there are few pointers on the possibilities and limits of Analytics, the book elides the equally important role of the reader as a potential consumer of hardcore Enterprise-level OR technologies. (Bertsimas and Freund do a better job in this respect.) Fortunately, given my background in large-scale system development, supplementing the text should be straightforward. So this lacuna does not concern me.

Tune in again after Q1 of 2009 for a postmortem on my choice!

Posted in Teaching, Business Intelligence, Operations Research | 1 Comment »