26 December 2008 by Sanjay Saigal.

The BBC recently reported on the long-simmering power struggle in Christiandom’s holiest site: the Church of the Holy Sepulchre in Jerusalem. This site – by tradition the place of Jesus Christ’s burial and resurrection – is contested by six denominations that occupy tiny bits of it: Roman Catholic, Greek Orthodox, Armenian Orthodox, Syrian Orthodox, Egyptian Copt and Ethiopian Orthodox. Each faction constantly maneuvers to improve its territory, their shenanigans (real and perceived) often boiling over in eruptions of violence.

The church itself is in a state of near-collapse. But since its ownership is in dispute, all attempts at repair are stymied. The situation is dismally Pareto optimal: any effort at improving matters makes at least one other party unhappy. The following heartrending yet funny anecdote captures the hopelessness of the situation:

The intractable nature of the territorial arguments over the site are epitomised by the short wooden ladder that rests on a ledge above the church’s main entrance.

It has been there since the 19th Century because rival groups cannot agree who has the right to take it down.

Under the Status Quo agreement, rights to the windows reached by the ladder belong to the Armenians, but the ledge below is controlled by the Greeks.

So the messiness persists, despite its ongoing cost to the Christian community. (Not to mention the potential catastrophic downside – complete collapse of the complex’s badly deteriorating roof.)

I was reminded of the Church of the Holy Sepulchre while navigating the recent brouhaha in the Analytics blogonook created by a post in the Intelligent Enterprise blog. Doug Henschen quizzes IBM’s Ambuj Goyal on Big Blue’s “analytics strategy” following its recent acquisition of Ilog, a leader in decision technology components (and my past employer). Henschen’s didactic frame is the notional question: “Will IBM Add Analytics to its Toolbelt?”

Henschen summarizes the interview’s takeaway in his lede as “[IBM contends that] predictive and statistical modeling — key offerings for the likes of SAS and SPSS — are overrated. IBM has what Goyal describes as better, cheaper alternatives in a mix of techniques developed for industry- and domain-specific challenges.” This startling conclusion has been met with (I’ll use a polite word) skepticism by Analytics-oriented blogs. James Taylor at the EDM blog smells the ghost of sales campaigns past:

Sadly this reminded me of the old days of IBM - when FUD (fear, uncertainty and doubt) was IBM’s reponse (sic!) to anything they did not do well. Predictive analytics are not overrated, at least not by anyone who understands them. It is true that predictive analytics, like all good technologies, are sometimes overused by over-enthusiastic supporters and that they can’t do everything. IBM’s lack of this technology is a mistake as without it their solution set is incomplete and no amount of FUD will change that.

Anne Milley at SAS’s company blog SAScom is predictably indignant in light of Goyal’s presumed attack on SAS’s knitting. Milley labels Goyal’s comments “dizzying spin” and suggests that IBM execs “deride the value [of Analytics] because they haven’t been able to monetize the analytics in their research labs even as others achieve significant returns”.

The BBC report came to mind because, as in Jerusalem, the disagreement occasioned by Goyal’s interview is fundamentally doctrinal. It centers on the existential (and ungrammatical) question – what is Analytics?

I have previously mentioned Tom Davenport’s HBR article called “Competing on Analytics”. (Davenport has also published a subsequent book with this title.) Davenport never quite defines Analytics, but his view is obviously expansive; his keystone success story is Marriott Corporation, whose Total Hotel Optimization system relies, at its core on Linear Programming (i.e., Optimization). Davenport sows confusion by using interchangeably using Analytics and labels like “statistical masters”. Analytics is not Statistics. Statistics is a tool in the arsenal of the Analytics professional. But it doesn’t describe the category.  

The SAS Institute has adopted the branding framework of Analytics to compete in the Decision Management space. Historically the purveyor of a statistical toolkit (which later morphed into a Data Warehouse platform,) SAS had weak or non-existent offerings in Optimization and Inferencing/Rules, and came late to the Business Intelligence and Predictive Analytics (BI/PA) wave. So its stance of statistical methods as somehow defining Analytics is natural.

Conversely, prior to the Cognos acquisition, IBM had negligible footprint in the BI/PA space. In terms of Optimization, while boasting incredibly qualified R&D and Consulting groups, it has been unable to develop a profitable software offering. (In the ‘80s IBM attempted to market an optimization toolkit called OSL, made many sub-optimal decisions along the way, and finally killed the product in the late ‘90s. More recently, IBM has tried to leverage an open source toolkit called COIN-OR as a brand-builder, in my opinion essentially futilely.) The acquisition of Ilog provides IBM immediately with best-of-breed offerings in Optimization and Inferencing (or Business Rules). However, it still lacks anything like SAS’s statistical platform. Goyal’s pitch suggests that they know it. And they are trying to advance the notion that what they have is what the market needs.

As is not uncommon, differences born of necessity are being painted in Marketing’s primary colors to manipulate each vendor’s “rightness”. While this makes perfect sense to advance each party’s immediate business imperatives, it does not help the cause of Analytics in the Enterprise. James Taylor correctly observes that:

The important thing is to focus on the decision and then figure out how to solve it… Business rules, optimization, data mining, predictive analytics and adaptive control [are all] necessary ingredients for Enterprise Decision Management and business success.

Indeed.  Let me illustrate with an Analytics success story. In a highly successful Space and Assortment Planning project at a greeting card manufacturer, most of the techniques in Taylor’s list were invoked. SAS was used to derive the space elasticity curves that drove revenue forecasts, CPLEX was used to optimized a complex sequence of layout models, and JRules was used to manage scenarios, preferences and exceptions. As the then Ilog project lead, I cannot recall a discussion where we, or any technology partisan on the large project team, attempted to push a technology agenda. The team uniformly focused on creating the fastest-performing, most accurate and most efficiently usable application.

Along similarly cooperative lines, instead of advocating the relative importance of one’s own bag of tricks and denigrating the competition’s, actors in the Analytics ecosystem need to adopt Davenport’s catholic approach. Let’s focus on expanding the impact of the field, on making it as pervasive a function in business as Accounting or Human Resources. To mash up the rallying cry of Bill Clinton’s 1992 campaign with the verbal stylings of the logorrheic Sarah Palin, “It’s all of the above, stupid”!

Posted in Business Intelligence, Operations Research, Vision, Intechné | 1 Comment »

19 December 2008 by Sanjay Saigal.

As a member of the advisory council for the upcoming INFORMS Practice Meeting in Phoenix, I am assembling what I hope will be a boffo slate of speakers for the track on Managing Risk & Uncertainty. Researching recent work in the area, I encountered an excellent 1997 paper by Dick Barr and Tom Siems titled Bank Failure Prediction Using DEA to Measure Management Quality. Early warning indicators (called Key Risk Indicators, KRIs, in the Risk Management community) of bank failure include one that is difficult to directly extract from balance sheets: Management Quality. Barr and Siems used Data Envelopment Analysis, or DEA (a linear programming-based efficiency measure, look here for a tutorial) to identify an analytically meaningful surrogate for Management Quality. The resulting multi-factorial risk model was remarkably predictive: it could correctly label a bank as strong or an incipient failure with 96% accuracy, a year to 18 months out.

I wonder whether the early warning system was used. Siems is listed as employed by the Federal Reserve Bank of Dallas, a regulatory body. So there appears a prima facie opportunity to apply the model. On the other hand, I would not be surprised to hear that the paper’s publication was its terminal “development milestone”.

Speaking later with Doug Smith, another financial risk estimation guru, I commiserated how often models such as Barr & Siems are left on the shelf. As Doug characterized the unfortunate imperatives facing his technical collaborators employed in finance: “The pressure to create profits meant that the results of risk models were ignored”.

Doug’s lament brought to mind a frequent problem for the analytics practitioner: the unfortunate habit of our “customers” (whether line managers within our own organizations or external consulting clients) to cherry-pick which analysis to use, and which to ignore. The situation becomes especially tricky when analytical findings collide with political winds. The latest such example comes from the British Isles, where a pay-as-you-go congestion charge system pushed by Newcastle University researchers was deep-sixed by Manchester voters. Feasibility, it turns out, is in the eyes of the customer. In this case, the citizenry decided that the traffic smoothing benefits of the congestion charge were trumped by the nefariousness of the new “tax”. Never mind the value of time!

While there are good estimates for project acceptance/failure (e.g., here) I have not come across estimates on how often analytics projects fail, or are shelved, for extraneous reasons. Do you have data, or even stories, to share?

Posted in Risk | 3 Comments »

12 December 2008 by Sanjay Saigal.

In recent weeks I have been working with Sam Savage, well-known OR personality and a consulting professor at Stanford. We’re focusing on developing a practice framework for Probability Management. Whazzat, you ask? In sum, Probability Management is all about robust decision-making in the presence of uncertainty. (Pretty much the vision for Intechné!)

Since real world decision problems are almost always ill-structured and fuzzy, our tools of choice belong to the worlds of simulation, and statistical visualization. Stochastic optimization plays a role too, but in a very different form than typically understood, say, in Operations Research circles. In general, we are not interested in creating IT systems that generate “best possible” recommendations. Rather, we enable managers to interactively explore the decision space of good solutions, using something similar to a business intelligence (BI) approach. The key difference between BI and Probability Management is that while BI is essentially descriptive (identifying multi-factorial relationships, typically for historical data) Probability Management is prescriptive: our clients learn what to do better.

I intend to write further on this topic, but for now let me point interested readers to our newly redesigned web site. The organization is a loose consortium of academic and commercial folks involved in the field, as vendors, users, and advisors. Check out the Interact! tab. It contains illustrative Excel models that describe the relevant concepts far better than long-winded descriptions. If you find it interesting and wish to discuss further, contact me.

Posted in Probability Management, Business Intelligence, Intechné, Risk | 1 Comment »