26 April 2008 by Sanjay Saigal.

Over the past few years, as IT has grown to become a key enabler across business functions, software engineering practice has focused on implementation methodology. Frameworks such as Agile Programming, Rational Unified Process (RUP), and more recently, Eclipse Process Framework Project (EPF), promise to improve IT implementation by distilling industry best practices into usable tools and artifacts. The ultimate goal for these methodology frameworks is to industrialize IT project work, to make it scalable, repeatable and predictable. There is some evidence of success: the Standish Group estimates that over the 1994-2004 period, US IT project success rates more than doubled, to where one in three of all projects are deemed successful. They explicitly credit new IT methodologies: “Doing projects with iterative processing as opposed to the waterfall method, which called for all project requirements to be defined up front, is a major step forward.”

Figure 1: Standish Group 2006 US IT Survey

Note that even armed with full-featured methodologies such as RUP, IT projects are high-risk. As Figure 1 (above) shows, though the US success rate was up to 35% by 2006, one in five project still failed outright. The remaining 46% are condemned to the purgatory of “late, over budget, or below requirements”, presumably enroute to shelfware status.

In Competing on Analytics, a 2006 Harvard Business Review article (available here in research report form, and here as a teaching aid) Tom Davenport argues for a cross-Enterprise approach to integrating Operations Research (OR, also sometimes referred to as Advanced Analytics, Management Science or Quantitative Analysis) into a company’s decision-making. As organizational decision-making becomes increasing IT-based under competitive pressures, more tactical and operational decision-support systems (DSS) have cores built on integer programming, advanced forecasting, Monte Carlo simulation, ant colony optimization, and similarly complex techniques. Moving from piston aircraft to turbines to jets to rockets requires increasingly complex power plant engineering and operating regimes. Similarly, going from simpler rule-based systems that automate decision trees to optimization-driven systems that implement OR technology require a re-examination of IT development processes.

As Davenport documents with examples from companies such as Marriott and Procter & Gamble, OR can maximize value extraction from a company’s resource base. But it also throws up new risks and challenges in implementation. For instance, the major challenges in implementing a database query-driven system (for instance, a standard CRM) involve business process mapping, user adoption, server and bandwidth sizing, etc. However, turbo-charge that application with an OR engine (for example, by embedding a constraint programming-based product configurator,) and you now need to worry about reliable search performance, solution persistence, and other optimization artifacts. Though high-value in concept, OR-powered IT implementations increase project risks along new dimensions. As OR capabilities become checklist items on RFPs, the need for a robust, scalable, and practical OR-aware implementation methodology becomes more urgent.

In upcoming articles, I will discuss practice methodology development for OR-powered systems, starting with a review of how practitioners view practical OR.

Posted in Methodology, Operations Research | 1 Comment »

19 April 2008 by Sanjay Saigal.

At the recently concluded INFORMS 2008 Practice Meeting, multiple colleagues asked about our vision for Intechné. Quite simply, our vision for the company is to reliably deliver smart decision-making capability to our clients.

It goes without saying that smart business decision-making involves advanced analytical techniques from the fields of Operations Research, Statistics, and Artificial Intelligence. These include Predictive Analytics and Data Mining (to detect correlative, possibly causal, relationships in historical data), Monte Carlo Simulation and Decision Analysis (to simulate the impact of such relationships and tease out key sensitivities in anticipative decision-making) and various flavors of Optimization (both mathematically-driven algorithms and less-structured, heuristic approaches). But the application of a wide spectrum of techiques does not necessarily guarantee smart decisions. Intechné differentiates itself by explicitly focusing on an often overlooked issue in applying advanced analytics in the enterprise: Risk.

Viewed in the context of applying advanced analytics to business improvement, risk is like the weather: everybody talks about it, but nobody does anything about it. Or, nothing systematic at any rate. At the purely technical level, approaches such as mathematical optimization produce “brittle” decisions; very small changes in input can produce dramatically different recommendations. Perhaps that’s not of concern in the few situations where human operators are not in the associated decision chain. But in general, analytics are used to support decision, not execute them. Unexplainable, non-intuitive, or volatile decisions often force operators to work around their decision-support systems, or even completely ignore them. For instance, we found a sophisticated SAP/APO installation essentially ignored by its users (demand planners at a Food & Beverage company) because it couldn’t auto-profile different product types. While the overall MAPE was ok, sales forecasts for individual products diverged from reality in unexpected ways.

When it comes to delivering decision-support technology based on advanced analytics, a host of implementation risks arise beyond standard IT development risks. For instance, the response times of constraint programming models can decay exponentially with input size. (This is quite different from, as an example, rule-based decision engines.) Encountered unexpectedly, such non-responsiveness leads to expensive and disruptive modeling/algorithmic rework at an advanced project stage.

All in all, the business of delivering smart business decision-making is characterized by these, and many other risks. In informal feedback from colleagues and clients, we find that project mortality in this area is unacceptably high: about one in three advanced analytics projects fails to perform to expectation.

An active risk management orientation lies at the core of our vision for Intechné. In forthcoming communications we describe how this orientation is incorporated into our practice culture, and how it has been shown to improve client results.

Posted in Vision, Risk | No Comments »