Whom the Gods Wish to Destroy, they First Call Risk-Protected

Joe Nocera opens his recent NYT Magazine article on the failure of Financial Risk Management by referring to “a persistent tension between those who assert that the best decisions are based on quantification and numbers, determined by the patterns of the past, and those who base their decisions on more subjective degrees of belief about the uncertain future.” Like many of the statements in the long (7600 word), this will give pause to anyone who makes a living in Analytics. While it is true that in Finance, quantitative vs. “soft” approaches inspire quasi-religious passions, mainstream enterprises have long incorporated quantitative analysis as an indispensible tool. Note that word – tool. Analytic methods typically help along decision-making, they don’t replace human judgment.

Consider forecasting. It’s an inevitable function in a manufacturing or service enterprise. Forecasting systems typically include a statistical core – time-series trend analysis, regression-based “causal” models, or combinations thereof. But even in the CPG sector, where large volumes allow statistical forecasting to shine, demand planners inevitably perfect the forecast by injecting intelligence that the most advanced technology cannot replicate. For instance, most forecasts systems do not automatically incorporate adverse weather, or holiday-induced long weekends, or competitor actions. In fact, in my observation the more sophisticated the statistical forecasting system, the more sophisticated the planning staff’s analytic role.

But I digress. My point is simple – with experience, the conflict between Quants and traditional analysts (Quals?) becomes more notional than actual. I bring it up, however, to point out the serious lag between actual practice, and its reporting. But perhaps the schism has a more insidious side-effect.

Nocera’s article fingers the risk measure known as Value at Risk (VAR) as “what led to the financial meltdown”. (A good technical introduction to VAR is available here.) In a nutshell, for any financial instrument, VAR measures the maximum expected loss over a given time horizon. Since loss is uncertain, a probability or comfort factor is also involved. Institutions typically look for 99% or 95% certainty, depending on application. A number of “experts” are quoted denouncing or defending the role of VAR (including  innocent electrons sacrificed to Nassim Taleb’s predictable fire-bombing of practically all quantitative methods; this slow-unfolding but inevitable crisis is hardly a Black Swan, but to a man with a nuclear finger, all battles look like WWII).

Nocera’s indictment is unveiled as he recounts the gradual institutionalization of VAR:

• Instead of one of many risk indicators, VAR was treated as the risk measure. Speaking of investors and CEOs of financial institutions, Nocera says: ”In the bubble, with easy profits being made and risk having been transformed into mathematical conceit, the real meaning of risk had been forgotten. Instead of scrutinizing VaR for signs of impending trouble, they took comfort in a number and doubled down, putting more money at risk in the expectation of bigger gains. “It has to do with the human condition,” said one former risk manager. “People like to have one number they can believe in.”

  The tendency to rely on point estimates is a frequent cause of the so- called Flaw of Averages, which Probability Management is designed to combat. That the estimate is produced by sophisticated analytical computation doesn’t suffice. Understanding risk requires one to look at multiple alternative futures. That is best captured by a probability distribution (or a histogram, which is a granulated distribution). Or, minimally, by a ranking of scenarios.

• VAR can be non-additive over time: VAR computations are often based on the market at calculation time. But they aren’t updated as market conditions change, possibly dramatically in volatile times. When it comes to making new investments, that’s not a problem. But added to existing investments evaluated under different regimes, VAR may seriously misstate portfolio risk.
  
  
• VAR is super-additive over the market: It’s hard to understand how letting each institution rely on its own VAR measure could have sounded like a bright idea to Clinton-era regulators: ”The Securities and Exchange Commission, for instance, worried about the amount of risk that derivatives posed to the system, mandated that financial firms would have to disclose that risk to investors, and VaR became the de facto measure. If the VaR number increased from year to year in a company’s annual report, it meant the firm was taking more risk. Rather than doing anything to limit the growth of derivatives, the agency concluded that disclosure, via VaR, was sufficient.”

  Even if each market participant accurately managed its own risk, it’s stunningly obvious that given the proliferation of incestuous instruments such as credit swaps, the risk to the market was many orders of magnitude higher than any summation of individual risks.

Rereading the article, I am astounded by how many basic principles of quantitative analysis were violated by supposedly sophisticated actors. Perhaps it does come down to the Quant/non-Quant disconnect that opens the article: executives and regulators usually come through the latter door.

This entry was posted on 9 January 2009 at 13:53 and is filed under Intechné. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response or trackback from your own site.