This is the second part of a long post. The first part is here.
The Wired article I mentioned in yesterday's post describes Li's copula model as "a simple and elegant
model, one that would become ubiquitous in finance worldwide," and makes it clear that its simplicity and - its close cousin - tractability became the reason for its downfall: "[Quants'] managers, who made the actual calls, lacked the math skills to
understand what the models were doing or how they worked. They could, however,
understand something as simple as a single correlation number. That was the
problem." (As explained in my previous post, correlation turned out to vary over time, and to be very sensitive to estimation errors of the inputs, so the fact that the model required a single parameter gave managers a false sense of security.)
One of the
inserts caught my eye - it describes the copula model in a stylized, colorful formula, and presents the equal sign as "a dangerously precise concept, since it leaves no room
for error. Clean equations help both quants and their managers forget that the
real world contain a surprising amount of uncertainty, fuzziness, and
precariousness." What has the world become, if equal signs are now "dangerously precise concepts"?
In a January 2009 article in BusinessWeek ("Financial
Models Must Be Clean and Simple"), Emanuel Derman and Paul Wilmott
emphasize that "at bottom, financial models are tools for approximate
thinking." While Derman's and Wilmott's
stance in favor of simple and elegant models sounds contrarian given the
copula debacle, what it seems they mean is that the blame shouldn't rest on the approximate models, but on the models' users forgetting their models are approximate.
Such stylised representations of the world are valuable as long as people who use the models remain aware of their assumptions and limitations. It is, after all, easier to remember assumptions if one makes only a few. Derman and Wilmott also insist: "The most important questions about any model are: What does it
ignore, and how wrong is it likely to be?" - two questions quants, and people impressed by mathematics, have not learned to ask as often as they should. In addition, I enjoyed reading the "model maker's
Hippocratic Oath" at the end of the column.
Derman, now at
Columbia University after years at Goldman Sachs, is also quoted in The
Plato's Cave -- Mathematical models are a powerful way of predicting financial
markets. But they are fallible" (January 22, 2009), which recounts the
beginnings of quantitative finance,
and then describes the CDOs (collateralised-debt
obligations from the mortgage pools) crisis; in particular, CDOs were "impossible to model in anything but the most rudimentary
way". This is a case where the alternative to simplicity and elegance would have been intractability, as opposed to richer models.
The article also comments on Value-at-Risk and "tail
risk" (for VaR, see this old post of mine about an excellent New York Times article on the topic) and presents the remarkable idea, due to economist Daron Acemoglu
at MIT, that "modern finance may well be making the tails fatter",
because "you are swapping everyday risk for the exceptional risk that the
worst will happen and that your insurer [like AIG] will fail."
The 2005 article in WSJ, also mentioned in yesterday's post ("How A Formula Ignited Market That Burned Some Big Investors", September 12, 2005), quotes the head of market risk management at JP
Morgan as saying: "We're not stupid enough to believe [the model] is
omniscient. All risk metrics are flawed in some way, so the trick is to use a
lot of different metrics." (I wonder how his team has fared in the crisis - was he, in the end, stupid enough, just like the others?) Firms claimed they combined Li's model with
their own proprietary frameworks, but fell abysmally short of expectations - maybe everybody assumed everybody else would go to the trouble of creating their own models and went for the lazy option of using Li's formula only, hoping no one would notice.
Despite his bravado, the manager at JP Morgan is
fundamentally correct: people need to use many different models to develop a full picture of risk. They won't understand the limitations of a framework better simply because it is simple, elegant and convenient. On the contrary - it will tempt them to forget its limitations because they
will want to use it all the time.
Quants and investors must become used to
multiple models, focusing on different aspects of the problem, resulting in different optimal allocations, so that it is clear the allocations are only guidelines and the quants
can reclaim the responsibility of making decisions, instead of simply following the computer's recommendations without understanding how it got those numbers. Quants' bosses
need to become accustomed to the fundamental uncertainty and inaccuracy
inherent to mathematical models. There is, after all, always something
the models don't capture.
The WSJ article also shows remarkable prescience in identifying the risk of
"garbage in, garbage out" - "as with any model,
forecasts investors make by using the model are only as good as the
inputs" - and gives an example of early hiccup regarding the prized copula
function, involving General Motors. Stanford's Darriell Duffie, quoted
both in the Wired and the WSJ articles, gets the last
word: already in 2005, he stated: "The question is, has the market adopted
the model wholesale in a way that has overreached its appropriate use? I think
it has." It's a pity no one heeded his warning back then.
A student of mine drew my attention to an article called "Management
Science and the Management of Science," which the editor-in-chief of Management
Science - one of the leading scholarly journals in my field - wrote for the
December 2008 issue of the journal. In it, Hopp comments on the move of operations management research over the past few decades from tactical, low-key considerations (computing the parameters of the
inventory replenishment policy, for instance) to strategic, far-reaching ones (how to best
structure a supply chain), in part to gain more relevance in the eyes of top management.
Hopp writes: "Although [tactical issues] are of interest to engineers and
middle managers, they are not central to the concerns of senior management. So,
in the 1990s, [operations management] researchers began to aim higher at
questions of strategy." My student suggested - and this is one of the best
comments I've heard about the financial crisis in a while - that maybe quants
will now reposition themselves and reorient their models away from tactical, precise
issues toward more strategic, high-level problems.