« Cloned Dogs | Main | Buzz »

January 05, 2009

Comments

Ken Houghton

I'm not dsquared, but I'll play him for this moment:

What VaR are they using that works with a strictly normal distribution--the one Bankers Trust threw away ca. 1988??

No one (in their right mind) assumes a standard normal distribution; we knew about "fat tails" in the early 1990s.

Kwak, otoh, is spot-on, as usual. But this again comes as no surprise to anyone who knows anything about, say, Argentina's 2001 debt crisis or the Internet stock bubble or the Structured Note market ca. mid to late 1994. And since I've just mentioned (respectively), Emerging Markets, Equities, and Debt trading, that pretty much covers any mid-to-high level Financial Services professional who is older than Miley Cyrus.

Blaming VaR, or some perverted version of VaR, is absurd.

Cosma Shalizi

There is absolutely nothing in the definition of VaR or the basic methods which force you to use Gaussian distributions. If you want to assume a fat-tailed distribution of shocks and calculate VaR, you can certainly do that. (Of course you need to estimate the distribution first, but that's doable.) In fact, what's supposed to be the industry standard (as in, part of the Basel II regulations, iirc) is to calculate VaR by randomly resampling historical data on actual price movements. No doubt there are many idiots who do calculate VaR assuming everything is Gaussian, but presenting that as an intrinsic flaw in the concept would be equally idiotic. So phooey to Smith.

The much bigger problem is that financial markets are non-stationary, and they change on a fairly short time-scale at that, so that there is never all that much relevant data around to estimate these distributions. But I'll leave saying more about that to d^2, since I've got a paper to finish.

belle waring

see, blogging is even more awesome because it's self-correcting!

Cosma Shalizi

Also, speaking of Sir Mixalot and "fat tails".

dsquared

I was actually a VaR hater before it was cool to be so, and *significantly* earlier than Taleb's 1996 "Derivatives Strategy" interview. There are a couple of memos in the Bank of England archives at Roehampton (I say this for the benefit of future biographers[1]) in which I described the original JP Morgan RiskMetrics document as "basically a dressed-up standard deviation", and one in which I suggested an alternative measure with a regulator-imposed correlation structure. On the other hand, as Cosma and Ken say, it is Just Not True that anyone (possible exception of some of the RM marketing team, who did often appear to have inhaled deeply of their own product) thought it was the be-all and end-all.

The underlying problem here is that pre-RiskMetrics, there was a genuine problem which they were attempting to solve. And that is, that it seems pretty important that the chief executive of a bank should know the standard deviation of the aggregate securities portfolio of that bank. But, on the other hand, just telling him what the standard deviation is, is giving him a really unusable statistic; imposing some sort of parametric structure on it and giving the VaR gives a much more intuitive interpretation[2]. It is, intrinsically, a useful piece of information to know.

The big failure here was a failure of leadership, not of risk management, and I'd be very tempted to blame the general "ooh! index funds! ooh, modern portfolio theory! you can't predict the future you know, markets are efficient" culture which promoted the idea that you can manage a company, still less a financial institution, without actually putting your neck on the block and making a forecast.

[1] "In this famous blog comment, the Glorious Leader is actually considerably exaggerating both his prescience and his technical competence at the time; nonetheless the Roehamptom Archive does bear out his claim to have criticised the use of VaR and his suggestion of an alternative, albeit that his specific proposal was not workable". "Davies: Man of the Century", Barbara Cartland, Wrexham University Press, 2115

[2] Shouldn't the chief executive be able to understand a standard deviation! These people are paid millions blah blah blah. Not really. The estimated sd will change simply because the size of the portfolio changes. It's a really difficult number to deal with.

dsquared

Eric Falken is rather sound on the subject. I noted a few months ago that, of course, insurance actuaries (and more or less everyone else who takes more than one risk and wants to get an idea of their aggregate position) also use measures of risk which reduce things to a single number (because basically it's an amazingly fundamental risk management tool) and indeed insurance companies sometimes get wiped out by hurricanes or mesothelioma, but it's part of the process. Anything which has a risk, has a risk of catastrophic loss.

dsquared

triple posting like the loon I am, though, I'll note that this:

For instance, during his early years as a risk manager, pre-VaR, Guldimann often confronted the problem of what to do when a trader had reached his trading limit but believed he should be given more capital to play out his hand. “How would I know if he should get the increase?” Guldimann says. “All I could do is ask around. Is he a good guy? Does he know what he’s doing? It was ridiculous. Once we converted all the limits to VaR limits, we could compare. You could look at the profits the guy made and compare it to his VaR. If the guy who asked for a higher limit was making more money with lower VaR” — that is, with less risk — “it was a good basis to give him the money.”

from the head of the VaR team at the time, is clearly and obviously theoretically wrong, it was known to be wrong at the time (Andre Perrold wrote the article IIRC), and it's really rather worrying that he apparently still hasn't realised it's wrong.

Is there any reason why the NYT decided to strip Sir Dennis Weatherstone of his knighthood, by the way? Is that stylebook there?

" As Guldimann wrote years later, “Many wondered what the bank was trying to accomplish by giving away ‘proprietary’ methodologies and lots of data, but not selling any products or services.” He continued, “It popularized a methodology and made it a market standard, and it enhanced the image of JPMorgan.”"

IMO this is self-serving and not true; I and others strongly suspect that JPM pushed and sold RiskMetrics so hard because they wanted to influence the direction of regulation (and they succeeded massively in doing so).

Btw, I've just crunched a couple of numbers on Taleb's returns which might be of interest. If we take his statements in the report at face value that he's made three big profits (1987, 2001 and now) and that he makes between 65% and 115% profit in a good year (btw, is it 65% or is it 115%!? great risk control dude), then they're not actually all that great.

If you have to wait 13.5 years (between Oct 1987 and Mar 2001) and you're losing 5%/year on your option-buying strategy, then by the time the big payout comes, you've lost 50% of your money. If you make a 65% return in that one year you're clearly worse off than having stuck the money under the mattress; even at the 115% level, you've only made a 0.5% compound return over the period.

Taking a more charitable average 10 year gap between payoffs (because I would guess that Taleb actually made money in 1998 as well), then his breakeven cost per year (versus an assumed 3.15% return which is a reasonable money market rate, also the compound return on the FTSE100 between 1 Oct 87 and yesterday) would be 1.9% for the 65% profit and 4.5% for the 115%.

At a 5 year gap between payoffs, the breakeven is 6.5%-11.5%, but I think this is much too favourable. I would guess that Taleb's strategy costs about 4% in a normal year and pays off roughly once a decade, so it produces at best LIBOR returns and probably more like 2%. Which is not awful given the risk characteristics (albeit that it is certainly not devoid of tail risk - one single counterparty default would leave it pretty much fucked), but not superstellar either.

PG

Is there any reason why the NYT decided to strip Sir Dennis Weatherstone of his knighthood, by the way? Is that stylebook there?

I don't think there's a consistent style regarding use of the titles "Sir" or "Dame." Sometimes Elton John is preceded by "Sir"; sometimes not. I imagine it depends on whether the person writing/ editing the article finds the title incongruous and thus a good way to liven things up a bit.

Doug

I'm thinking the technical term at work here was "devil take the hindmost".

nnyhav

Cosma & d^2 have this covered, but perhaps I can add something between the lines from closer to the front lines:

What I said elsewhere:
I'd agree that too much stress is put on normality. Overly simplistic, yes, and mere gestures on fundamental issues (overreliance on history and on ratings were baked into the overreliance on VaR) [...] Large firms used direct historical sampling (more than the 2 years indicated in NYT) across thousands of data series to estimate VaR (so higher moments are baked in as well), forgetting that past performance is no guarantee ... not to mention the problems of classification for mixed and leveraged assets. For individual asset classes, there have been numerous attempts to generalize distributions e.g. for option pricing, and Taqqu & Samorodnitsky's Stable Non-Gaussian Random Processes lays the basis for stochastic analysis, but multivariate approaches [to estimation] remain lacking. Tail estimation is another matter (see e.g. Embrechts' Extreme Value Theory). Deficiencies in multivariate and marginal techniques are further compromised by sparse observational data and by asynchronous behaviors (GARCH notwithstanding). All of these alternatives were established by the late 90s, but none of the attendent problems overcome.

To which I'll add:
VaR was augmented by stress measures, but the latter were anchored by VaR both in approach (e.g., overreliance on a longer history) and in results (how many orders of magnitude difference were believable?); the anchor was reinforced by capital adequacy requirements being based on the VaR measure.

The comments to this entry are closed.

Email John & Belle

  • he.jpgjholbo-at-mac-dot-com
  • she.jpgbbwaring-at-yahoo-dot-com

Google J&B


J&B Archives

Buy Reason and Persuasion!

S&O @ J&B

  • www.flickr.com
    This is a Flickr badge showing items in a set called Squid and Owl. Make your own badge here.

Reason and Persuasion Illustrations

  • www.flickr.com

J&B Have A Tipjar


  • Search Now:

  • Buy a couple books, we get a couple bucks.
Blog powered by Typepad

J&B Have A Comment Policy

  • This edited version of our comment policy is effective as of May 10, 2006.

    By publishing a comment to this blog you are granting its proprietors, John Holbo and Belle Waring, the right to republish that comment in any way shape or form they see fit.

    Severable from the above, and to the extent permitted by law, you hereby agree to the following as well: by leaving a comment you grant to the proprietors the right to release ALL your comments to this blog under this Creative Commons license (attribution 2.5). This license allows copying, derivative works, and commercial use.

    Severable from the above, and to the extent permitted by law, you are also granting to this blog's proprietors the right to so release any and all comments you may make to any OTHER blog at any time. This is retroactive. By publishing ANY comment to this blog, you thereby grant to the proprietors of this blog the right to release any of your comments (made to any blog, at any time, past, present or future) under the terms of the above CC license.

    Posting a comment constitutes consent to the following choice of law and choice of venue governing any disputes arising under this licensing arrangement: such disputes shall be adjudicated according to Canadian law and in the courts of Singapore.

    If you do NOT agree to these terms, for pete's sake do NOT leave a comment. It's that simple.

  • Confused by our comment policy?

    We're testing a strong CC license as a form of troll repellant. Does that sound strange? Read this thread. (I know, it's long. Keep scrolling. Further. Further. Ah, there.) So basically, we figure trolls will recognize that selling coffee cups and t-shirts is the best revenge, and will keep away. If we're wrong about that, at least someone can still sell the cups and shirts. (Sigh.)