Yglesias finds a working paper with a perfectly sensible little finger exercise demonstrating how CDO tranching creates safe-looking but highly fragile structures:
"Suppose we have 100 mortgages that pay $1 or $0. The probability of default is 0.05. We pool the mortgages and then prioritize them into tranches such that tranche 1 pays out $1 if no mortgage defaults and $0 otherwise, tranche 2 pays out $1 if 1 or fewer mortgages defaults, $0 otherwise. Tranche 10 then pays out $1 if 9 or fewer mortgages default and $0 otherwise. Tranche 10 has a probability of defaulting of 2.82 percent. A fortiori tranches 11 and higher all have lower probabilities of defaulting. Thus, we have transformed 100 securities each with a default of 5% into 9 with probabilities of default greater than 5% and 91 with probabilities of default less than 5%.
Now let’s try this trick again. Suppose we take 100 of these type-10 tranches and suppose we now pool and prioritize these into tranches creating 100 new securities. Now tranche 10 of what is in effect a CDO will have a probability of default of just 0.05 percent, i.e. p=.000543895 to be exact. We have now created some “super safe,” securities which can be very profitable if there are a lot of investors demanding triple AAA.
[...]
Suppose that we misspecified the underlying probability of mortgage default and we later discover the true probability is not .05 but .06. In terms of our original mortgages the true default rate is 20 percent higher than we thought–not good but not deadly either. However, with this small error, the probability of default in the 10 tranche jumps from p=.0282 to p=.0775, a 175% increase. Moreover, the probability of default of the CDO jumps from p=.0005 to p=.247, a 45,000% increase!
This is actually very good. But the comments, dear God, the comments! About a dozen people (including, I chuckle to note, Robert Waldmann, who would certainly have got this right if he'd been concentrating) step in to lambast the authors for oversimplification. A sample:
It’s not just misestimating the probability of default as .05 rather than .06. It’s also assuming that the probability of a mortgage defaulting is independent of the probability of every other mortgage defaulting. If there are economic developments that affect many borrowers at once, then the probability of borrower A defaulting *and* borrower B defaulting is not simply the prob(A defaults) * prob(B defaults). This is indeed what happened in 2008.
Moreover, the very act of A defaulting can alter the probability of B defaulting. If A defaults, and lives near B, then the rate of foreclosures in B’s neighborhood increases, which decreases B’s property value, which could push B underwater, and lead to B defaulting, which leads to C, D, and E defaulting, etc.
You know all of this, but I wanted to point out that the independence assumption is perhaps even more fundamental than using a mistaken probability. You can at least see what happens when you change .05 to .06, but non-independence can be a lot more complicated and make your model wrong not just by changing a single parameter.
Thanks for the sermon vicar, but ...
There is no assumption of independence in the original finger example, is there? In the first case, there's an expectation of the default rate. In the second case, there's some noise around this expectation, and that noise is perfectly correlated, isn't it? It's a realisation that is 0.01 higher in every single pool. The finger example is specifically and definitely one in which the entire sting is delivered by the fact that variables which were assumed to be independent turn out to be perfectly correlated.
I think that the problem here is that people tend to think of "correlation" as something measured by a Pearson coefficient, and so if they don't see one explicitly mentioned (or a copula structure specified or something), they just progress straight to the sermon and miss the big picture. Something to remember.
(Update, five seconds after posting: Actually I think the problem is more about imprecise use of the word "probability" in the original example. The number .05 doesn't refer to "the probability of default", it refers to the expected default rate, that's the only way the example makes sense. Expected default rates can't be correlated (because they're not random variables - they're expectations of random variables) which is why there's no correlation structure specified. Then "the probability moves to 0.06" in the second paragraph has to mean that the experienced default rate is 0.06 - ie, that the random variables concerned are actually perfectly correlated. The mapping from the default rates to the probability of default of the tranches makes sense though, because different loans are allocated to different pools at random).
(Update, half an hour after that. I suppose that in calculating the expectations in the first paragraph, there's an implicit assumption of independence. But that assumption is immediately relaxed and shown to be wrong! The real driver of this toy model is the fact that when the true default rate is discovered to be .06, this is true in every single pool.)
If you think of the mortgages as one period securities then the model might be realised, to keep things as simple as possible, by identical two-state default marginals (P(default) = 0.5 or 0.6)and a symmetric joint distribution/copula. The original point is that the CDO^2 is insanely sensitive to estimates of the marginal. It is probably also even more sensitive to even fine details of the copula.
ReplyDeleteYes you're right but the copula in the finger exercise is a very simple one - when one security is in the 0.6 marginal, so are all the others.
ReplyDeleteYes, to be specific the exercise assumes independence so all the CDO default probabilities can be calculated by the binomial distribution.
ReplyDeleteThe independence assumption comes in in generating the default probabilities for the tranches. Saying the marginal probability of each mortgage defaulting is 0.05 is compatible with a wide range of joint probabilities of multiple mortgages defaulting, e.g., if even one defaults then all default. The particular default rate for "tier 10" that they quote comes straight from the binomial formula, and independence across mortgages. Of course Coval & c. realize that it's not enough to use just the marginal distribution + independence (see p. 8 of their working paper), though I have certainly seen practitioners make similar errors.
ReplyDeleteTo be more precise, the calculation assumes that event "Irene's mortgage defaults" is statistically independent of the event "Joey's mortgage defaults", though the two events have the same probability. Are you perhaps thinking of these probabilities as random variables, and so the assumption that they're all equal as a strong dependence assumption? (Your comment 68 to Yglesias's post suggests as much.)
Hit post too soon: Of course it's possible to go through the same sort of pricing exercise with any joint distribution over the mortgage defaults you please, but in the nature of things one has very little evidence about high-dimensional joint distributions, so the error bars, were one to calculate them, would be very large. My impression (and I'm happy to be corrected by those who know more) was that most people stopped at the bivariate distribution, and sensitivity analyses were very rare indeed. (And as for just using a coupla, well...)
ReplyDeleteI was just thinking about balls and urns. A mortgage is an urn with 5 white balls and 95 black balls in it - you create the underlying pool by getting 100 urns (so 500 white balls and 9500 black balls) and drawing a ball from each one of them. Then you discover that each urn actually had 6 white balls in it.
ReplyDeleteYes there's an independence assumption about the urns, but the actual work of the example is being done by the fact that the additional white balls are allocated one per urn. If they weren't then you could have the same overall number of white balls and black balls but the AAA securities would have been safe (easy example - say in case 2 you suddenly change to having six urns with 100 white balls each and 94 urns with 100 black balls each - in this case you get 6 white balls with certainty and the AAAs are a lot safer).
So although the binomial theorem is being used with independence, it isn't fair to say that the finger example is ignoring correlation - the correlation of the errors is very important.
I would also add that it isn't really true that correlation structure was what did the damage. The correlations in the pools actually *were* low pre-crisis - this is why the whole copula movement got going. In any case, very low variance of the observed default rate will actually do most of the work of independence for you in these tranched structures.
ReplyDeleteWhat then happened was that there was a massive shock that turned out to be correlated. But the problem wasn't the correlation - it was the shock.
Depends upon your correlation model doesn't it.Either correlatin works ordinarily but this was an outrageously and implausibly large shock or you model correlation a bit differently and the shock comes within normally seen ranges.
ReplyDeleteThe copula approach was already in use before things went wrong. unfortunately there was a rather convenient formula for a one parameter Gaussian copula which you could actually compute. Everyone knew it wasn't right but compensated by bumping up the correlation a little (even this sometimes has counterintuitive effects) but unfortunately it was still severely misspecified and with such a model actual events were still a large multiple of sigma event. Better models were known but were hard to compute and gave unhelpful results.
Yes good point (this is practically a symposium). Because I'm a sceptic about probability in general, I am structurally inclined to favour simple (or even nonexistent) correlation models and treat shocks as large events, because then you can put down the model and put your real-world head on and start thinking about what that large event might be. (I still think this is the best way to do modelling in most cases even though empirically, most large events are actually collections of small events happening all at the same time).
ReplyDeleteMy objection was with the discussion of the case in which the probability of default was 5% for each mortgage. I didn't get on to the case of 6%. As you note " in calculating the expectations in the first paragraph, there's an implicit assumption of independence. "
ReplyDeleteExactly. The probabilities of default for tranches given the assumption that each mortgage has a 5% probability of default are calculated assuming that each payments on each individual mortgage are independent random variables.
At the end of the post, Tabarrok has stuck to the claim that if the probabilities of default of each mortgage were exactly 5%, then the calculated probabilities of default of a tranche and of a tranche made of a pool of such tranches would be correct. This claim is false. Given the assumption of 5% all he can conclude is that the probability of default of the CDO is greater than equal to zero and less than or equal to one.
As far as I can tell, your post asserts that I made a gross error and then concedes that my claim was exactly correct -- Tabarrok assumed that payments on each mortgage are independent random variables and did not state that assumption. Since such correlations are absolutely totally central to the current world economic situation, I do not consider this a minor oversight.
But it is a minor oversight, because the "independent" 5% probability is immediately given a massive correlated shock. That's how it delivers the result, and it delivers it in a way which is totally analogous to the real world.
ReplyDeleteWell, exposure to a common factor is a common way of modelling dependence structures and if you have a finite sample you'll never be sure whether you got the default probability wrong, the correlatin wrong or were just very unlucky if things go badly.
ReplyDeleteWhich matters more, being wrong about the default probability or being wrong about the correlation? The correlation. Consider tranche 20. If there is no correlation the probability of default is virtually nil (OK 25 basis points) for even a default probability of 10% but with a 5% default probability and a 20% (gaussian) correlation the expected loss on tranche 20 is ten times that.
To make matters worse if you underestimate the the correlation you will also underestimate the default probability thus compounding the already dramatic underestimation.
Still, I don't see why the model upset so many people. It makes a point quite neatly. Why does it have to be the most severe? It is not as if the market assumed that defaults were independent -- CDOs were "correlation products". Would Alex Tabarrok even have read the paper if it had done the numbers with correlation as an example?
Anyway, I maintain the problem was sociological not technical. It's no good having the perfect model if nobody believes you or it doesn't let you trade.
The real problems were not the defaults but their impact on the pricing of super-senior bonds and the presence of those bonds on the balance sheets of the global financial conduits. Basically split cap trusts only bigger.
PS having said that the problem is sociological I see that Frank Partnoy of all people is suggesting replacing the rating agencies with CDS prices. He is clearly an agent of chaos -- security through positive feedback!
ReplyDeleteI would still say that this model has exposure to a common factor in it - when Tabarrok (or whoever) says "then you find out that the true default rate was .06", that's effectively a common factor.
ReplyDeleteBasically split cap trusts only bigger.
yes very true - I seem to remember Anthony Bolton made this point at the time.