Calm down, Danny me boy, sit down in comfy chair and let's all brew up a nice pot of Shut The Fuck Up
Still pretty angry, but long and outraged lists of things tend to dilute rather than reinforce the impact, and listing all the errors I can find would probably take me over the "fair use" limit for quoted material. There is an awful lot of statistically illiterate crap in Megan McArdle's piece in The Atlantic, and some laughably one-sided presentation[1] but I'm going to concentrate on two sentences.
Yet though its compromises made it particularly unreliable, the Lancet study remains the most widely known. Its conclusions were the earliest and most shocking of the scientific estimates and thus generated enormous media attention. The more-careful counts that followed prompted fewer, and less prominent, articles.
McArdle is here referring to the 2006 Johns Hopkins study (Burnham et al), which she believes to have been falsified by the WHO study published in the New England Journal of Medicine, and which found a much lower rate of violent death (although a correspondingly higher rate of non-violent death; Lancet hacks seem to regard this fact as something that can just be brushed away in what I call "Greengrocers' Arithmetic"[2]).
However, of course, Burnham et al (2006) was not "the earliest" such study. It was (and this is not exactly a little-known fact) a follow-up to Roberts et al (2004), the original Lancet study. And, as is abundantly clear to anyone who has read the study and can multiply, the rate of violent death found in Roberts et al (2004) is very nearly exactly the same as the rate of violent death in the WHO study.
The first Lancet and the NEJM studies confirm each other on violent death rates, and all three studies give a fairly similar answer for total death-rates post-invasion (the WHO study does not calculate a rate of "excess deaths" from non-violent causes, but barring really quite strange correlations in the data, simple subtraction of the death rates gives a consistent answer). So the two possibilities here are that either the three studies are all broadly confirming each other on the total death rate, and something about Burnham et al (2006) caused respondents to classify a greater proportion of the deaths as violent[3], or that Burnham et al (2006) is a complete outlier because its violent death count is so high, and the fact of a very high nonviolent death rate in both Roberts et al (2004) and the WHO study is just some sort of coincidence.
But this is by the by. The important point here is that Megan McArdle, in what she believed to be her definitive, highly researched article, appearing in the print version of The Atlantic, still managed to get confused between the two Lancet studies and claim that "the earliest" study was falsified by the NEJM article. Nobody caught this mistake between copy and print, but it wasn't anyone else's job to.
I'm astounded. McArdle has apparently recanted on previous support for the war, and has apparently also quietly given up on a number of the astonishingly stupid arguments she made about Burnham et al (2006) at the time - in particular the "mythical Central Death Certificate Repository". So it is difficult to see why she might have wnated to write this article (which has very little to say about the studies themselves - half of it is boilerplate pop-science versions of Tversky's anchoring hypothesis with very little relevance to anything), as she no longer has any particular point of view to defend. As far as I can see, the only purpose is to try and lay the ghost of the fearful ass she made of herself back at the time, and re-establish some sort of credibility on the subject, while whining about how nasty everyone else was. The vanity is really quite overwhelming. And even in that article, she couldn't even be bothered to keep the Lancet studies straight. And such a person, such a journalist, someone with such a lack of self-awareness, has the outright temerity to try and psycho-analyse people like me and Tim Lambert, and to claim that we rejected the obvious truthiness of the NEJM violent death count because we were disappointed to find that 450,000 Iraqis had not really died? That we were lost in partisanship of Les Roberts and could not find it in our hearts to celebrate? I find it really rather disgusting.
[1] Most notably, trying to sound all knowledgable by running through the Hicks-Rei critique of Burnham et al's interviewing protocols and potential sources of bias in the subjective element in selecting house for interview, but somehow neglecting to mention that the WHO decided not to carry out any interviews at all in Anbar province because it was too dangerous and then made a "statistical correction". In actual fact, the correction made to the WHO study was about the best way you could have massaged the data and the article itself is quite upfront about what they've done and the uncertainty it might introduce, but to not even mention it is quite ludicrous.
[2] Because it tries to compare apples to oranges, picks cherries and ends up with a lemon.
[3] Not at all necessarily incorrectly - there are very good reasons to believe that in Roberts et al (carried out much more recently after the war) and the WHO study (carried out by government personnel who identifed themselves as such), respondents had a significant incentive to misclassify death squad killings or deaths of family members who were part of the insurgency.
(The footnote number for Greengrocer is wrong.)
ReplyDeleteI didn't want the footnote numbers to be "absurdly precise".
ReplyDeleteThe correspondence on overall death rates between NEMJ survey and Lancet surveys means that this bit is quite stupid, as well as offensive:
ReplyDelete[few of the study’s supporters expressed much pleasure at the news that an extra 450,000 people might be walking around in Iraq]
Even if NEMJ survey was correct and the violent death rate was much lower, the similar overall death rates means that people just died of other causes, not that 450k people are actually alive.
Yes absolutely - I think that the logical straw that people grab at here is that the WHO didn't actually calculate an excess deaths figure in the paper (so the 450k figure which is just arrived at by simple subtraction probably isn't exact) and so it can be ignored.
ReplyDeleteI really can't express how angry I am about that by the way. I vividly remember how I felt when I first saw the headline figures from Burnham (2006). I actually thought I was going to physically vomit - pretty much the only time something like that has happened to me. The idea that someone mght actually have believed that I read it and thought "Oh goody! I am proved right after all!" - I mean christ, what kind of a person do you have to be to think like that about someone?
Conversely, my reaction to the WHO study was basically numb - by that point it was clear that the whole thing was a nightmare of historic proportions and it looked fairly obvious that the total death count hadn't moved, and all I could think was "ho hum, another round of wingnuts trying to abuse this for propaganda".
If that piece appeared in the print magazine, as I greatly fear, the Atlantic really is asking for a shoeing.
ReplyDeleteBut McArdle is proof that being wrong in a cheerful, witless and frankly evil manner is a very good way to ascend the greasy pole of American political journalism.
I despise her.
Is there a brief but impartial summation anywhere of the differences between the several Lancet and NEJM surveys, or is it possible to provide one? (A couple of years ago I was working in a medical library and could have accessed them myself, but alas, no more. Alas in this regard, anyway, if no other.)
ReplyDeleteTim Lambert's site is probably the best resource - it's only "partial" in the sense of favouring accuracy over error.
ReplyDeleteTake a look to the new project of mine. It is a website based on really interesting idea. ;) I hope You will love it.
ReplyDeleteBy the way, it can be very useful for You if You want to be more visible.
Try the power of it - firsts 28 uploads into 1 euro category is for free.
Address of the website: http://www.individualdream.com/
Unrelated to this article, but as far as CAIF goes, some googling turns up this:
ReplyDeletehttp://justinsomnia.org/2007/11/importing-haloscan-comments-into-wordpress-23-from-blogger/
and this:
http://weblog.philringnalda.com/2002/05/14/importing-yaccs-comments-in-movable-type
One of which might be useful.
"wnated"
ReplyDelete