Lies, damn lies, and National Review
The Lancet has recently published a study of the number of deaths in Iraq caused by the invasion. From the abstract:
Author Mark Goldblatt notes in an article on National Review that, in these calculations, one should also consider the fact that, before the invasion, Iraq was subject to sanctions that were lifted after the American occupation. By some estimates, the sanctions were causing about 150,000 deaths a year. This means that, since 2003, about 450,000 deaths might have been avoided because of the end of the sanctions.
Considering that one would have expected 450,000 fewer deaths, and one gets instead 650,000 more, the conclusion would be that the extra deaths caused by the occupation would be in excess of a million. Of course, simply adding the two numbers is problematic for a few reasons: some of the effects of the sanctions (for example, on the health care system) may be similar to the effects of the occupation, and hence would be having similar (rather than nearly disjoint) effects. Most importantly, it has been alleged that the estimates on deaths caused by the sanctions were overstated.
Anyways, what is Goldblatt approach? He subtracts one number from the other! You know how this works: I owe you \$20 dollars, now lend me another \$30 and I will give you the \$10 difference tomorrow. If I may suggest an improvement to his methodology, he should also subtract the number of deaths that occurred in Switzerland over the same period of time. I am sure he would get even more accurate estimate.
Update: see also Tim Lambert at scienceblogs.
Data from 1,849 households that contained 12,801 individuals in 47 clusters was gathered. 1,474 births and 629 deaths were reported during the observation period. Pre-invasion mortality rates were 5.5 per 1000 people per year (95\% CI 4.3–7.1), compared with 13.3 per 1000 people per year (10.9–16.1) in the 40 months post-invasion. We estimate that as of July, 2006, there have been 654,965 (392,979–942,636) excess Iraqi deaths as a consequence of the war, which corresponds to 2.5\% of the population in the study area. Of post-invasion deaths, 601,027 (426,369–793,663) were due to violence, the most common cause being gunfire.
Author Mark Goldblatt notes in an article on National Review that, in these calculations, one should also consider the fact that, before the invasion, Iraq was subject to sanctions that were lifted after the American occupation. By some estimates, the sanctions were causing about 150,000 deaths a year. This means that, since 2003, about 450,000 deaths might have been avoided because of the end of the sanctions.
Considering that one would have expected 450,000 fewer deaths, and one gets instead 650,000 more, the conclusion would be that the extra deaths caused by the occupation would be in excess of a million. Of course, simply adding the two numbers is problematic for a few reasons: some of the effects of the sanctions (for example, on the health care system) may be similar to the effects of the occupation, and hence would be having similar (rather than nearly disjoint) effects. Most importantly, it has been alleged that the estimates on deaths caused by the sanctions were overstated.
Anyways, what is Goldblatt approach? He subtracts one number from the other! You know how this works: I owe you \$20 dollars, now lend me another \$30 and I will give you the \$10 difference tomorrow. If I may suggest an improvement to his methodology, he should also subtract the number of deaths that occurred in Switzerland over the same period of time. I am sure he would get even more accurate estimate.
Update: see also Tim Lambert at scienceblogs.
posted by Luca at 4:53 PM
26 Comments:
10/14/2006 10:36:00 PM
"That means Bush’s decision to invade may actually have saved almost 100,000 lives."
He actually intends to ask you to pay $10 rather than pay the difference himself.
10/15/2006 11:23:00 AM
Also, most sanctions deaths were before 1997, while the pre-war baseline used by the study was 2002-3.
10/15/2006 11:39:00 AM
Indeed, lies, damn lies from all directions. Goldblatt's note about the fallacy of statistical studies here is on the mark. Even when social scientists try to get it right they almost never do. Certainly here with all the political incentives nothings should be trusted. But usually people have no problem believing any kind of lie as long as it approves what they already believe.
In this case, by the way, adding the numbers seems to me and even worse kind of demagogical propaganda. Do you have any reason to believe that the sanctions would have been lifted if it wasn’t for the war? (The reason Goldblatt's subtract seem to be that he claims that the study actually added the numbers (i.e., compared with the situation without war and without sanctions). This may be true or false, I have no idea).
In any case, were you really waiting for these numbers to hate Bush? If on a range of 10 or 20 years numbers will show that this war saved lives, would this change any of your believes? I doubt it!
10/15/2006 02:07:00 PM
Anonymous #3, just read the abstract of the Lancet paper. Goldblatt may be too busy to do it, and he may prefer to make things up, but you have it in front of you. They compare the actual number of deaths per year in 2002-03 (as given by census data) to the actual number of deaths per year in 2004-06 (as given by the household sampling).
10/15/2006 02:25:00 PM
I had no intention to support Goldblatt (just to give a more complete account of his claims). On the other hand, it will take much more than the abstract of the Lancet paper to convince me of its accuracy.
10/15/2006 08:01:00 PM
There seems to be a lot of confusion, so Luca maybe you can explain. As far as I understand, subtracting the numbers is correct:
- 650K deaths due to the war
- 150K lives saved due to lifting sanctions
- net total: 400K additional deaths due to the US invasion (assuming, presumably, that lifting sanctions goes hand-in-hand with the war).
I think the confusion is that it's not clear what numbers the Lancet is comparing.
10/15/2006 09:29:00 PM
Anonymous #6, first of all check your arithmetic. Then, the paper is available for free on the web, but even the abstract above is quite clear.
The Lancet study is not claiming that the war caused 650,000 deaths. It claims that from 2003 to 2006 there have been 650,000 extra deaths compared to the death rate pre-2003. The death-rate pre-2003 includes the effects of the sanctions.
If it were true that lifting the sanctions saved x lifes (and, as Aram notes, x would have not been such a large number), then one can only conclude (if the Lancet numbers are correct) that the effects of the war (bombings, ruined infrastractures and, most of all, sectarian violence) have claimed 650,000 + x lives.
If one believes that war was the only way to lift the sanctions, then one can say that "on balance" the war costed 650,000 lives. (Because the other x would have been lost anyways.)
Once more, the claim of the article is that the war caused 650,000 extra deaths by 2006, compared to what would have happened had the 2002-03 status quo been maintained.
Now it's entirely possible that the methodology ended up overestimating the mortality rate in 2003-06. Goldblatt, however, criticizes the study in a way that shows that either he hasn't bothered to read the first few lines of it, or that he isn't equipped with the most basic rudiments of logic, or that he is deliberately spreading misinformation (which is especially ironic in an article that claims to be about lies and statistics).
10/15/2006 09:53:00 PM
Umm... and since when do you read the National Review?
10/16/2006 01:42:00 AM
Guys, just make up some name to post with! So we know if anony 6 is the same as anon 7!
10/16/2006 01:47:00 AM
A correction to Luca's first comment: the new Lancet study (like the previous one) does not rely on a census or any other published piece of information to estimate the pre-invasion death rate. It gathered the pre-invasion death rate in the same way as the post-invasion death rate, by sampling households. Each household was asked to provide the dates of all births, deaths, and migrations in or out of the household in 2002-2006, with accompanying death certificates for any deaths.
The study merely notes that the estimated pre-invasion death rate they found matches closely with reliable published figures, like the CIA World Fact Book. Now, it is worth noting that this 2002-2003 death rate is much lower than other published figures, in particular a certain UN sponsored report. However, that report merely extrapolated the high death rates of the mid-late 90s (due to the sanctions) forward into 2002. As it turned out--and this was verified by a number of studies using on the ground information (which form the basis for, among other things, that CIA estimate)--the previously high death rate due to the sanctions was almost entirely eliminated by the oil-for-food program, which began in 1998 and ramped up in the years following.
Thus any notion that the invasion, by ending the sanctions regime, was saving thousands of lives was mistaken, based on outdated information. The oil-for-food program may have been corrupt, but it nonetheless fulfilled its goal of ending the humanitarian crisis. In any case, the task of actually identifying excess deaths is completely taken care of by the Lancet study's design.
10/16/2006 04:17:00 AM
Anonymous #9 said: “In any case, the task of actually identifying excess deaths is completely taken care of by the Lancet study's design.” You are giving an awful lot of credit to the Lancet study. Performing such a statistical study reliably in a diverse population that follows so many loyalties and political agendas is extremely hard and possibly impossible. In particular, I see no reason to believe that the study managed to factor out malicious attempts by the population to deceit it (its main source of data is after all the population). We have seen for example in Gaza how unreliable death counts and other data from the population can be. Back to Iraq, I have heard the pumped up number of pre-war deaths attributed to the sanctions used passionately, and not as an argument to go to war. I doubt that anybody took back his false arguments. So I reiterate “lies, damn lies from all directions”.
On the other hand, I totally agree with Luca’s original point regarding Goldblatt’s criticism. I originally misunderstood this point.
10/16/2006 09:11:00 AM
We can at least debate whether or not the methodology used by Lancet study and its results are accurate. They explain how they arrived at their results.
In contrast, the Bush regime just dismisses the study, announces its own numbers, and gives no insights in to the magic process that underlies their numbers.
It is sad that there is no real accountability for the repeated lies of the Bush regime while Lancet study is being heavily criticized.
10/16/2006 10:06:00 AM
A correction to Luca's first comment: the new Lancet study (like the previous one) does not rely on a census or any other published piece of information to estimate the pre-invasion death rate. It gathered the pre-invasion death rate in the same way as the post-invasion death rate, by sampling households.
You are right, thanks.
10/16/2006 11:58:00 AM
#3:
I don't mean to claim this study should be taken as the last word on Iraqi mortality. (Although given the interests of those who could conceivably produce such a study, it likely will be the last word for some time.) I'd just rather the criticisms of it be confined to areas where it is actually weak.
Establishing a baseline mortality rate is not one of them. The study used the exact same methodology to compute the pre-invasion death rate as the post-invasion rate. Insofar as we trust its estimate of the latter, we should trust its estimate of the former as well. The fact that its pre-invasion death rate is broadly similar to both the rate established in the 2004 Lancet study and the results of several other pre-invasion estimates is just icing on the cake. Similarly for the fact that the new study's estimate of the 2003-2004 death rate is in broad agreement with the 2004 study's.
Your next critique, that the surveyed population may have lied to the interviewers for political purposes, similarly betrays unfamiliarity with the methodology of the study. The surveyers asked to see death certificates authenticating 87% of the deaths reported; in 92% of those cases, the certificate was produced. Since the surveyed household clusters were randomly chosen and not notified ahead of time, it is (to put it mildly) implausible that they would have fraudulent death certificates sitting around on the off chance they would be surveyed in an excess mortality study. (On the other hand, since the death certificates are apparently used to claim survivor's benefits, perhaps there is a market in fraudulent certificates for other reasons. But the main use of death certificates seems to be to gain use of a cemetery, and in any case the notion of a black market in counterfeit death certificates is highly speculative.)
Discussion of the study has centered on the point estimate of excess deaths which, while understandably the simplest headline result, is in fact one of the weakest parts of the study. Instead of arguing about what our gut tell us about 650,000 vs. 350,000 vs. whatever, these are the conclusions which the study makes convincingly:
1. The violent death rate among Iraqis is much, much higher after the invasion than before. It is likely that the number of excess violent deaths since the invasion (as compared to a hypothetical baseline of the 2002-2003 rate extended indefinitely) is roughly in the neighborhood of half a million deaths.
2. The violent death rate has risen dramatically in each successive 14-month period since the invasion (late March 2003-April 2004, May 2004-May 2005, and June 2005-June 2006). In particular, the violent death rate in 2005-2006 was roughly 4 times that of 2003-2004, roughly twice that of 2004-2005, and roughly twice the rate of non-violent deaths across the entire surveyed period. (The rate of non-violent deaths appears to have stayed more or less the same.) While the magnitudes are quite different, this increase in the relative rates of violent deaths tracks very closely with numbers reported by various passive techniques such as those used by Iraq Body Count (substantiated reports in English language news sources) and the Iraqi Government (Baghdad morgue figures supplemented by other Ministry of Health data), thus giving further confidence in the results.
3. The violent death rate in Baghdad is not significantly higher than that for Iraq as a whole, and is in fact significantly lower than that of four provinces comprising a population equal to Baghdad's. This result is starkly different from the geographic distribution of the deaths tallied by Iraq Body Count or the official government figures, which are heavily weighted towards Baghdad. Indeed, as a result of the heavy reported casualties in Baghdad the coalition has recently redeployed its forces to concentrate on securing Baghdad. Given that IBC relies entirely on the reports of journalists (who are overwhelmingly stationed in Baghdad, and indeed in the Green Zone), and the government figures come primarily from the Baghdad morgue, it is no surprise that their figures would be heavily biased towards deaths in Baghdad, as suggested by the Lancet study.
This last point is, IMO, perhaps the most important result of the paper.
10/17/2006 11:00:00 PM
To Anonymous #9, You are absolutely right. I am totally ignorant of the details of the Lancet paper and too lazy to improve. Perhaps my skepticism was uncalled for. Perhaps this is the one case that such a complicated and politically charge statistical study was done right. Perhaps this is the one time we are not lied to. I still have my unsubstantiated doubts but I promise to keep them to myself.
10/17/2006 11:02:00 PM
For a dissenting view, see response to Lancet estimates in the Iraq Body Count website:
http://www.iraqbodycount.net/press/pr14.php
10/17/2006 11:05:00 PM
Fixing link:
http://www.iraqbodycount.net/press/pr14.php
10/19/2006 03:19:00 AM
" We have seen for example in Gaza how unreliable death counts and other data from the population can be."
Anon.#3 instead of making generous inclusions without support, a usual debate tactic. Either back this up with facts that show a statistical survey done similar to the lancet study was done in Gaza, or shut up.
You are right, This changes no one's opinions. I am calling your hand not because I disagree with your politics, which I probably do, but I disagree with your present rationale for objecting to a study that was an attempt to quantify something that the Bush Admin relies on magic. and using cheap debate techniques to support an onerous physical condition, regardless of the number who have died.
10/19/2006 08:21:00 AM
Either back this up with facts that show a statistical survey done similar to the lancet study was done in Gaza, or shut up.
Those facts about unreliable death counts in Gaza and Lebanon are common knowledge. For example, in a place called Kanna, few months ago, the Lebanese claimed they have 60 dead people. Only later the aid forces admitted there were only 28 dead.
10/19/2006 10:47:00 AM
EdoRiver said: "Anon.#3 instead of making generous inclusions without support, a usual debate tactic. Either back this up with facts that show a statistical survey done similar to the lancet study was done in Gaza, or shut up.
You are right, This changes no one's opinions. I am calling your hand not because I disagree with your politics,"
Yeah sure … and I bet that everyday you try to patronizingly shut up people who agree with your politics. As long as you feel good about yourself ... In my case you did beautifully well – I’m signing off this lovely discussion.
10/19/2006 10:48:00 AM
Sorry, the last message was from me (rather than Anonymous #6)
10/19/2006 11:36:00 AM
Here is just one example:
http://www.honestreporting.com/articles/45884734/reports/Jeningrad_What_the_British_Media_Said.asp
Of course, when the details came out nobody in the media really cared anymore.
10/19/2006 11:37:00 AM
Here is just one example:
http://www.honestreporting.com/
articles/45884734/reports/
Jeningrad_What_the_British_Media_Said.asp
Of course, when the details came out nobody in the media really cared anymore.
10/20/2006 06:44:00 AM
Thanks for the link!
10/22/2006 11:36:00 PM
Try to separate between whether there was a Lancet-like serious study that was done in Gaza in Lebanon rather than quoting sensational newspaper headlines that turned out to be wrong. (BTW, the same happenned with 9/11, where the initial estimates of casulties were about 20,000.)
10/23/2006 04:05:00 AM
Not exactly sensational newspaper headlines but rather a well thought propaganda act by official Palestinian representatives that admitted the truth only after getting all of the advantages of propaganda. (This did not prevent the propaganda to continue in the form of movies and books etc. As a side remark, I wonder when the American film industry will start giving awards to movies portraying the 9/11 terrorists similarly to the way Palestinian terrorists are portrayed in “Paradise Now”.)
This is not a separate incident and is part of an all around propaganda effort that includes in particular hospitals and morgues. One example is that many unrelated deaths in Gaza are attributed to the conflict. I’m sure you’ll be able to find enough examples by searching google. The relation to the Lancet study is in relying on local population (and even on death certificates).
Post a Comment
<< Home