A British polling outfit surveyed Iraqis about deaths in their household and came up with a tally of 1.2 million dead Iraqis as the central point of their estimate (obviously, with a big number like that, the confidence interval includes a wide range). Kevin Drum points out skeptically that this survey seems to indicate that car bombings are being massively underestimated in Iraq -- out of 20 or so a day, only 2-3 are getting reported. I agree with him that this seems wrong, but it also seems implausible to me that families would be massively overreporting deaths.
« Trending Left | Main | Edwards and the Region »
Surveying Iraq's Mortality
14 Sep 2007 12:30 pm
Comments (43)
This strikes me as a very poor way of totaling up casualties. Why not just use official records? These surveys run the risk of overlap since different people may be acquainted with the same victims. I realize that during an actual battle theere may be no reliable statistics, but surely that's not the case now when most of the violence is very low level, due not to pitched battles but to random acts of murder which will be reported to the authorities.
No offense to Kevin (or to you, Matt), but I'm always a bit surprised when commenters suggest that certain numbers from Iraq "don't sound right". That was, of course, the basis for much, if not most, of the criticism of the earlier Lancet studies.
But where in God's name does this "feel" for the right numbers possibly come from? Unless you've spent a lot of time -- a lot of time -- in countries where civil authority has basically collapsed (as in Iraq), how could you possibly know whether events such as car bombs are underreported by one-half, one third, one-tenth, or any number? I certainly have no idea -- none-- what to expect out of mortality reporting from such areas.
but surely that's not the case now when most of the violence is very low level, -- Jonf
Gen. Petraeus? Is that you? Welcome!
It's not that the number of dead Iraqis feels wrong, it's that the idea that car bombs are being undercounted by the US military by a factor of ten feels wrong. On the other hand, as I wrote in the post, however, the idea that families are overreporting deaths to pollsters also seems wrong to me in which case a figure in the high hundreds of thousands to low millions must be right.
This was mentioned on Kevin's site, but it bears repeating. The concepts of "household" and "family" are a little different in the Middle East. These concepts can include extended family and friends, so double counting of deaths is likely due to cultural mistranslation. To give you an idea about "family", I am American, but I am a "brother" in a family in Istanbul. So when an Iraqi says his "brother" was killed, the dead man probably isn't in his nuclear family and might not even be an Iraqi. This isn't exaggeration, it's just a different view of the concept of "family."
I really hope that Drum and Jonf are right.
One death should not be reported twice, since households do not overlap. However, if Iraqis interpret whatever Arabic word was used in the survey as "family" then answer by reporting deaths in their extended family, then deaths might be double counted. Still with such a small sample, it is hard to see how many deaths could be double counted let alone counted 7 to 10 times unless extended families are really really extended.
One conceivable explanation is refusal to participate. If the surveyors say they are conducting a survey on civilian deaths in Iraq, people with no deaths to report might not open their door or agree to participate. Selective response based on the variable of interest destroys a survey. Another possibility (which would depend on misconduct by the survey workers) would be interviewing neighbors who volunteer to participate. They will almost all wish to participate in order to report a death (no one tugs you sleeve and then says "I have no information for you"). Such households should not be included, but, since walking around asking Iraqis about deaths is a good way to become a statistic oneself, I wouldn't blame people who get two filled out questionaires after knocking on one door.
Here is the exact question asked:
How many members of your household, if any, have died as a result of the conflict in Iraq since 2003 (ie as a result of violence rather than a natural death such as old age)? Please note that I mean those who were actually living under your roof.
That last sentence would seem largely to rule out confusion over different cultural meanings of the word "family" or "household" as contributing to an overcount.
I know why Iraqi families would over-report deaths; it's to persuade Americans to give up support for the war, and get out, so we should stay there, or else more Iraqis would kill each other, which would lead to even more deaths, which would help... shit, is this a conservative feedback loop or a moebius strip?
This strikes me as a very poor way of totaling up casualties. Why not just use official records?
The Iraqi government or the American military could definitely do a better job estimating the number of Iraqi civilians killed. The problem is that they don't, or rather when they do, they tend to game the numbers (Violence in Basra doesn't count because it's mostly Shia-Shia. Same goes for tribal violence in Anbar which is mostly Sunni-Sunni). We've also seen huge disparities in the counts obtained by the U.N. vs. the Iraqi government and the counts by the GAO also vary dramatically from those provided by General Petraeus.
In short, official records are suspect, even when available, because the officials themselves want to project a better picture.
If you compare American survey results for church attendance to the attendance reported by the actual churches, or if you compare how many Americans claim to have voted with the actual voting totals, or if you compare how many female sex partners the average male reports to the number of male sex partners the average female reports, you will lose all confidence in surveys of this nature. But maybe Iraqis are more honest in dealing with total strangers asking nosy, politically-charged questions than Americans.
This was a helpful post from earlier this year on different studies on Iraqi deaths:
http://www.samefacts.com/archives/the_war_in_iraq_/2007/06/the_deaths_in_the_pottery_barn.php
1.2 million is high, but not out of the ballpark if you look at these other studies...
Re: But where in God's name does this "feel" for the right numbers possibly come from?
From past experience with previous wars, and with previous cases of wildly inflated casualty figures from estimated sources. Remember the 10,000 dead we were assured had perished on 9-11, or in Hurricane Katrina? Or the "tens of thousands" who were supposed to have died in the Chernobyl disaster (the actual toll was dozens). A certain skepticism is always in order in such matters where hard evidence is lacking.
Re: but surely that's not the case now when most of the violence is very low level, -- Jonf
Gen. Petraeus? Is that you?
Oh good grief, "Low level" here means that we are not looking at mass casualty events like, say, the battle of Guadacanal or the firebombing of Dresden, when many thousands may perish all at once. We are looking at a bunch of murders (and yes, lots and lots of them) where the death tolls are usually in the single digits. I make no claim that the violence in Iraq has decreased one scintilla, only that it is not of the same magnitude as a pitched battles or a citywide fire bombing raid.
A certain skepticism is always in order in such matters where hard evidence is lacking.
I think you missed my point (as I am happy to concede I missed yours regarding "low level" violence). I fully agree that skepticism is warranted of any numbers. But Kevin (and, to a lesser extent, Matt) was saying that an under-reporting of car bombs by a factor of 6 to 10 didn't feel right. And my point is, how could you possibly know? Would an undercount of a factor of 2 or 3 "feel" more right? Why?
If there are studies that have shown that in war areas deaths tend to be underreported by a factor of X, then I'd change my mind on this. But in the absence of such data, I strongly dispute that most people's life experiences are such as to give them a remotely accurate "feel" for what X should be.
I have my own "feeling" for what the true death toll is--a few times greater than the official and semi-official counts we get from the Iraq government or Iraq Body Count, but probably less than a million.
But I also have enough sense not to take my "feeling" very seriously, for the reasons Glenn states. I could make a case for my position, but it isn't a strong one.
y81: in your examples, people have a reason to exaggerate their numbers. Why would someone brag about how many family members died?
One thing that came up in comments on Drum's site was the possibility that car bombs are being used as a catch-all cause of death, if the family is unwilling to say it was a political execution, or whoever informed them of the death didn't want to say it was a political execution.
(that's IF car bombs are actually being underreported.)
"From past experience with previous wars, and with previous cases of wildly inflated casualty figures from estimated sources. Remember the 10,000 dead we were assured had perished on 9-11, or in Hurricane Katrina?"
I don't remember that for Katrina but I surely do from 9-11.
Of course, there are differences. Those were one-off events, fairly isolated, in a stable country with a government and law enforcement that works hard to get numbers like these right. The 10,000 number was on that exact day, during the frenzy and fear, before people knew fully was going on. It worked itself down once people did get on the ground and start counting. Which was a day or two IIRC.
Iraq is an ongoing civil war, slowly but steadily burning through a pretty large country. There's virtually no government in place and what is there is known to game the official statistics wildly. Do the Shiites in charge really have an incentive to accurately report how many dead Sunnies are turning up?
But, you know, that is why the variance is so high on these numbers. I believe the Lancet study was between 300,000 and 1 million. Lets take that low number, even cut it in half, and say the "actual" number was 150,000. Is that an acceptable amount of deaths? Violence has increased greatly since that study was published, keep in mind. At the time of the Lancet study, the Administration was saying "around 50,000", and to me that is far more ridiculous and laughable than 1 million.
"Here is the exact question asked:
How many members of your household, if any, have died as a result of the conflict in Iraq since 2003 (ie as a result of violence rather than a natural death such as old age)? Please note that I mean those who were actually living under your roof."
It's good to see that the pollsters made a good attempt to clarify the "family" issue. I'm not really surprised because the extended concept of "family" is evident to anyone who has spent a few days in the Middle East. But I'm not sure this really clears up the "household" issue. In times of war, it is likely that many people will be forced to temporarily move in with relatives. Given the over two million internally displaced people in Iraq (7% of population), I think it's safe to say that there are quite a lot of people living with relatives. The question does not specify whether the person was living under the roof AT THE TIME OF DEATH. Thus, a person who lived in four different houses as a displaced person before getting killed by a car bomb could be counted more than once.
Obviously this is a much smaller effect than the "family" issue, but it would still inflate the numbers. My own opinion is that these number are probably a little high (+25%), and the official numbers are probably very low (-80%). The official numbers count news reports and statistics from morgues and hospitals. But a lot of dead bodies won't make it to the morgue due to Muslim requirements of a quick burial. Interestingly, many of those missing dead bodies probably showed up at the local mosque briefly before burial. But I don't know of anybody who keeps statistics on that. It might be a good place to look.
graeem, people exaggerate their churchgoing to express their support for the concept of going to church. I can certainly imagine Iraqis exaggerating their familial death toll to express (i) a complaint about the high level of violence and/or (ii) their resentment of the Americans or their own government for not preventing it.
Anybody that believes that approx. 5% of the population of the country of Iraq has been killed as a result of the war is smoking crack. 5%? 1.2 million people? My quick math estimates this would be 730/day. Come on, get a grip. When some Brit whacko group estimated 600K a year or two ago, most reasonable observers discounted that figure out of hand.
Some commenters wondered why Iraqis might exaggerate their claims. Perhaps the Coalition policy of monetary compensation is a factor. I agree with the over-counting due to familial/tribal notions of "family members".
When some Brit whacko group estimated 600K a year or two ago, most reasonable observers discounted that figure out of hand.
Are you talking about The Lancet study? If you are, you should know that "reasonable observers" doesn't mean what you seem to think it means.
The Lancet numbers and the latest numbers suggest that, for every coalition soldier who has been killed, about 200 civilians have been killed. That kind of ratio seems quite plausible to me.
The Lancet's latest wheeze is to campaign for a car-free London by 2030. These people just live in another world.
Of course, the 5% figure is ridiculous, especially when you consider that the number of injured would have to be 3-4 times greater. But I guess if people believe there's an African AIDS crisis in which some countries have a 10-30% infection rate, they'll believe almost anything.
Jim W - so in a month that we lose 60 soldiers the Iraqis have 12,000 killed? Come on. In a month that we lose 75 they lose 15,000? These figures require "a suspension of belief".
Anybody that believes that approx. 5% of the population of the country of Iraq has been killed as a result of the war is smoking crack
The estimates on refugees and internally displaced approaches 20% of the Iraqi population. How would you classify that number?
The poll was conducted by Opinion Research Business, or ORB, and their website has a summary of the findings.
From there you can follow links to files containing the raw data, which include a PDF and an Excel sheet.
Note that the standard margin of error based on the sample size is ±2.4%, so the actual estimated number of dead ranges from 1,191,286 to 1,249,873.
The summary also notes that this number exceeds the number of dead in the 1994 Rwanda genocide (that was 800,000), and that almost one in two households in Baghdad has lost a family member, which is the highest rate of any region in the country.
The raw data also includes results about the numbers of injured and refugees, which are apparently not reported in the LA Times. The summary at the ORB site says that more than a million have been injured, and that nearly half of the millions who have left their homes have left the country; most of these have gone to Syria.
Thomas - about right.
Not sure of the definition of "refugee" vs. "internally displaced. For arguments sake we will define displaced outside the country as a refugee.
The Lancet's latest wheeze...
The Lancet didn't do the study. Johns Hopkins did. Lancet just happened to be where it was published.
And the British government (among many others) took the study very seriously:
the Ministry of Defence's chief scientific adviser said the survey's methods were "close to best practice" and the study design was "robust". Another expert agreed the method was "tried and tested".
http://news.bbc.co.uk/1/hi/uk_politics/6495753.stm
Thanks Glenn. I just saw the text of the question at dailykos and came here to correct my comment. I see you were more prompt.
What JJ said. The Lancet is not a "group" it is a highly respected medical journal. The group which wrote the article is based in Johns Hopkins which is in Baltimore which is in Maryland which is in the USA (I don't know how clearly I have to explain things to people who think "The Lancet" is a whacko British group).
The new number is not what one would expect extrapolating the Hopkins numbers. The old numbers were total excess deaths not deaths from violence. I would assume that many were deaths from infectious diseases and malnutrition due to disruption of health care, food distribution, water purification and sewage treatment. 1.2 million violent deaths implies more total extra deaths due to the war.
I have some thoughts as to why IBC and AP numbers for deaths due to car bombs might be underestimates. I agree that they are unlikely to totally miss car bombings, but they might undercount the dead. First bodies might be buried under rubble. Increasing death counts are common (9/11 is an exception) as the count is usually bodies recovered not a guess. If reporters don't follow up recovery efforts related to the bomb that blew up the day before yesterday, Iraqi bodies might end up uncounted by IBC. Also it is standard to report dead and injured as exclusive categories on the assumption that someone who is injured by an explosion and still alive hours later will survive for years. Emergency care for Iraqi civilians is not up to first world standards, so there may be more deaths days to weeks after the bombing. Both arguments are that insufficient follow up implies that estimates based on media reports are biased down.
Still no way can I imagine a factor of 8 difference.
I think the most likely explanation is lying by respondents for mysterious reasons. Also some role might be played by selective non response (people with no deaths are less likely to participate in the survey) or selective response by people not contacted at random and should not be interviewed. (the neighbor comes over to say that someone in their household died)
I would assume that many were deaths from infectious diseases and malnutrition due to disruption of health care, food distribution, water purification and sewage treatment.
The estimate was 601,000 from violence and 54,000 otherwise.
It is my understanding that, based on the Hopkins numbers, extrapolated through (I think) June or July of this year, that at least one million Iraqis have died as a result of the war (though not necessarily by direct violence.) The US military was directly implicated in, IIRC, around thirty percent of the cases.
There is of course an actual range to those estimates but the point is the highest confidence level was about 600,000 or so in the Lancet study. The extrapolation of one million thus has similar high confidence.
It really doesn't matter if the poll is in error by 2.4% or 10%. The numbers are close enough for government work...
The fact is that the US invasion has killed a million people more than would have died under the continued existence of Saddam (assuming he didn't start another war with Iran in the near future - oh, wait, the US is going to do that anyway...)
And the US military is directly responsible for the deaths of probably close to one third of a million Iraqi civilians at this point.
Just wait until you see the civilian casualties figures for the "shock and awe" campaign in Iran...Estimates for the use of nuclear "bunker busters" are already predicting at least 10-50,000 civilian casualties and long-term death numbers across most of Iran, Afghanistan and elsewhere as a result of radiation dispersion as high as three million...
This is the option Obama doesn't want "off the table."
Bottom line: the US military are mass murderers whose leaders should be tried for the same war crimes as the Nazis in WWII.
... (I don't know how clearly I have to explain things to people who think "The Lancet" is a whacko British group) ...
It's a waste of time to try to explain anything at all to such a person.
It's worth pointing out, however, so that others will notice that the person in question is himself a whacko, and doesn't need to be taken seriously.
Where are the bodies?
When Saddam killed into six figures over ten years, he needed mass graves to get rid of all the bodies.
Since northern Iraq is peaceful, and most people are concentrated in urban zones, this number dead would leave the stink of decay permanently in the air in all cities. Since Baghdad represents about half the population of the war zone there would be about 600,000 dead in Baghdad. The disease from this would probably kill and sicken another half million. Ironically, the only way that level of disease could be prevented is via a highly-functioning civil society, disposing of the bodies and providing medical care to the wounded.
Oh yes, the wounded... where are they? If there are half a million dead from car bombs, there are another million wounded, huh?
It sucks over there, of this there is no doubt. But insane figures like this, and the Lancet as well, only cloud the debate. We must stay reality-based.
It may be that we are being realistic, but not believable, and the truth will come out some time in the future.
Even realism has an obligation to be realistic. --George Packer.
As someone alluded to above, this is a third world country, and a war zone at that. It could be that many people die from injuries that would be very treatable here. I wonder if anyone's done studies of how many new graves there are. Remember, this is a very dangerous place for western journalists, who would be the ones to give us first hand accounts of details like this.
Guys, you have to acknowledge you are simply in denial.
Give or take 10%, with the different studies slowly piling - and for chrissake stop denying their confidence intervals with nothing more than whim, they are not opinions but results- the truth is there: about 1 Mio death from violence so far, a third from American troops.
If you scale up the first number to the dimension of China,you get something like 25Mio, easy in Mao's territory for the higher estimates of the cultural revolution.
300.000 deaths from a foreign army: if you scale down to a country like Bosnia, you come to about 50.000. The war there is estimated to have caused about 100.000 deaths, concentration camps and war crimes included.
I understand: Were I an American, even an opponent of the war in Iraq like Drum or Yglesias, I would be in denial too.
But the faster you elaborate your mourning over a holier USA and act for stopping the absurdity, the better all of us are.
Anyone who seriously believes excess deaths in Iraq are greater than, say, 200,000 is not a serious person.
No, Aceed. Anyone who thinks they know with certainty who "a serious person" is, based on little or no knowledge of the studies (their methodology, who did them) is not a serious person.
Calling someone "not a serious person" just based on your gut, is not exactly name calling, but close.
You stupid little turd, I'll make it as easy for you to understand as possible. The Lancet report stated that over 80% of the respondents, who claimed they had family members who had been killed, were in possession of death certificates. So where is the official record of death certificates issued for this 80% of the extrapolated 600,000 or now 1.2 million? Nowhere to be seen, that's where. So save your crocodile tears for the deaths of people you don't actually give two hoots about.
You stupid little turd, I'll make it as easy for you to understand as possible.
There are alternatives to mouth breathing, dude.
As for your objection (as opposed to your objectionableness):
Death tolls from Iraqi health officials, the source of US government figures, are also suspect. The Johns Hopkins team says that the process for issuing death certificates still works well in Iraq, but the system for monitoring the number of certificates issued does not. Even before the war, note the researchers, the government's surveillance system captured only one-third of all deaths.
It is possible to walk on two appendages and spare your knuckles some wear and tear. Let me know when you make some progress.
The Johns Hopkins team says that the process for issuing death certificates still works well in Iraq, but the system for monitoring the number of certificates issued does not.
Well, they would say that, wouldn't they? Else their house of cards would collapse.
Even before the war, note the researchers, the government's surveillance system captured only one-third of all deaths.
Really? So does this mean the death toll under Saddam was three times higher than official figures suggest, meaning that excess deaths due to war are much lower, or did the John Hopkins team backdate their survey to include pre-war?
Interesting how the trolls come out of the woodwork for this.
Well, they would say that, wouldn't they? Else their house of cards would collapse.
Just calling it a house of cards doesn't make it one. Yes, they would say that because they are professional researchers and have evidence that this is the case, or they wouldn't make the claim. They don't call them emprical scientists for nothing. If they talk out of their *sses, their colleagues call them on it and it wrecks their careers.
...did the John Hopkins team backdate their survey to include pre-war?
That would be a pretty stupid mistake if they did. In that case, they probably wouldn't be Johns Hopkins epidemiological researchers. And the UK government scientists wouldn't be telling the Blair government that the methodology was sound.
By the way, here's a good article from Johns Hopkins' alumni magazine on the Lancet study:
http://www.jhu.edu/~jhumag/0207web/number.html
As the article discusses, the study has held up. And it's not like it hasn't gotten scrutiny. Tim Lambert over at Deltoid has catalogued the episodes of the controversy over the last couple years. Ideological crackpots, it seems, have a funny relationship with statistics:
http://scienceblogs.com/deltoid/lancetiraq/
Comments closed September 28, 2007.
| 
Heads in the Sand
I predict that the next rhetorical gambit of the right (if they deign to deal with humanitarian concerns at all) is to say that Iraqis are better off dead under the leadership of Bush than alive under the leadership of Saddam.
On second thought, I suppose its more likely that they'll take the route of "holocaust denial".
Posted by Jim W | September 14, 2007 12:47 PM