What America doesn’t want to know:
… when [Les] Roberts took on the challenge of tracking civilian casualties in Iraq, he was quickly reminded that, as with the use of sampling in the US census, statistical methodology can become highly politicized. He found his Iraq work misunderstood, misrepresented, even written off as propaganda. Lifting the fog of war, Roberts discovered, isn’t a question of finding the most accurate number, but one people are willing to accept.
In the 18 months after the American invasion, the numbers suggested, roughly 100,000 Iraqis had died as a result of the war, 60 percent of them violently. That dwarfed the figure from the widely cited website Iraq Body Count, which had tallied no more than 19,061 deaths by scouring press reports and official documents. The Iraqi government’s numbers were also much lower. The researchers sent their study to the prestigious medical journal The Lancet, which published it in October 2004.
The unexpectedly large death toll elicited skepticism, and questions about the methodology. The study had a wide “confidence interval” of 8,000 to 194,000. “This isn’t an estimate. It’s a dart board,” scoffed Slate military writer Fred Kaplan.
But leading epidemiologists and statisticians insist the study is valid. A confidence interval is structured like a bell curve, with the numbers in the bulging middle far more likely to be accurate than those at the tapering ends. It was a larger interval than Roberts and Burnham had hoped for—a consequence of their sample size and the uneven distribution of violence in Iraq. That didn’t render their estimate meaningless, however, just easy to dismiss. “I expected to be criticized,” says Roberts, who has since joined the public health faculty at Columbia University. “I was more struck by the lack of press coverage.”
In the spring and summer of 2006, the team’s researchers canvassed the country yet again, visiting more than 1,800 households clustered around 47 sites. As of that July, Roberts and Burnham would later estimate, the war had claimed about 655,000 Iraqi lives, suggesting that about 1 in 7 Iraqi families had lost someone because of the ongoing violence. As in the first study, there was a wide confidence interval—plus or minus about 275,000 deaths. But even the low end of the range suggested a death toll far beyond anything previously reported.
That October, after the new findings appeared in The Lancet, the critics pounced, again honing in on what they called fuzzy math. In a Wall Street Journal op-ed Steven E. Moore, a pollster and former adviser to Coalition Provisional Authority chief Paul Bremer, declared, “I wouldn’t survey a junior high school, no less an entire country, using only 47 cluster points.”
“That’s wrong,” says Jennifer Leaning, a professor at Harvard University’s School of Public Health. “You can sample very large populations with 33 clusters.” Other epidemiologists I contacted agreed.
The White House struck back with its own basic message: The study was bunk. Never mind that Roberts and Burnham had used methods similar to those employed for the Kosovo survey and others approvingly cited by the Bush administration. With the notable exception of This American Life producer Alex Blumberg, most reporters dutifully slapped Roberts’ research with the “controversial” label. And when asked about the study directly, President Bush declared that it had been “pretty well discredited.”
“By whom? By him and his political staff?” snaps Bradley Woodruff, who retired last year from his job as a senior cdc epidemiologist. Woodruff has conducted mortality surveys himself, and considers Roberts’ research solid. But when cbs‘s 60 Minutes sought to interview Woodruff about the Lancet study in 2007, the cdc wouldn’t allow it. And when Rep. Dennis Kucinich invited Woodruff to Washington to discuss the study, his bosses nixed that, too. “I never had this kind of censorship under previous administrations,” he says.
[M]ore than two years later, the Iraq study remains mired in controversy. But other recent findings suggest that Roberts and Burnham were on the right track. In the summer of 2006, the World Health Organization conducted a large family health survey along with Iraq’s Ministry of Health, interviewing about five times as many people as Roberts and Burnham had, and in a more distributed fashion. In August, Mohamed Ali, a who statistician, reported his preliminary results to colleagues at a Denver statistics conference: Nearly 397,000 Iraqis had died because of the war as of July 2006.
[W]hile epidemiologists and statisticians are still pondering questions raised by differences between the two surveys, there’s no longer much doubt among them that Iraq’s civilian casualties number in the hundreds of thousands.
This grim statistic continues to elude most Americans. According to a February 2007 AP poll, Americans’ median estimate of the number of Iraqis killed since the invasion was just 9,890. And while the Pentagon has presented limited estimates of civilian casualties, it has yet to release any numbers for the total toll since the invasion.
Roberts had set out to provide a legitimate number that might be used to inform public policy. For now, at least, that policy has been to keep the truth buried in academic journals—and beneath the sands of Iraq.
Read the whole thing at MoJo, here