In an e-mail to Smartertimes.com, the New York Times offers the following in response to yesterday's Smartertimes.com questioning the Times's unattributed claim that Nevada "has the nation's highest rate of asthma":
"fyi, you cited the wrong cdc report.
here's the one with the information. Nevada is listed as having the greatest estimated percentage on a state-by-state basis -- 7.2 percent -- though admittedly it's based on self-reported cases of asthma, and on census figures now out of date.
http://www.cdc.gov/epo/mmwr/preview/mmwrhtml/00055803.htm
we had some of the same questions and revised the story for later editions, one you clearly did not receive.
hope this helps.
cheers"
Well, this is progress. The Times has now done what it didn't do for its readers -- cite the exact source for its claim that Nevada has "the nation's highest rate of asthma." The editor of Smartertimes.com eagerly clicked through to the report cited by the Times at the link above -- only to find a federal Centers for Disease Control and Prevention report titled "Forecasted State-Specific Estimates of Self-Reported Asthma Prevalence -- United States, 1998."
Aha. Now, this may be a concept beyond the comprehension of the editors at the New York Times, but "forecasted" asthma "estimates" are not the same as asthma rates. The CDC study the Times now cites doesn't actually count asthma cases -- it interviews a regional sample and estimates how many cases a state would have if the characteristics of the sample held true for the state. In the statistics talk of the Times-cited study: "Using methods that have been applied elsewhere to forecast cancer rates, state-specific asthma prevalence estimates for 1998 were calculated using a three-step procedure: 1) race-, sex-, and age-specific asthma prevalence rates were calculated for each of the four U.S. census regions using data from the 1995 National Health Interview Survey (NHIS); 2) each state's 1998 demographic composition as estimated by the Bureau of Census was multiplied by the corresponding regional prevalences; and 3) linear extrapolations of region-specific increases in asthma prevalence from 1980 to 1994 were applied to the 3-year period from 1995 to 1998 for each state."
And as the CDC report the Times now cites makes clear, "The findings in this report are subject to at least two limitations." The most important of these is that "these results are based on the assumption that age, sex, and race-specific rates of asthma do not vary within any of the four geographic regions of the United States. Each state's estimated prevalence reflects its regional placement in the United States and its demographic composition. These analyses do not account for differences among states in the relative presence or absence of environmental risk factors in asthma prevalence, possible differences in genetic susceptibility toward the condition, or other sociodemographic indicators (e.g., poverty status). As a result, these findings underestimate the variability in asthma prevalence between states within regions."
Even if the Times wants to claim that these "forecasted estimates" are the same as an asthma rate, there's another problem with that claim that Nevada has "the nation's highest rate of asthma." As the CDC study the Times cites in its own defense states plainly, "Differences in asthma prevalence rates between states were not significant. By region, 1-year period prevalence estimates ranged from 6.4% to 6.8% in the Northeast, 5.8% to 6.1% in the South, 6.6% to 6.7% in the Midwest, and 6.0% to 7.2% in the West." Here's how the New York Times, on September 28, 2000, interpreted a Los Angeles Times poll that showed George W. Bush with a six-point lead over Al Gore among likely voters: "since the margin of sampling error is plus or minus four percentage points, the findings have to be considered a statistical tie." (The four points apply to both Mr. Bush's support and to Mr. Gore's, so the six points are within the eight-point range.) The CDC forecast itself says the state-to-state differences within regions were "not significant." The forecasted rates are close enough that, if it were a political poll showing a Republican in the lead, the Times would probably call it a "statistical tie."
And the New York Times claims that Smartertimes.com cited the "wrong report"?
In fact, unlike the report the Times cites, the one Smartertimes.com cited yesterday, at http://www.cdc.gov/epo/mmwr/preview/mmwrhtml/00052262.htm, is not based on "estimates" or "forecasts" but on actual reports of causes of death. That report shows a much greater state-to-state variance in death from asthma, and it doesn't support the claim that Nevada "has the nation's highest rate of asthma."
So, just to sum up, the Times has taken "forecasted estimates" that a study says are "not significant" and turned them into the claim that Nevada "has the nation's highest rate of asthma." It didn't tell its readers about another study showing that several states have higher rates of actual reported deaths from asthma. And when challenged on the point, the newspaper's response is not to run a correction, but to tell Smartertimes.com, "you cited the wrong cdc report." The whole episode is just a wonderful little case study in the Times's carelessness and arrogance.