(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Anne Case (as quoted by Jesse Singal):
We spent a year working on this paper, sweating out every number, sweating out over what we were doing, and then to see people blogging about it in real time — that’s not the way science really gets done. . . . And so it’s a little hard for us to respond to all of the blog posts that are coming out. . . . And if this is all people shooting from the hip, I don’t think that’s any way to move science forward, to move the research forward.
Angus Deaton (as quoted by Kate Murphy):
There’s been a tremendous amount of recalculating of our numbers in the media and blogs. It’s all happening at the speed of light. You do worry whether it’s being done appropriately. But we’re not complaining.
“We’re not complaining”? That’s the best you can do???
How bout this:
We spent a year working on this paper, sweating out every number, sweating out over what we were doing, and we’re happy to see see people blogging about it in real time.
We very much appreciate the effort put in by Laudan Aron, Lisa Dubay, Elaine Waxman, and Steven Martin, Philip Cohen, and Andrew Gelman to uncover the aggregation bias in our analysis, to correct for that bias, and to explore subtleties that we did not have a chance to get into in our paper. As Gelman noted, these corrections are in no way a debunking of our work—our comparisons of non-Hispanic American whites to groups in other countries and other ethnic groups still stand.
We think it’s great that, after our paper was published in PNAS, it was possible to get rapid feedback. Had it not been for bloggers, we’d still be in the awkward situation of people trying to trying to explain an increase in death rates which isn’t actually happening. We join Paul Krugman and Ross Douthat in thanking these bloggers for their unpaid efforts on the behalf of everyone interested in this research. We count ourselves lucky to live in an era in which mistakes can be corrected rapidly, so that we and others do not have to wait months or even years for published corrections which themselves could contain further errors.
As economists, we recognize that research work is always provisional, and that anyone studying the real world of human interactions has to accept that mistakes are part of the process. It is only through the efforts of our entire research community—publishing in journals, publishing in blogs, through informal conversations, whatever—that we move toward the truth. We always considered our PNAS paper to be just a single step in this process and we are glad that others have taken the trouble to correct some of our biases and omissions.
Again, we thank the many researchers who have taken a careful look at our analyses. It’s good to know that our main findings are not affected by the corrections, we welcome further research in this area, and we hope that future discussion of our work, both in the scientific literature and in the popular press, make use of the corrected, age-adjusted trends.
– Sincerely, Anne Case and Angus Deaton
P.S. We have heard some people criticize the researchers noted above because they published their work in blogs rather than in peer-reviewed journals. We would never make such a silly, uninformed criticism. Since appearing in print, our work has received a huge amount of publicity. And, to the extent that we made mistakes or did not happen to explain ourselves clearly enough, it is the responsibility of others to publish their corrections and explanations as rapidly as possible. Blogs are a great way to do this. Blogs, unlike newspaper interviews, allow unlimited space to develop arguments and to present graphs of data. And we are of course aware that peer-reviewed journals make mistakes too. We published our paper in the Proceedings of the National Academy of Sciences, a journal that last year published a notorious paper on himmicanes and hurricanes, another discredited paper claiming certain behavior by people whose ages end in 9, and another paper on demographics which neglected to apply a basic age adjustment. So, yes, publication in journals is fine, but we very much welcome researchers who are willing to stick out their necks and correct the record in real time on blogs.
See, that wasn’t that hard!
Case and Deaton did great work. No need for them to get so defensive and attack-the-messenger-y about it. Indeed, I had an email exchange with Angus Deaton a few days ago and he was completely polite and reasonable, directly pointing me to the data set that they had used in their paper. In my opinion, Case and Deaton have acted as exemplary scientists in this entire episode, with only the very minor exception of a difficulty in graciously handling public corrections of their work. Hence my disappointment. Really only a small thing in the grand scheme of things, but it still bugs me, hence this post.
And, contra Case, this sort of give-and-take, whether it appears in blogs or wherever, is exactly the way science really gets done. Read your Lakatos.
P.P.S. Regarding the line above, “trying to trying to explain an increase in death rates which isn’t actually happening”: As I’ve emphasized throughout, the comparison to other countries remains interesting, as the age-aggregation bias is small compared to the observed declines of 20% in mortality rates in other countries and among other groups. But in the U.S. press there’s been a lot of explanation of why things are getting worse here, not just in a comparative but in an absolute scale. And, as the age-adjusted data show, the mortality rate in this group has been flat since 2005—increasing for women, decreasing for men, with an average change of just about zero. If people want to explain that, fine, but Case and Deaton have all of us to thank for the fact that people can now be focused on explaining this pattern, not an artifactual pattern of a steady increase in death rates which is what everyone was talking about before.
Or, as discussed in the comments, Case and Deaton don’t have to thank me, or Laudan Aron, Lisa Dubay, Elaine Waxman, and Steven Martin, or Philip Cohen, at all. But could they please refrain from slamming us? I’d like for Case and Deaton to show our work the same respect and consideration that we are showing theirs.
The post Why is it so hard for them to acknowledge a correction? appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
