(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Dan Gianola pointed me to this one. It’s an article by Ming-Jin Liu, Cai-Hua Xiong, Le Xiong, and Xiao-Lin Huang with the innocuous title, “Biomechanical Characteristics of Hand Coordination in Grasping Activities of Daily Living,” and a boring abstract:
Hand coordination can allow humans to have dexterous control with many degrees of freedom to perform various tasks in daily living. An important contributing factor to this important ability is the complex biomechanical architecture of the human hand. However, drawing a clear functional link between biomechanical architecture and hand coordination is challenging. It is not understood which biomechanical characteristics are responsible for hand coordination and what specific effect each biomechanical characteristic has. To explore this link, we first inspected the characteristics of hand coordination during daily tasks through a statistical analysis of the kinematic data . . .
Which all of a sudden takes a surprising twist:
The explicit functional link indicates that the biomechanical characteristic of tendinous connective architecture between muscles and articulations is the proper design by the Creator to perform a multitude of daily tasks in a comfortable way.
Whoa! I didn’t see that one coming.
It’s a bit sad, actually. God used to run the world, he’d throw down thunderbolts and smite people, then he was in charge of your immortal soul, and now, he’s reduced to . . . providing comfort! The God of the Ergonomic Gaps, as it were. A real step down for the guy.
Plos-One papers are allowed Reader Comments, and this one has 44 already: “Improper” . . . “Shameful” . . . “this paper should be retracted immediately” . . . “Alas, it’s not a hoax” . . . “Horrifying” . . . “Unacceptable in a scientific journal” . . . “This paper should be retracted immediately” . . . you get the picture.
Also a hilariously inept bit of backpedaling by one of the paper’s authors:
Our study has no relationship with creationism. . . . We will change the Creator to nature in the revised manuscript. . . .
One reason I’m suspicious of this claim is because of this list of subject areas of the paper:
“Built structures,” huh?
From my perspective, the paper is more amusing than bothersome, I guess because Plos-One isn’t one of the journals that I publish in. If something comparable were published in the American Political Science Review or the Journal of the American Statistical Association, I’d be pretty annoyed.
One advantage of Plos-One is that it identifies the editor for each paper. This one is listed as Renzhi Han, Ohio State University Medical Center. So I googled and found this:
Wow, a whole lab named after himself! That’s pretty impressive already. . . . Another search brings us to this post on Retraction Watch, where Han says he’s sorry this happened. And, in an update, Gianola reports that the paper has been retracted.
I guess the editor of the paper didn’t read all the way down to the end of the abstract! I guess he was busy managing the “Han Lab”—that must take a lot of time!
Larger questions
So this particular case is pretty boring: someone slipped in some Creator stuff in an otherwise utterly boring paper submitted to a journal that will publishes anything (which is not a bad thing; it’s good to have journals such as Plos-One that can serve an archival purpose without requiring that each paper be some sort of “breakthrough”), the editor accepts the paper without reading it (indeed, without even reading the abstract, it seems), there’s an uproar, the paper gets retracted.
More interesting, perhaps, is the idea that creationism is out of bounds in scientific discourse. Actually, I do think a creationist paper could be published in a mainstream science journal such as Plos-One. It’s just that they’d actually have to make an argument for their point. Scientific arguments for creationism have been made before, and they’ve been found wanting, but if someone wants to try again, I think they’d be able to get published. After all, medical journals have published articles on the purported effects of intercessory prayer (tl;dr: p=.04, garden of forking paths, failed replications, etc; basically power pose without the Ted talk, the book contract, and the Harvard publicity), so it’s not like there’s some ban on publishing science-based speculations on the supernatural. The problem with the recent Liu et al. paper is that they just slipped in some mentions to the Creator without ever making a case for it.
The closest thing to a creationist event in statistics publication was the notorious publication in 1994 in the journal Statistical Science of a paper on the so-called Bible Code, yet another effort in multiple comparisons that hit the bestseller list, using the generosity of the scientific publishing process to give it undeserved credibility. Unfortunately for the Bible Code dudes, Ted talks didn’t exist back in 1994 so they weren’t able to fully duplicate the success of power pose. They did get mentioned in NPR, though.
How did Statistical Science end up publishing a paper with such low statistical quality (even setting aside the scientific implausibility of their being such a “code”)? Three things, I think:
1. Back in 1994 we were much less aware of how easy it was to obtain statistical significance via researcher degrees of freedom in data processing and analysis. Back then, we knew about the file-drawer effect and we knew about multiple comparisons, but there was much less awareness of how many multiple potential comparisons are in any analysis. The editors of Statistical Science were insufficiently attuned to the garden of forking paths (and I too didn’t recognize the importance of this phenomenon in trying to interpret empirical claims).
2. Technical details. The Bible Code paper was full of stuff about ancient Hebrew. Who knows anything about that? So it’s simplest to take the authors’ word for everything and assume they’re accurately describing their data and what they did.
3. Bending over backward to be fair. Should a scientific journal reject a paper just because it’s about the supernatural? We want to avoid suppressing unorthodox ideas. It’s a tough call. My take on this: Yes, we want to avoid suppressing unorthodox ideas. But we should not give such papers “the benefit of the doubt.”
The problem with the Bible Code paper was not just in its lack of scientific content. It was also crappy from a statistical perspective. And these two things go together: if you want to scientifically prove something that isn’t true, you have to cheat (or leave yourself enough wiggle room to be able to make a bunch of mistakes in your favor) somewhere. It’s no coincidence that a paper purporting to prove a crappy theory will have crappy statistics. Good statistics wouldn’t do the job here! Only crappy statistics will bring you over the finish line. And, in retrospect, the reviewers of the Bible Code paper should’ve been able to notice the crappy statistics.
But, again, back in 1994 we were much less aware of the havoc being wreaked by null hypothesis significance testing coupled with uncontrolled researcher degrees of freedom. Back in 1994, Uri Simonsohn was still in high school [college, actually — ed.]!
One more time
And it happened again! I’m referring, of course, to the 2011 publication of that ESP paper by Daryl Bem in top psychology journal JPSP.
Same story, same script: a supernatural theory, researcher degrees of freedom all over the place, technical mumbo-jumbo, and a desperate desire on the part of the editors not to censor the unorthodox.
And, I like what I wrote just above so I’ll repeat it here:
The problem with Bem’s ESP paper was not just in its lack of scientific content. It was also crappy from a statistical perspective. And these two things go together: if you want to scientifically prove something that isn’t true, you have to cheat (or leave yourself enough wiggle room to be able to make a bunch of mistakes in your favor) somewhere. It’s no coincidence that a paper purporting to prove a crappy theory will have crappy statistics. Good statistics wouldn’t do the job here! Only crappy statistics will bring you over the finish line. And, in retrospect, the reviewers of that paper should’ve been able to notice the crappy statistics.
But, again, back in 2011 we were much less aware of the havoc being wreaked by null hypothesis significance testing coupled with uncontrolled researcher degrees of freedom. Back in 2011, Uri Simonsohn was still an assistant professor!
More examples
Did I mention power pose yet? Himmicanes? Fat arms and political attitudes? Monthly cycle and voting? Same story, just without the overtly supernatural elements.
P.S. We seem to have been getting a lot of spam from patriotic people using fake names. You guys should take it up with the editor of the paper who described it as a mistake, and the editors of Plos-One who retracted it, stating, “This evaluation confirmed concerns with the scientific rationale, presentation and language, which were not adequately addressed during peer review. Consequently, the PLOS ONE editors consider that the work cannot be relied upon and retract this publication.” Their call, not mine.
P.P.S. Title changed to respect the possibility that phrases such as “the proper design by the Creator to perform a multitude of daily tasks in a comfortable way” do not represent creationist views but rather are just poor translations into English.
The post Creationist article Article with creationist language published in Plos-One appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
