(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
I happened to come across this old post today with this amazing, amazing quote from a Harvard University public relations writer:
The replication rate in psychology is quite high—indeed, it is statistically indistinguishable from 100%.
This came up in the context of a paper by Daniel Gilbert et al. defending the reputation of social psychology, a field that has recently been shredded—and rightly so—but revelations of questionable research practices, p-hacking, gardens of forking paths, and high-profile failed replications.
When I came across the above quote, I mocked it, but in retrospect I think it hadn’t disturbed me enough. The trouble was that I was associating it with Gilbert et al.: those guys don’t know a lot of statistics so it didn’t really surprise me that they could be so innumerate. I let the publicist off the hook on the grounds that he was following the lead of some Harvard professors. Harvard professors can make mistakes or even be wrong on purpose, but it’s not typically the job of a Harvard publicist to concern himself with such possibilities.
But now, on reflection, I’m disturbed. That statement about the 100% replication rate is so wrong, it’s so inane, I’m bothered that it didn’t trigger some sort of switch in the publicist’s brain.
Consider the following statements:
“Harvard physicist builds perpetual motion machine”
“Harvard biologist discovers evidence for creationism”
That wouldn’t happen, right? The P.R. guy would sniff that something’s up. This isn’t the University of Utah, right?
I’m not saying Harvard’s always right. Harvard has publicized the power pose and all sorts of silly things. But the idea of a 100% replication rate, that’s not just silly or unproven or speculative or even mistaken: it’s obviously wrong. It’s ridiculous.
But the P.R. guy didn’t realize it. If a Harvard prof told him about a perpetual motion machine or proof of creationism, the public relations officer would make a few calls before running the story. But a 100% replication rate? Sure, why not, he must’ve thought.
We have a ways to go. We’ll always have research slip-ups and publicized claims that fall through, but let’s hope it’s not much longer that people can claim 100% replication rates with a straight face. That’s just embarrassing.
P.S. I have to keep adding these postscripts . . . I wrote this post months ago, it just happens to be appearing now, at a time in which we’re talking a lot about the replication crisis.
The post We have a ways to go in communicating the replication crisis appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
