(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
I happened to recently reread this article of mine from 2010, and I absolutely love it. I don’t think it’s been read by many people—it was published as one of three discussions of an article by Brad Efron in Statistical Science—so I wanted to share it with you again here.
This is the article where I introduce three meta-prinicples of statistics:
The information principle: the key to a good statistical method is not its underlying philosophy or mathematical reasoning, but rather what information the method allows us to use. Good methods make use of more information.
The methodological attribution problem: the many useful contributions of a good statistical consultant, or collaborator, will often be attributed to the statistician’s methods or philosophy rather than to the artful efforts of the statistician himself or herself. I give as examples Rubin, Efron, and Pearl.
Different applications demand different philosophies, demonstrated with a discussion of the assumptions underlying the so-called so-called false discovery rate.
Also this:
I also think that Efron is doing parametric Bayesian inference a disservice by focusing on a fun little baseball example that he and Morris worked on 35 years ago. If he would look at what is being done now, he would see all the good statistical practice that he naively (I think) attributes to “frequentism.”
As I get older, I too rely on increasingly outdated examples. So I can see how this can happen but we should be aware of it.
Also this:
I also completely disagree with Efron’s claim that frequentism (whatever that is) is “fundamentally conservative.” One thing that “frequentism” absolutely encourages is for people to use horrible, noisy estimates out of a fear of “bias.”
That’s a point I’ve been banging on a lot recently, the idea that people make all kinds of sacrifices and put their data into all sorts of contortions in order to feel that they are using a rigorous, unbiased procedure.
And:
as discussed by Gelman and Jakulin (2007) [here], Bayesian inference is conservative in that it goes with what is already known, unless the new data force a change. In contrast, unbiased estimates and other unregularized classical procedures are noisy and get jerked around by whatever data happen to come by—not really a conservative thing at all.
Back in 2010 I didn’t know about the garden of forking paths, but since then I’ve seen, over and over, how classical inference based on p-values and confidence intervals (or, equivalently, noninformative priors) leads to all sorts of jumping to conclusions. Power pose, embodied cognition, himmicanes: these are all poster boys for the anticonservatism of classical statistics and the potential of Bayesian inference to bring us to some sort of sanity.
And this comment on “frequentism” more generally:
Of course, frequentism is a big tent and can be interpreted to include all sorts of estimates, up to and including whatever Bayesian thing I happen to be doing this week—to make any estimate “frequentist,” one just needs to do whatever combination of theory and simulation is necessary to get a sense of my method’s performance under repeated sampling. So maybe Efron and I are in agreement in practice, that any method is worth considering if it works, but it might take some work to see if something really does indeed work.
“It might take some work to see if something really does indeed work”: I like that. Clever and also true, and also it leaves an opening for statistical theory and, for that matter, frequentism: we want to see if something really does indeed work. Or, to put it more precisely, to understand the domains in which a method will work, and to also understand where and how it will fail.
Lots more good stuff in this 4-page article. As the saying goes, read the whole thing.
The post Bayesian Statistics Then and Now appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
