Bayesians are frequentists

Bayesians are frequentists. What I mean is, the Bayesian prior distribution corresponds to the frequentist sample space: it’s the set of problems for which a particular statistical model or procedure will be applied.

I was thinking about this in the context of this question from Vlad Malik:

I noticed this comment on Twitter in reference to you. Here’s the comment and context:

“It’s only via significance tests that model assumptions are checked. Hence wise Bayesian go back to them e.g., Box, Gelman.” and

While I’m not qualified to comment on this, it doesn’t sound to me like something you’d say. With all the “let’s report Bayesian and Frequentist stats together” talk flying around, I’m curious where statistical significance does fit it for you.

Necessary evil or, following most of the comments on your Abondon Stat Sig post, something not so necessary? My layman impression was that you’d fall into the “do away with it” camp.

Is stat sig necessary to “evaluate a model”? Perhaps I misunderstand the terminoloy, but my thinking is that experience/reality is the only thing that evaluates a model ie. effect size, reproducibility, usefulness… I see uses for stat sig as one “clue” or indicator, but I don’t see how stat sig helps check any assumptions, given it’s based on fairy big assumptions.

My reply:

Actually that quote is not a bad characterization of the views of myself and my collaborators. As we discuss in chapters 6 and 7 of BDA3, there are various ways to evaluate a model, and one of these is model checking, comparing fitted model to data. This is the same as significance testing or hypothesis testing, but with two differences:
(1) I prefer graphical checks rather than numerical summaries and p-values;
(2) I do model checking to test the model that I am fitting, usually not to test a straw-man null hypothesis. I already know my model is false, so I don’t pat myself on the back for finding problems with the fit (thus “rejecting” the model); rather, when I find problems with fit, this motivates improvement to the model.

By the way, I followed the above links and they were full of ridiculous statements such as “scientists will never accept invented prior distrib’s”—which is kind of a shocker to me as I’ve been publishing scientific papers with “invented prior distrib’s” for nearly 30 years now! But I guess people will say all sorts of foolish things on twitter.

The post Bayesians are frequentists appeared first on Statistical Modeling, Causal Inference, and Social Science.