Philosophy and the practice of Bayesian statistics (with all the discussions!)

February 7, 2013

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

My article with Cosma Shalizi has appeared in the British Journal of Mathematical and Statistical Psychology. I’m so glad this paper has come out. I’d been thinking about writing such a paper for almost 20 years. What got me to actually do it was an invitation a few years ago to write a chapter on Bayesian statistics for a volume on the philosophy of social sciences. Once I started doing that, I realized I had enough for a journal article. I contacted Cosma because he, unlike me, was familiar with the post-1970 philosophy literature (my knowledge went only up to Popper, Kuhn, and Lakatos). We submitted it to a couple statistics journals that didn’t want it (for reasons that weren’t always clear), but ultimately I think it ended up in the right place, as psychologists have been as serious as anyone in thinking about statistical foundations in recent years.

Here’s the issue of the journal, which also includes an introduction, several discussions, and a rejoinder:

Prior approval: The growth of Bayesian methods in psychology (pages 1–7)
Mark Andrews and Thom Baguley

Philosophy and the practice of Bayesian statistics (pages 8–38)
Andrew Gelman and Cosma Rohilla Shalizi

How to practise Bayesian statistics outside the Bayesian church: What philosophy for Bayesian statistical modelling? (pages 39–44)
Denny Borsboom and Brian D. Haig

Posterior predictive checks can and should be Bayesian: Comment on Gelman and Shalizi, ‘Philosophy and the practice of Bayesian statistics’ (pages 45–56)
John K. Kruschke

The error-statistical philosophy and the practice of Bayesian statistics: Comments on Gelman and Shalizi: ‘Philosophy and the practice of Bayesian statistics’ (pages 57–64)
Deborah G. Mayo

Comment on Gelman and Shalizi (pages 65–67)
Stephen Senn

The humble Bayesian: Model checking from a fully Bayesian perspective (pages 68–75)
Richard D. Morey, Jan-Willem Romeijn and Jeffrey N. Rouder

Rejoinder to discussion of ‘Philosophy and the practice of Bayesian statistics’(pages 76–80)
Andrew Gelman and Cosma Shalizi

The basic themes are laid out by Mark Andrews and Thom Baguley in their introduction:

Bayesian methods are not just another set of topics in advanced statistics such as, for example, structural equation modeling or nonlinear regression. For some, they represent a new paradigm (in the Kuhnian sense of term) for the field. As such, their increasing adoption has potentially profound implications for the nature and practice of data analysis in psychology, possibly affecting everything from the editorial policies of journals to how statistics is taught to students.

Despite their growing appeal, however, there remains a troubling lack of clarity about what exactly Bayesian methods do and do not entail and about how they differ from their so-called classical counterparts. Bayesian methods are often portrayed as being based on a subjective rather than frequentist interpretation of probability, with inference being an updating of personal beliefs in light of evidence. In practice, however, most modern applications of Bayesian methods to real-world data analysis problems are characterized by pragmatism and expediency: Bayesian methods are adopted because they promise (and arguably often deliver) solutions to important or difficult problems.

They continue:

As we see it, the choice of priors is like the choice of the probabilistic model of the data. For example, given a set of observations . . . we might model this data as . . . The choice of this probabilistic model need not be a reflection of our true beliefs about how this data was generated. Rather it can be seen as literally just a model that can potentially provide insight into the nature and structure of the data. By the same reasoning, the priors . . . need not be a reflection of our true beliefs about the parameters, but are just part of our general modelling assumptions. Just as the generative model provides a probabilistic model of the data, the priors provide a probabilistic model of parameters. Just as we assume that our data is drawn from some probability distribution with fixed but unobserved parameters, so too we assume that the values of the parameters are drawn from another probability distribution (also with fixed but unobserved parameters). . . . One reason for a degree of humility in our analysis is that no probability model and hence no statistical model in psychology is complete.

The punchline:

Priors, therefore, are just assumptions of our model. Like any other assumptions, they can be good or bad and may need to be extended, revised or possibly abandoned on the basis of their suitability to the data being studied.

P.S. Two other, less formal versions of our argument (in the Frey-Arrow style) are here and here.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science