(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Following a recent email exchange regarding path sampling and thermodynamic integration (sadly, I’ve gotten rusty and haven’t thought seriously about these challenges for many years), a correspondent referred to the marginal distribution of the data under a model as “the evidence.”
I hate that expression! As we discuss in chapter 6 of BDA, for continuous-parametered models, this quantity can be completely sensitive to aspects of the prior that have essentially no impact on the posterior. In the examples I’ve seen, this marginal probability is not “evidence” in any useful sense of the term.
When I told this to my correspondent, he replied,
I actually don’t find “the evidence” too bothersome. I don’t have BDA at home where I’m working from at the moment, so I’ll read up on chapter 6 later, but I assume you refer to the problem of the marginal likelihood being strongly sensitive to the prior in a way that the posterior typically isn’t, thereby diminishing the value of the marginal likelihood as a model selection statistic for problems in which there is no well motivated way to choose the prior. If so, I understand, but I think you might be fighting a losing battle as “the evidence” is seemingly now popular in the stats literature as well as the physics …
I replied that I’ll fight that battle forever. I really really hate the use of linguistically-loaded terms such as “bias,” “evidence,” “empirical Bayes,” etc.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science