(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Andy Solow writes:
I have a question about Bayesian statistics. Why is it wrong to use the same data to formulate the prior and to update it to the posterior? I am having a hard time coming up with – or finding in the literature – a formal reason.
I asked him to elaborate and he wrote:
Here’s an example. Independent observations X1, X2, …, Xn from the uniform distribution on (0, U). Interested in the posterior distribution of U. Likelihood is 1/U**n for U > xmax, 0 otherwise. Exponential prior for U with parameter log 2/xmax so that prob(U < xmax) = 1/2. Apart from lacking justification, the prior depends on the data.
I replied: Yes, in general you don’t want the prior to depend on the data. When prior is expressed to depend on data, we think of it as an approx to a prior distribution that depends on some unknown parameter that is estimated from the data. So the idea is that there is a coherent model that is being approximated for convenience.
For another example, consider regression models with standardized predictors, as discussed in my 2008 paper, “Scaling regression inputs by dividing by two standard deviations.” If you rescale predictors based on the data, then put informative priors on their coefficients (as we recommend in our other 2008 paper), then you have a prior that depends on data.
I’d like to think of this as an approximation to a prior that depends on a hyperparameter—the population sd of the predictor in question—which in turn is estimated from data using a simple point estimate. Thus, an approximation to a hierarchical model.
Now that Stan is available it would make sense to set up the full hierarchical model, partly for practical purposes when sample sizes are small, partly because the full model will be easier to extend if we want to go to other scalings, and partly as a demonstration so that if we do use the approximate approach, we’ll know what we’re approximating.
Maybe we can put this in our Bayesian econometrics text. I’m thinking of an intro chapter, or maybe an appendix, with all the basic regression models with different priors.
The post Data-dependent prior as an approximation to hierarchical model appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
