(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Demetri Spanos writes:
I bumped into your paper with John Carlin, Beyond Power Calculations, and encountered your concept of the hypothetical replication of the point estimate. In my own work I have used a similarly structured (but for technical reasons, differently motivated) concept which I have informally been calling the “consensus posterior.”
Specifically, supposing a prior distribution for the true effect size D, I observe the experimental data and compute the posterior belief over values of D. Then I consider the “replication thought experiment” as follows:
1. Assuming my *posterior* as the distribution for true value of D …
2. Consider the posterior distribution that another researcher would obtain if they performed the identical experiment, assuming they hold the same prior but that the true distribution of D is as my posterior distribution
3. I then get a distribution over posterior distributions that I think of as the set of beliefs other researchers might hold, given they start from the same assumptions and have the same experimental capability that I do. Then I can calculate various point values from this “consensus” distribution. As I’m sure is clear to you, the consensus distribution is always much wider and less spiky than even the semi-smoothed distributions arising from a weakly-informative prior.
4. In my field (automated control systems, distributed computing, sensor fusion, and some elements of machine learning), I have found this provides a much more stable and well-conditioned signal for triggering automated sensor-driven behaviors than conventional techniques (even conventional Bayesian techniques). We are especially sensitive in our work to multimodal distributions and especially relative magnitudes of local peaks (because we often want to trigger a complex response or set of responses, not just get one average-case point estimate).
5. perhaps the most important
Now, having just read your paper, the variant I describe above seems “obvious,” so I wonder if you have thought on this subject before and could point me to any additional resources. Or, perhaps, if you see some fatal flaw that I have missed I would appreciate being told about that as well. “Works in the field” doesn’t necessarily mean “sound in principle,” and I would like to know if I am treading somewhere dangerous.
My reply: This could work, but much depends on how precise is your posterior distribution. If your data and your prior are weak, you could still have problems with your distribution being under-regularized (that is, being too far from zero).
The post Boostrapping your posterior appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
