Category: Decision Theory

Hey—take this psychological science replication quiz!

Rob Wilbin writes: I made this quiz where people try to guess ahead of time which results will replicate and which won’t in order to give then a more nuanced understanding of replication issues in psych. Based on this week’s Nature replication paper. It includes quotes and p-values from the original study if people want […]

The post Hey—take this psychological science replication quiz! appeared first on Statistical Modeling, Causal Inference, and Social Science.

“Identification of and correction for publication bias,” and another discussion of how forking paths is not the same thing as file drawer

Max Kasy and Isaiah Andrews sent along this paper, which begins: Some empirical results are more likely to be published than others. Such selective publication leads to biased estimates and distorted inference. This paper proposes two approaches for identifying the conditional probability of publication as a function of a study’s results, the first based on […]

The post “Identification of and correction for publication bias,” and another discussion of how forking paths is not the same thing as file drawer appeared first on Statistical Modeling, Causal Inference, and Social Science.

Bayesian model comparison in ecology

Conor Goold writes: I was reading this overview of mixed-effect modeling in ecology, and thought you or your blog readers may be interested in their last conclusion (page 35): Other modelling approaches such as Bayesian inference are available, and allow much greater flexibility in choice of model structure, error structure and link function. However, the […]

The post Bayesian model comparison in ecology appeared first on Statistical Modeling, Causal Inference, and Social Science.

“Richard Jarecki, Doctor Who Conquered Roulette, Dies at 86”

[relevant video] Thanatos Savehn is right. This obituary, written by someone named “Daniel Slotnik” (!), is just awesome: Many gamblers see roulette as a game of pure chance — a wheel is spun, a ball is released and winners and losers are determined by luck. Richard Jarecki refused to believe it was that simple. He […]

The post “Richard Jarecki, Doctor Who Conquered Roulette, Dies at 86” appeared first on Statistical Modeling, Causal Inference, and Social Science.

“Richard Jarecki, Doctor Who Conquered Roulette, Dies at 86”

[relevant video] Thanatos Savehn is right. This obituary, written by someone named “Daniel Slotnik” (!), is just awesome: Many gamblers see roulette as a game of pure chance — a wheel is spun, a ball is released and winners and losers are determined by luck. Richard Jarecki refused to believe it was that simple. He […]

The post “Richard Jarecki, Doctor Who Conquered Roulette, Dies at 86” appeared first on Statistical Modeling, Causal Inference, and Social Science.

Trapped in the spam folder? Here’s what to do.

[Somewhat-relevant image] It seems that some people’s comments are getting trapped in the spam filter. Here’s how things go. The blog software triages the comments: 1. Most legitimate comments are automatically approved. You write the comment and it shows up right away. 2. Some comments are flagged as potentially spam. About half of these are […]

The post Trapped in the spam folder? Here’s what to do. appeared first on Statistical Modeling, Causal Inference, and Social Science.

Trapped in the spam folder? Here’s what to do.

[Somewhat-relevant image] It seems that some people’s comments are getting trapped in the spam filter. Here’s how things go. The blog software triages the comments: 1. Most legitimate comments are automatically approved. You write the comment and it shows up right away. 2. Some comments are flagged as potentially spam. About half of these are […]

The post Trapped in the spam folder? Here’s what to do. appeared first on Statistical Modeling, Causal Inference, and Social Science.

Response to Rafa: Why I don’t think ROC [receiver operating characteristic] works as a model for science

Someone pointed me to this post from a few years ago where Rafael Irizarry argues that scientific “pessimists” such as myself are, at least in some fields, “missing a critical point: that in practice, there is an inverse relationship between increasing rates of true discoveries and decreasing rates of false discoveries and that true discoveries […]

The post Response to Rafa: Why I don’t think ROC [receiver operating characteristic] works as a model for science appeared first on Statistical Modeling, Causal Inference, and Social Science.

Response to Rafa: Why I don’t think ROC [receiver operating characteristic] works as a model for science

Someone pointed me to this post from a few years ago where Rafael Irizarry argues that scientific “pessimists” such as myself are, at least in some fields, “missing a critical point: that in practice, there is an inverse relationship between increasing rates of true discoveries and decreasing rates of false discoveries and that true discoveries […]

The post Response to Rafa: Why I don’t think ROC [receiver operating characteristic] works as a model for science appeared first on Statistical Modeling, Causal Inference, and Social Science.

Don’t call it a bandit

Here’s why I don’t like the term “multi-armed bandit” to describe the exploration-exploitation tradeoff of inference and decision analysis. First, and less importantly, each slot machine (or “bandit”) only has one arm. Hence it’s many one-armed bandits, not one multi-armed bandit. Second, the basic strategy in these problems is to play on lots of machines […]

The post Don’t call it a bandit appeared first on Statistical Modeling, Causal Inference, and Social Science.

Don’t call it a bandit

Here’s why I don’t like the term “multi-armed bandit” to describe the exploration-exploitation tradeoff of inference and decision analysis. First, and less importantly, each slot machine (or “bandit”) only has one arm. Hence it’s many one-armed bandits, not one multi-armed bandit. Second, the basic strategy in these problems is to play on lots of machines […]

The post Don’t call it a bandit appeared first on Statistical Modeling, Causal Inference, and Social Science.

When LOO and other cross-validation approaches are valid

Introduction Zacco asked in Stan discourse whether leave-one-out (LOO) cross-validation is valid for phylogenetic models. He also referred to Dan’s excellent blog post which mentioned iid assumption. Instead of iid it would be better to talk about exchangeability assumption, but I (Aki) got a bit lost in my discourse answer (so don’t bother to go […]

The post When LOO and other cross-validation approaches are valid appeared first on Statistical Modeling, Causal Inference, and Social Science.

When LOO and other cross-validation approaches are valid

Introduction Zacco asked in Stan discourse whether LOO is valid for phylogenetic models. He also referred to Dan’s excellent blog post which mentioned iid assumption. Instead of iid it would be better to talk about exchangeability assumption, but I (Aki) got a bit lost in my discourse answer (so don’t bother to go read it). […]

The post When LOO and other cross-validation approaches are valid appeared first on Statistical Modeling, Causal Inference, and Social Science.

“Seeding trials”: medical marketing disguised as science

Paul Alper points to this horrifying news article by Mary Chris Jaklevic, “how a medical device ‘seeding trial’ disguised marketing as science.” I’d never heard of “seeding trials” before. Here’s Jaklevic: As a new line of hip implants was about to be launched in 2000, a stunning email went out from the manufacturer’s marketing department. […]

The post “Seeding trials”: medical marketing disguised as science appeared first on Statistical Modeling, Causal Inference, and Social Science.

“Seeding trials”: medical marketing disguised as science

Paul Alper points to this horrifying news article by Mary Chris Jaklevic, “how a medical device ‘seeding trial’ disguised marketing as science.” I’d never heard of “seeding trials” before. Here’s Jaklevic: As a new line of hip implants was about to be launched in 2000, a stunning email went out from the manufacturer’s marketing department. […]

The post “Seeding trials”: medical marketing disguised as science appeared first on Statistical Modeling, Causal Inference, and Social Science.

How dumb do you have to be…

I (Phil) just read an article about Apple. Here’s the last sentence: “Apple has beaten earnings expectations in every quarter but one since March 2013.”
[Note added a week later: on July 31 Apple reported earnings for the fiscal third…

Parsimonious principle vs integration over all uncertainties

tl;dr If you have bad models, bad priors or bad inference choose the simplest possible model. If you have good models, good priors, good inference, use the most elaborate model for predictions. To make interpretation easier you may use a smaller model with similar predictive performance as the most elaborate model. Merijn Mestdagh emailed me […]

The post Parsimonious principle vs integration over all uncertainties appeared first on Statistical Modeling, Causal Inference, and Social Science.

Parsimonious principle vs integration over all uncertainties

tl;dr If you have bad models, bad priors or bad inference choose the simplest possible model. If you have good models, good priors, good inference, use the most elaborate model for predictions. To make interpretation easier you may use a smaller model with similar predictive performance as the most elaborate model. Merijn Mestdagh emailed me […]

The post Parsimonious principle vs integration over all uncertainties appeared first on Statistical Modeling, Causal Inference, and Social Science.