Posts Tagged ‘ Decision Theory ’

Chasing the noise in industrial A/B testing: what to do when all the low-hanging fruit have been picked?

June 16, 2018
By

Commenting on this post on the “80% power” lie, Roger Bohn writes: The low power problem bugged me so much in the semiconductor industry that I wrote 2 papers about around 1995. Variability estimates come naturally from routine manufacturing statistics, which in semicon were tracked carefully because they are economically important. The sample size is […] The post Chasing the noise in industrial A/B testing: what to do when all…

Read more »

About that quasi-retracted study on the Mediterranean diet . . .

June 15, 2018
By

Some people asked me what I thought about this story. A reporter wrote to me about it last week, asking if it looked like fraud. Here’s my reply: Based on the description, there does not seem to be the implication of fraud. The editor’s report mentioned “protocol deviations, including the enrollment of participants who were […] The post About that quasi-retracted study on the Mediterranean diet . . . appeared…

Read more »

Forking paths come from choices in data processing and also from choices in analysis

June 5, 2018
By

Michael Wiebe writes: I’m a PhD student in economics at UBC. I’m trying to get a good understanding of the garden of forking paths, and I have some questions about your paper with Eric Loken. You describe the garden of forking paths as “researcher degrees of freedom without fishing” (#3), where the researcher only performs […] The post Forking paths come from choices in data processing and also from choices…

Read more »

Comments on Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection

June 5, 2018
By

There is a recent pre-print Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection by Quentin Gronau and Eric-Jan Wagenmakers. Wagenmakers asked for comments and so here are my comments. Short version: They report a known limitation of LOO when it’s used in a non-recommended way for model selection. They report that their experiments show that […] The post Comments on Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection appeared first…

Read more »

Ambiguities with the supposed non-replication of ego depletion

June 4, 2018
By

Baruch Eitam writes: I am teaching a seminar for graduate students in the social track and I decided to dedicate the first 4-6 classes to understanding the methodological crises in psychology, its reasons and some proposed solutions. In one of the classes I had the students read this paper which reports an attempt to reproduce […] The post Ambiguities with the supposed non-replication of ego depletion appeared first on Statistical…

Read more »

Against Screening

June 3, 2018
By

Matthew Simonson writes: I have a question that may be of interest to your readers (and even if not, I’d love to hear your response). I’ve been analyzing a dataset of over 100 Middle Eastern political groups (MAROB) to see how these groups react to government repression. Observations are at the group-year level and include […] The post Against Screening appeared first on Statistical Modeling, Causal Inference, and Social Science.

Read more »

Some experiments are just too noisy to tell us much of anything at all: Political science edition

May 29, 2018
By
Some experiments are just too noisy to tell us much of anything at all:  Political science edition

Sointu Leikas pointed us to this published research article, “Exposure to inequality affects support for redistribution.” Leikas writes that “it seems to be a really apt example of “researcher degrees of freedom.'” Here’s the abstract of the paper: As the world’s population grows more urban, encounters between members of different socioeconomic groups occur with greater […] The post Some experiments are just too noisy to tell us much of anything…

Read more »

Garden of forking paths – poker analogy

May 21, 2018
By

[image of cats playing poker] Someone who wishes to remain anonymous writes: Just wanted to point out an analogy I noticed between the “garden of forking paths” concept as it relates to statistical significance testing and poker strategy (a game I’ve played as a hobby). A big part of constructing a winning poker strategy nowadays […] The post Garden of forking paths – poker analogy appeared first on Statistical Modeling,…

Read more »

Slow to update

May 11, 2018
By

This post is a placeholder to remind Josh Miller and me to write our paper on slow updating in decision analysis, with the paradigmatic examples being pundits being slow to update their low probabilities of Leicester City and Donald Trump in 2016. We have competing titles for this paper. Josh wants to call it, “The […] The post Slow to update appeared first on Statistical Modeling, Causal Inference, and Social…

Read more »

Should Berk Özler spend $2 million to test a “5 minute patience training”?

May 10, 2018
By

Berk Özler writes: Background: You receive a fictional proposal from a major foundation to review. The proposal wants to look at the impact of 5 minute “patience” training on all kinds of behaviors. This is a poor country, so there are no admin data. They make the following points: A. If successful, this is really […] The post Should Berk Özler spend $2 million to test a “5 minute patience…

Read more »


Subscribe

Email:

  Subscribe