Category: Decision Theory

Should we be concerned about MRP estimates being used in later analyses? Maybe. I recommend checking using fake-data simulation.

Someone sent in a question (see below). I asked if I could post the question and my reply on blog, and the person responded: Absolutely, but please withhold my name because this is becoming a touchy issue within my department. The boldface was in the original. I get this a lot. There seems to be […]

The post Should we be concerned about MRP estimates being used in later analyses? Maybe. I recommend checking using fake-data simulation. appeared first on Statistical Modeling, Causal Inference, and Social Science.

My talk tomorrow (Tues) noon at the Princeton University Psychology Department

Integrating collection, analysis, and interpretation of data in social and behavioral research Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University The replication crisis has made us increasingly aware of the flaws of conventional statistical reasoning based on hypothesis testing. The problem is not just a technical issue with p-values, not can […]

The post My talk tomorrow (Tues) noon at the Princeton University Psychology Department appeared first on Statistical Modeling, Causal Inference, and Social Science.

“She also observed that results from smaller studies conducted by NGOs – often pilot studies – would often look promising. But when governments tried to implement scaled-up versions of those programs, their performance would drop considerably.”

Robert Wiblin writes: If we have a study on the impact of a social program in a particular place and time, how confident can we be that we’ll get a similar result if we study the same program again somewhere else? Dr Eva Vivalt . . . compiled a huge database of impact evaluations in […]

The post “She also observed that results from smaller studies conducted by NGOs – often pilot studies – would often look promising. But when governments tried to implement scaled-up versions of those programs, their performance would drop considerably.” appeared first on Statistical Modeling, Causal Inference, and Social Science.

“Using numbers to replace judgment”

Julian Marewski and Lutz Bornmann write: In science and beyond, numbers are omnipresent when it comes to justifying different kinds of judgments. Which scientific author, hiring committee-member, or advisory board panelist has not been confronted with page-long “publication manuals”, “assessment reports”, “evaluation guidelines”, calling for p-values, citation rates, h-indices, or other statistics in order to […]

The post “Using numbers to replace judgment” appeared first on Statistical Modeling, Causal Inference, and Social Science.

Watch out for naively (because implicitly based on flat-prior) Bayesian statements based on classical confidence intervals! (Comptroller of the Currency edition)

Laurent Belsie writes: An economist formerly with the Consumer Financial Protection Bureau wrote a paper on whether a move away from forced arbitration would cost credit card companies money. He found that the results are statistically insignificant at the 95 percent (and 90 percent) confidence level. But the Office of the Comptroller of the Currency […]

The post Watch out for naively (because implicitly based on flat-prior) Bayesian statements based on classical confidence intervals! (Comptroller of the Currency edition) appeared first on Statistical Modeling, Causal Inference, and Social Science.

“Statistical and Machine Learning forecasting methods: Concerns and ways forward”

Roy Mendelssohn points us to this paper by Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos, which begins: Machine Learning (ML) methods have been proposed in the academic literature as alternatives to statistical ones for time series forecasting. Yet, scant evidence is available about their relative performance in terms of accuracy and computational requirements. The purpose […]

The post “Statistical and Machine Learning forecasting methods: Concerns and ways forward” appeared first on Statistical Modeling, Causal Inference, and Social Science.

What does it mean to talk about a “1 in 600 year drought”?

Patrick Atwater writes: Curious to your thoughts on a bit of a statistical and philosophical quandary. We often make statements like this drought was a 1 in 400 year event but what do we really mean when we say that? In California for example there was an oft repeated line that the recent historic drought was […]

The post What does it mean to talk about a “1 in 600 year drought”? appeared first on Statistical Modeling, Causal Inference, and Social Science.

The AAA tranche of subprime science, revisited

Tom Daula points us to this article, “Mortgage-Backed Securities and the Financial Crisis of 2008: A Post Mortem,” by Juan Ospina and Harald Uhlig. Not our usual topic at this blog, but then there’s this bit on page 11: We break down the analysis by market segment defined by loan type (Prime, Alt-A, and Subprime). […]

The post The AAA tranche of subprime science, revisited appeared first on Statistical Modeling, Causal Inference, and Social Science.