My talk at the Brookings Institution this Fri 11am

The replication crisis in science: Does it matter for policy?

Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University

I argue that policy analysts should care about the replication crisis for three reasons: (1) High-profile policy claims have been systematically exaggerated; (2) This has implications for how to conduct and interpret new research; (3) Much of the work that has been called into question is attached to a manipulable-voter model that has malign political implications.

The replication crisis is typically discussed in the context of particular silly claims, or in terms of the sociology of science, or with regard to controversies in statistical practice. We can also consider the content of unreplicated or otherwise shaky empirical claims in political science, which often seem to be associated with a model in which attitudes and behavior can be easily manipulated using irrelevant stimuli. This set of theories, if true, would have important implications for politics, supporting certain views held on the left, right, and technocratic center of the political spectrum. Conversely, the lack of empirical support for the manipulable-voter model has political implications which are worth considering: if voters and politicians are not so easily swayed in this way, this suggests that we should try to more carefully understand their direct motivations.