“16 and Pregnant”

May 12, 2018

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

Ted Joyce writes:

In December 2015 the AER published an article, “Media Influences on Social Outcomes: The Impact of MTV’s 16 and Pregnant on Teen Childbearing,” by Melissa Kearney and Phil Levine [KL]. The NBER working paper of this article appeared in January of 2014. It received huge media attention as the authors claimed the show was responsible for 25% of the decline in teen childbearing from 2008-2010. . . .

Joyce and his colleagues [David Jaeger, Robert Kaestner] were skeptical:

So we buy the Nielsen data and we acquire the confidential birth certificate data and we have a go. Attached is the second version of that effort. . . .

In brief, KL are trying to identify the effect of a nationally broadcast program on teen fertility amidst large secular trends and at the nadir of the Great Recession. There is no time variation in the show across units and their IV strategy lacks anything resembling an exogenous instrument. Even a DD interpretation fails the parallel trend assumption. We feel it was their use of Twitter data and Google searches that captured the imagination of the reviewers, but here too we show all their social media results collapse as soon as we use all THEIR data and not just a selected sample.

The paper by Jaeger, Joyce, and Kaestner is called, “Did Reality TV Really Cause a Decline in Teenage Childbearing? A Cautionary Tale of Evaluating Identifying Assumptions,” and here’s their key claim:

We find that controlling for differential time trends in birth rates by a market’s pre-treatment racial/ethnic composition or unemployment rate cause Kearney and Levine’s results to disappear, invalidating the parallel trends assumption necessary for a causal interpretation. Extending the pre-treatment period and estimating placebo tests, we find evidence of an “effect” long before 16 and Pregnant started broadcasting.

I have not had a chance to read any of these papers in detail so I’m just presenting the controversy here without any endorsement (or anti-endorsement) of Jaeger, Joyce, and Kaestner’s argument.

When these sorts of things come up on the blog, some readers feel a bit cheated, that I’m bringing up this live issue and not giving my own take on it. All I can say is that some takes are easier to come by than others, and I think there’s value in all sorts of posts. Sometimes I can do a careful investigation of my own or follow a debate closely enough to have an informed opinion, sometimes I can share a controversy and let the experts chew on it. Science is full of uncertainty and turmoil and it’s not so bad to sometimes present a scientific dispute without trying to resolve it myself.

Also, this one’s about causal inference, which is one of our core blog topics.

The post “16 and Pregnant” appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

Tags: , ,