(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
A Brooks op-ed in the New York Times (circulation approximately 1.5 million):
People at the extremes are happier than political moderates. . . . none, it seems, are happier than the Tea Partiers . . .
Jay Livingston on his blog (circulation approximately 0 (rounding to the nearest million)), giving data from the 2009-2010 General Social Survey, which is the usual place people turn to for population data on happiness of Americans:
The GSS does not offer “bitter” or “Tea Party” as choices, but extreme conservatives are nearly three times as likely as others to be “not too happy.”
Livingston reports that the sample size for “Extremely Conservative” here is 80. Thus the standard error for that green bar on the right is approx sqrt(0.3*0.7/80)=0.05.
So how could Brooks have made such a mistake? I can think of two possibilities:
1. Brooks has some other data source that directly addresses the happiness of supporters of the Tea Party movement.
2. Brooks looked at the cumulative GSS file from 1972, averaging over the last forty years, and attributed this aggregate pattern to the Tea Party, which has only existed since 2009. I’d hate to think Brooks did this, but he did mention the GSS in his op-ed, and Livingston does provide this helpful graph:
If you look at the cumulative file you indeed see some of the patterns Brooks discusses in his article. But the data during the Tea Party period look much different.
I wouldn’t be too hard on Brooks, though. He seems to have made a mistake, but I’ve made this sort of mistake myself. It can be tricky working with survey data, especially in a high-visibility, low-rigor place such as the NYT op-ed page. In contrast, when we publish in scholarly journals, we have many chances for revision and peer review (more chances to catch the mistakes) and not too many people read what we write in any case!
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science