What do people do wrong? WSJ columnist is looking for examples!

December 26, 2012

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

Carl Bialik of the Wall Street Journal writes:

I’m working on a column this week about numerical/statistical tips and resolutions for writers and people in other fields in the new year. 2013 is the International Year of Statistics, so I’d like to offer some ways to better grapple with statistics in the year ahead. Here’s where you come in. If you have time in the next couple of days, please send me an idea or three about what people often do incorrectly when it comes to numbers, and how they could do better, without making things much more complicated. Maybe with an example of something you’ve seen that rubbed you the wrong way, and how you’d fix it. Bonus if you can tie it in to some sort of statistic that will be particularly relevant in 2013.

Any ideas of yours I use, I’ll credit, of course.

Examples of what I have in mind:

–Don’t report on how ubiquitous or important something is by saying there are 130,000 Google search results for it. Or if you do, at least check that you typed the query in a way that it’s only finding relevant results; and if you’re using the data to make the argument it’s becoming trendy, compare it to how many results there were a year ago, and compare that growth to the typical growth in search results. Or, much better, find a better data point.

–Don’t, on a controversial issue, find a study from one advocacy group on one side of the issue, a study from a group on the other, mention both and consider your work done. Look for research from less biased sources, and see what independent researchers think of the body of work.

–When two candidates are separated in the polls by a margin less than the statistical margin of error, don’t say they’re statistically tied. Especially if one candidate is leading the other in nearly every poll.

–Check your numbers, and then check them against a smell test. I’ve mixed up millions and billions, too, but if I’d looked twice I would have realized there can’t be 11 billion people in Ohio. Hopefully this conveys the idea.

My deadline is Thursday, 9 a.m. Eastern time.

Oddly enough, I can’t think of any good examples, nor can I think of any good suggestions beyond generic advice such as, “Just because something is counterintuitive, it doesn’t mean it’s true,” “Don’t trust anything written by Gregg Easterbrook,” and, of course, the ever-popular, “Hey—I don’t like that graph!” Maybe you can come up with something better for the readers of the Wall Street Journal?

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science