Journalist seeking scoops is as bad as scientist doing unreplicable research

Tom Scocca shares this dispiriting story:

Yesterday, as a news day, was an even worse cascade of lies and confusion and gibberish than usual. Yet what stood out the most was a single word: “Clarification.”

It appeared at the bottom of a very short Axios post by reporter Jonathan Swan, introducing a note that read, in full: “Clarification: This article and headline have been updated to add that it’s unclear whether the resignation offer has been accepted.”

The headline, at day’s end, was “Rod Rosenstein offered to resign.” It was still extremely debatable whether or not that claim . . . was true.

But here’s the kicker:

But either way, it was not what the item had said when it was first published. When it was published, what it said was, “Exclusive: Rod Rosenstein is resigning.” In a tweet, Swan had amplified and expanded it: “SCOOP: Rod Rosenstein has verbally resigned to John Kelly.”

The update—the point to be clarified—was that the SCOOP was false. . . .

So what happened yesterday was a crisis of government generated by the process of the New York Times trying to cover a crisis of government. . . . A whole flock of reporters was waiting around to see if Rosenstein lost his job; which one of them got the news first would not matter to the public at all. Everyone would replicate everyone else’s reporting in a matter of moments, and only the worst people would care who won.

And now, the key point:

Unless, that is, the win was not a win, and the story was not a story. What Axios passed off as an “update” was what an honest treatment would have called a “correction” or “retraction.” In a separate note, on a separate iteration of the coverage, Swan wrote:

I regret the way I wrote this morning’s version of the story. By saying Rosenstein had “verbally resigned” to Kelly rather than “offered his resignation,” I conveyed a certainty that this fluid situation didn’t deserve. It’s an important nuance, and I regret the wording.

Here “nuance” means “the entire substance of the story,” just as “clarification” meant “obfuscation.”

Why talk about this?

Why do I bring this up? Not to bang on Jonathan Swan, some dude I’d never heard of who happened to have a terrible, horrible, no good, very bad day.

I can’t even gather up the indignation to get mad at Swan and his editors for their lame pseudo-corrections. Everything happens fast in journalism, and good decision making can take time. It’s fine for me that Scocca got angry at his fellow scribblers here, but I don’t think it’s really my place to judge.

No, the reason I brought this up is because it reminds me so much of scientists who do work that doesn’t replicate: they’re out there, going for the scoop in PNAS or wherever, then when things don’t work out, they go back and change their story, rejiggering their hypothesis so that it is consistent with the latest random numbers that came their way, unwittingly demonstrating all the steps that Lakatos described of a degenerative research paradigm, and never giving up their stance that they are The Authority.

They’re so focused on getting the win—the scoop in Axios, the publication in the top-ranked journal and associated fawning publicity—but they’re forgetting the winner’s curse. The scoop that’s false, the research claim that is based on a bogus analysis and then, unsurprisingly, fails to replicate—that’s no win at all.

Don’t get me wrong: Taking risks can be good. In science, propose some outlandish theory and, who knows, it might end up working out! We won’t learn much if we stay too close to the path of the expected.

Mistakes are fine. Everybody makes mistakes. What’s important is to learn from them.

And I think it’s hard to learn from your mistakes if you don’t admit, to yourself and others, what your mistakes are.

In bad science, the mistake is not just that your idea didn’t pan out. Rather, the mistake is typically that your original study was too noisy, with poor theory and a weak connection between theory and data. Figure that out, and you can get somewhere.

Similarly, if you’re a journalist and you got a SCOOP that turned out to be wrong, try to figure out what went wrong. Did you just have an itchy trigger finger? Did you get played by someone planting a story? Whatever it is, take this as a learning experience. Recognize that the unexpected happened, and see what in your model was wrong, so that reality contradicted it.