Consider seniority of authors when criticizing published work?

August 11, 2017

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

Carol Nickerson writes:

I’ve written my fair share of letters to the editor and commentaries over the years, most of them languishing in the file drawer. It used to be impossible to get them published. The situation has improved a bit, but not enough.

In any case, I never think about the sex of the author(s). Or the race, for that matter. I do consider the seniority of the authors. When the lead author is junior — a graduate student, a post doc, a young assistant professor — I much prefer to bring problems to his or her attention using private e-mail. It’s more collegial and constructive. I know that many younger people in psychology have not had good statistical training at their colleges and universities. I like to attach relevant references of which he or she may not be be aware and so on. I’ve also done this with authors who are more senior. Interestingly, I have found that the junior authors are more likely to reply, more likely to initiate a corrigendum, etc. The senior authors are more likely to ignore my e-mail message.

The situation gets sticky when the lead author is a graduate student but the other author(s) are professors, and you know that said professors engage in a lot of questionable research practices or even research misconduct. Not too long ago, I discovered that an article published in Psychological Science re-used data from an article published earlier in another journal without so stating. In psychology, this is considered unethical. Of course, I had to alert the editor to this data re-use, but I worried a lot about the impact on the lead author (then a graduate student, now a post doc) and her career if the editor decided to retract the article. (He decided not to retract it, but to issue a corrigendum, which I think was the wrong decision.) Nick and I have had a lot of discussions about this dilemma. He thinks (a) that the student should know better and (b) that it is better for the student to have something like this happen earlier in his or her career, when recovery from such a setback is easier, than to have it happen later. I understand and have some sympathy with this view, but it still makes me feel terrible to go after a junior person.

I wonder if other people worry about this sort of thing. You could write a blog article about this sometime, Andrew, if you run out of other topics.

My reply: Let me break this up into three parts.

First, the question of contacting the author directly. I understand the appeal of this, but I usually don’t. Why not?

a. I don’t like conflict. It’s been my experience that when I do contact authors directly, they don’t like to admit error, and the conversation becomes awkward. When people send me their papers directly, I have no problem sending back by praise and criticism. But unsolicited criticism—and even unsolicited questions that are open-ended enough that they might imply criticism—that doesn’t always work so well.

b. There’s also a principle here, which is that published articles are . . . public. That’s what publishing means! So even if I can reach the authors directly, not everyone can, or will. I think there’s value in public comment on public papers. If people really don’t want their papers criticized in public, they shouldn’t go around publishing them and publicizing them.

Similarly, if you disagree with something I post here, please say so in a comment: that way others will see your reaction. It won’t just be me you’re talking to.

Second, the question of going easy on younger researchers. I don’t know about this. I published a false theorem when I was 28! I want that sort of thing corrected as soon as possible. One can also flip it around and ask, is it appropriate to be less nice to people, just cos they’re older?

I guess I’d like to think that it’s not mean to point out flaws in published work. Again, I’m happy for people to do this service for me, whether privately or publicly. I was grateful when someone sent me an airmail letter (!) with a counterexample to my false theorem, and I was grateful when someone wrote an angry blog post pointing out implausible estimates that I’d produced. In both cases, I would’ve been even happier if I’d done things right in the first place—but, conditional on the error, I appreciated the corrections.

I know that lots of you disagree with me on this issue, so by posting this I’m setting myself up for some criticism, but that’s fair. It’s good to get others’ perspectives.

The post Consider seniority of authors when criticizing published work? appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science