Preemptive Review Comments

February 14, 2013

(This article was originally published at Carlisle Rainey » Methods/Statistics, and syndicated at StatsBlogs.)

The most recent issue of PS arrived in my mailbox this morning and it has an interesting article called "Rookie Mistakes," which provides a long list of comments that peer reviewers repeatedly make about manuscripts.

My pet-peeve almost made the list at number 39.

Lack of statistical significance indicates that there is not enough evidence to reject the null hypothesis of no effect, so inferences based on the direction of nonstatistically significant coefficients should be offered cautiously or not at all.

I would elaborate the point and say that researchers should not interpret insignificant coefficients as "no effect" either. I've talked about that plenty, so I'll just point non-regular readers to this post and this paper.

Point 17 seems a little weird.

Hypotheses should be numbered consecutively, such as H1, H2, and H3, instead of being named with number-letter combinations, such as H1a, H1b, and H2, because number-letter combinations foster confusion: H2 is the third hypothesis in the aforementioned example.

I can't imagine that the number-letter combinations always foster more confusion that than simple numbering. (Though I did once read a paper with an H1, H1a, and H2, but no  H1b. That strategy left me scratching my head.) The reader won't remember numbers or number-letter combinations, so it seems like we should just get rid of numbering hypotheses altogether. If a hypothesis isn't important enough to have a descriptive name, perhaps it's not worth the reader's time.

Point 57 really stepped on my toes.

The manuscript should not contain errors in grammar, spelling, or punctuation; these errors indicate that the writing of the manuscript was not conducted carefully and suggests that the reported research might not have been conducted carefully, either.

I find typos in papers that I've edited dozens of times. It never ends.

Please comment on the article here: Carlisle Rainey » Methods/Statistics