What are the most common errors that reviewers make when reviewing health papers for you?
There are three errors that reviewers make. First, many junior reviewers write really long reviews to show that they were thorough. This doesn’t help—if the paper has 8 problems then the editor is often most interested in the top two.
Second, some reviewers can also have really high standards in a way that creates lots of Type II errors—never accepting a paper. At the Review of Economics and Statistics, we were writing to accept more papers, but reviewers made this hard by using an impossible standard for identification.
Finally, and this is rare, but a by-product of the “triple-aim” (described above): some reviewers write reports with innuendo and meanness—I never went back to them and still think very poorly of these individuals. To be mean, when protected by the veil of an anonymous review process, is a deep pathology.
My advice is: write short reviews—don’t over referee or rewrite the paper—you are the reviewer, not the author. Be kind. Be kind. Be kind. Kindness is not the same as low standards, but posing questions and raising challenges with curiosity and humility. Always remember that an editor is reading the review, sharing it with other editors, and one’s nastiness is noted and remembered especially when directed towards a new member of the profession.
That is from an interview with Amitabh Chandra of ReStat.