Is scholarly refereeing productive at the margin?

No, basically:

In economics many articles are subjected to multiple rounds of refereeing at the same journal, which generates time costs of referees alone of at least $50 million. This process leads to remarkably longer publication lags than in other social sciences. We examine whether repeated refereeing produces any benefits, using an experiment at one journal that allows authors to submit under an accept/reject (fast-track or not) or the usual regime. We evaluate the scholarly impacts of articles by their subsequent citation histories, holding constant their sub-fields, authors’ demographics and prior citations, and other characteristics. There is no payoff to refereeing beyond the first round and no difference between accept/reject articles and others. This result holds accounting for authors’ selectivity into the two regimes, which we model formally to generate an empirical selection equation. This latter is used to provide instrumental estimates of the effect of each regime on scholarly impact.

That is from a new NBER paper by Aboozar Hadavand, Daniel S. Hamermesh, and Wesley W. Wilson.  This is exactly the kind of work — critical, data-driven self-reflection about science — what Progress Studies wishes to see more of.

Comments

Comments for this post are closed