Rejection is Good for Citation (maybe)

A paper in Science covering over 80 thousand articles in 923 scientific journals finds that rejected papers are ultimately cited more than first acceptances.

We compared the number of times articles were cited (as of July 2011, i.e., 3 to 6 years after publication, from ISI Web of Science) depending on their being first-intents or resubmissions. We used methods robust to the skewed distribution of citation counts to ensure that a few highly cited articles were not driving the results. We controlled for year of publication, publishing journal (and thus impact factor), and their interaction. Resubmissions were significantly more cited than first-intents published the same year in the same journal.

The author’s argue that the most likely explanation is that peer review increases the quality of manuscripts. Rejection makes you stronger. That’s possible although the data are also consistent with peers being more likely to reject better papers!

Papers in economics are often too long but papers in Science are often too short. Consider the paragraph I quoted above. What is the next piece of information that you are expecting to learn? How many more citations does a resubmission receive! It’s bizarre that the paper never gives this number (as far as I can tell). Moreover, take a look at the figure (at right) that accompanies the discussion. The authors say the difference in citations is highly significant but on this figure (which is on log scale) the difference looks tiny! This figure is taking up a lot of space. What is it saying?

So what’s the number? Well if you go to the online materials section the authors still don’t state the number but from a table one can deduce that resubmissions receive approximately 7.5% more citations. That’s not bad but we never learn how many citations first acceptances receive so it could be less than 1 extra citation.

There’s something else which is odd. The authors say that about 75% of
published articles are first-submissions. But top journals like Science and Nature reject 93% or more of submissions. Those two numbers don’t necessarily contradict. If everyone submits to Science first or if everyone never resubmits to Science then 100% of papers published in Science will be a first submission. Nevertheless, the 93% of papers that are rejected at top journals (and lower ranked journals also have high rejectance rates) are going somewhere so for the system as whole 75% seems implausibly high.

Econ papers sometimes exhaust me with robustness tests long after I have been convinced of the basic result but Science papers often leave me puzzled about basic issues of context and interpretation. This is also puzzling. Shouldn’t more important papers be longer? Or is the value of time of scientists higher than economists so it’s optimal for scientists to both write and read shorter papers? The length of law review articles would suggest that lawyers have the lowest value of time except that doesn’t seem to be reflected in their consulting fees or wages. There is a dissertation to be written on the optimal length of scientific publications.