Publish Referee Reports?

A number of prominent scientists working on stem-cell research have written a letter calling for peer-reviews to be made publicly available:

We suggest a simple step that would greatly improve transparency, fairness and accountability; when a paper is published, the reviews, response to reviews and associated editorial correspondence could be provided as Supplementary Information, while preserving anonymity of the referees.

In cutting edge fields such as stem-cell research it's very common for scientists to be working on exactly the same problem with the first to publish receiving the bulk of the plaudits.  As a result, peer-sabotage can be a problem.  Sabotage is not such a problem in economics because researchers rarely work on exactly the same problem but laziness and low-quality reviews are real concerns.  (Note that my experiences with journal editors and reviewers has been more good than bad so I don't speak with sour grapes).

But would publishing anonymous referee reports really increase quality?  Blog comments are public and anonymous and they can be stupid, rude, and ill-informed (not this blog of course).  Indeed, the trend in blogs has been to remove anonymity as a way of increasing accountability and quality.  In science, however, anonymity is essential because the opportunities for repeat play and thus collusion are too common.

The primary effect of published referee reports would be on editors who would have their work put under greater scrutiny.  That is not necessarily a bad thing although editors are professionally under-rewarded in my view so more work is not necessarily going to lead to a better selection.

Note also that regardless of any change in quality, the value of the referee comments themselves should not be overlooked.  Referees may also appreciate the opportunity to have their work published in some form. Early results from the EMBO Journal which recently switched to publishing referee reports online appear to be satisfactory. Thus, I favor experiments along these lines. 

Here are previous MR articles on the peer-review process.

Addendum: Seth Roberts comments. See also Barkley Rosser in the comments.


Peer review is the rock that academia is built on, and also its Achilles's heal (to invoke two mixed metaphors). Climategate is going to rock peer review in disciplines that easily affect public policy.

Why not? It sounds like a great idea to me -- there's no accountability anyway for poor peer review, so it won't hurt the situation, and by creating some institutional accountability it might lead the journal to select its peer reviewers more carefully.

While we're doing institutional reform, why not pay the referees while we're at it?

Oh, and 'sour grapes'? It's kind of odd that a bad experience makes you a less credible critic. That speaks to one of my theories of why stuff sucks, because it is run by the people who like it the way it is. The rest of us are just whiners.

Good idea.
You could also make it interactive: use Googledocs as the platform and let peer reviewers edit, mark up the document with comments; set a timeline for comments. You could either let other reviewers see other reviewer's comments, or segregate them.
When the document is published, there could be a edited peer review comment section, along with a blogger section for people to comment on the articles.
We've moved away from paper, and people should look at using technology to organize and speed up the process.

I could be wrong, but I think the scooping aspect is overblown. It seems to me that the 'working paper' approach to economics might actually reduce 'scooping' by staking claim to a specific question. It is counterintuitive because it is very odd considering that there is little capital involved in economics papers. Is there a lot more lead-time in finding datasets and sources?

Otherwise there is less scooping in economics DESPITE putting your ideas out for public view. Odd? It seems there are even more variations in biology so that knowing what others are working on would reduce overlap.

I relay the following comment:

* * * * * * * * * *

[Nursing Research has] been doing this for selected manuscripts since 2004. Select "Open Review". It doesn't seem to take too much time for the editorial office, and the site seems to be getting more use as word spreads.

Nursing Research staff also periodically assess the quality of reviews, and publish the findings:

Blog posts are anonymous, but public anonymous peer reviews are not anonymous to the editor(s). In addition, people posting on blogs don't make a living on what they write there. I think these two factors combined make a crucial difference. It would also make editors more accountable.

How can anyone think that blog comments are comparable to peer review? For starters, on the one hand, in a submitted article undergoing the referee process, the author and the reviewers actually know what they are talking about. On the internet, commenters rarely have any relevant expertise (and bloggers often don't, either, though that is not the case here).

There is probably a market there, v.

There may be a case for this in some the biomedical field, but I do not see it spreading
in economics, and I as a journal editor will not be doing it.

1) Some potential referees may refuse to ref for one, and we have enough trouble getting
people to do so (and on time) as it is.

2) It seems that what a lot of people want is to "punish" bad referees or editors for
turning down papers that should not have been, revenge for all those lousy negative ref
reports they have gotten. But this is only going to apply to papers that are accepted,
so we get to see people demanding inappropriately to be cited with the authors doing so.
It is completely out of the question that journals will "publish" rejected papers with
the accompanying referee reports. No way.

I had to run before. Another reason it might not be a good idea is that it may put pressure on
editors to publish only papers that all the referees approve of. Some of the most cited papers
I have published I have done so over the objections of either a referee or even an associate
editor. I am not for putting incentives in place to support more publication of boring mediocrity.

Thanatos points to what editors need to do when they encounter referees who promise to review but sit on the paper: get other referees. Of course, this is not always as easy as it sounds, but there are ways around some of the more egregious forms of referee sabotage and hanky-panky besides publishing the reports of referees on papers that get published, which strikes me as doing nearly zero.

In economics, peer-sabotage isn't really that much of an issue, I would think. There is always a twist or two different in otherwise similar papers and both will usually survive.

There is, however, an issue in the review lag length. There is no excuse for leaving papers in review for more than two years, often only to reject them. (Or to request such an extensive revise for resubmit that the author gives up.) This is not uncommon even for relatively uncontroversial papers that were widely cited in working paper form.

Seth did get around to posting my comments, just for the record.

「自分自身を発見し、理想を貫く強い意志を持つこと」。new balance1918年、ボストンに小さな店をオープンしたティンバーランドの創業者airmax 95 、ネイサン・シュワーツが残したシンプルな企業理念は、アメリカのみならず、ティンバーランドのその名が世界的に知られるまでになった現在でも、経営陣から社員一人ひとりにまで浸透していますtimberland ブーツ。ティンバーランドには利益を上げることと同様に力を傾けている大きな目的があります。それは、より多くの方に新しい自分を発見していただけるような物を作ること。たとえば、大自然に囲まれての野外活動において、その愉しさを周囲の人々と分かち合えば、その喜びはきっと増すでしょう。timberland 6インチそうした目的を実現させるための商品作りにティンバーランド全員が取り組んでいます。
また、1989年、あるコミュニティ活動組織にブーツを50足寄付したことから始まり、ティンバーランドは現在までさまざまな社会貢献活動に携わってきました。そして今、timberland 6インチそれらの活動は世界中の人々とティンバーランドのスタッフがつながる重要な手段となり、さらには、私たちと協力することでお客様の社会貢献活動を実現していただくという理想的な関係が保たれています。ものづくりだけに執着することなく、顧客、社会との関わりを深めることで、ティンバーランドがどのような企業体なのか理解していただけると私たちは信じているのですtimberland 6インチ。

Nowadays,more and more people tend to buy things online,nike shox shoes are no exception.
When you search the cheap nike shox supplier online, you can find hundreds of websites.The price can be quite cheaper,shox running shoes,but the most concern for you maybe:how to choose a reliable Nike Shox R4 shoes supplier?
First,you need to make sure is the site you choose has a refund policy,shox shoe.A lot of sites do not have any sort of a guaranteed policy and this makes it seems unsafe,shox shoes.It can be risky buying these kinds of shoes.So be careful when choosing a website.
And another important thing is the customer service on the site,women shox shoes.Are they offering real-time customer service?Is there any e-mail address or telephone number that you can get in touch with them if there is any problem?
There is another thing that you need to compare with is the price,shox basketball shoes.There are thousands of sites available to you.I checked out some sites online and did a price comparison and there is no doubt that it is different.It's all about looking around and seeing which site has the cheapest price.

Tiffany not only explored the various jewelry processes of the time, but also branched out into new metals, such as platinum, which at the time was considered very hard to manipulate.Silver TiffanyIt seems to be the case that unusual colorations appealed to Tiffany, like the opal.He also preferred gemstones that were either opaque or translucent.Tiffany Jewelry Turquoise, jade, carnelian, lapis, moonstones, and opals were all chosen for their ability to filter light. Emphasis based on color was very prevalent in his works.Tiffany Earrings

In any case, more news is sure to pour out for Madden NFL and EA Sports in general. Peter Moore is definitely excited about the potential for Social Games and has gone on record many times about it. Stay tuned for more! ethical issues in nursing

You can take help by following my favorite site

I really like this write! I enjoy it so much! thanks for give me a good reading moment!

You are so generous! I just love finding beautiful and artistic freebies like this. They really brighten up my day, thank you!


good post

Newark airport , JFK airport(John F Kennedy Airport newyork) , Laguardia airport is our preferred source of clients, we want to fine tune our business to get more and more clients from these places, or client whom are looking to have some ride to go airports listed above

Hey friend, if i say i buy one cheap air max, air max 90, nfl jerseys authentic fron the website only $41.11,do you want to believe?

it is really a good website, they say all their products are authentic, i like the products ghd purple,

and christian louboutin, at first i dont believe that, just have a try,

and now, when i receive the products i am so happy, for their good quality and best service.

i take my air max 95 and nike shox just only 3 days for shipping. and it is really a nice style.

thanks so much.

hi its seems like you have done a hard work on it. I have got lots of information from your post. Really appreciate your work.!! It was describe very nicely keep us doing good work..

Comments for this post are closed