On-line comments?

by on September 14, 2006 at 2:05 pm in Education | Permalink

Nature, one of the world’s most prestigious scientific research journals, has embarked on an experiment of its own.

In addition to having articles submitted for publication subjected to peer reviews by a handful of experts in the field, the 136-year-old journal is trying out a new system for authors who agree to participate: posting the paper online and inviting scientists in the field to submit comments praising — or poking holes — in it.

Lay readers can see the submitted articles as well, but the site says postings are only for scientists in the discipline, who must list their names and institutional email addresses.  Nature says its editors screen out those they find irrelevant, intemperate or otherwise inappropriate.

Meanwhile, the papers also make their way through the journal’s traditional peer-review gauntlet.  Nature says it’s taking both sets of comments into account when deciding whether to publish.

So far, there have been only 70 posts on the 62 papers that authors have decided to put on the Web site, according to Linda Miller, U.S. executive editor of Nature, published by a unit of Macmillan Publishers Ltd.

The experiment (at http://blogs.nature.com/nature/peerreview/trial/) is an early sign of how the scientific publishing establishment is pushing the limits of its hallowed but opaque peer-review system.  Critics of the traditional process say it lets not only low-quality papers, but also sometimes fraudulent ones, slip through the gates.  The Nature trial of a Web-based system could usher the spirit of Wikipedia, the online encyclopedia edited by its readers, from the margins of scientific publishing to the mainstream.

That is part of an article by the excellent Nicholas Zamiska, in today’s Wall Street Journal, p.B1.  I have long wanted to see this happen, but I fear for economics, and perhaps other fields, it will not work.  The simple problem is that not enough people care about the results of enough papers to start a fruitful dialogue.  Still, at least opening up the papers for on-line comments seems to be a simple free lunch, at least for the world if not always for the reputation of the authors and journal editors.

Comments are on this post are…open…

Thomas Mayer September 14, 2006 at 2:25 pm

Let me suggest a modification. Journals should set up a web page for
comments on published papers. Sometimes when reaing a paper I see something
I think is a mistake, But even if the journal publishes comments, they need to be formal and elaborate, and usually I don’t have the time for more than a fairly quicj informal
comment.

Ivan September 14, 2006 at 2:51 pm

There is a difference between opaque and double-blind. Peer-reviews should be the latter.

Also, considering the marginal cost of online publication is zero, journals should all start blogs and exist by donation only. They should also open up and digitize their library.

I can’t tell you how many times I avoided paying for a paper on IEEE Explore by searching ‘” ” filetype:pdf’ and finding results buried on the author’s university site someplace.

Note that there is a very new and very successful robotics conference that has higher standards because of a double blind paper acceptance system. Robotics: Science and Systems
http://www.roboticsconference.org/

My only complaint is that RSS makes for a google-confusing acronym.

David Zetland September 14, 2006 at 10:19 pm

rvman has a good point, but good papers may NOT deserve comments. Better to have votes AND comments (a la Digg, which also allows votes ON comments).

I also beleive that papers without interest should be buried as worthless (I’ve written a paper on it “An auction market for journal articles”) but note that academics will be violently opposed to having their work judged in a market-setting. They prefer to freeride on the reputation of a journal. Those who point to work that cites theirs cannot claim value either — readers don’t actually care about the topic as much as other researchers. (Citations are loosely, but not strongly correlated with value…)

pam59 September 15, 2006 at 3:50 am

it is very common that journals are set up on page
and are open for comments. why not dot it? good
thing about it is you can personlize it and it is
also catchy.

bob September 16, 2006 at 3:44 pm

“…the site says postings are only for scientists in the discipline, who must list their names and institutional email addresses.”

Would they allow a paper on a strange and radical theory submitted by an unknown patent agent?

Mr.Spywareup October 11, 2006 at 9:53 am

The Offical site of Spyware
spyware

Zmajrom June 2, 2007 at 12:11 pm
Nbkvqsa June 2, 2007 at 10:09 pm

ferte March 31, 2008 at 9:12 am

Comments on this entry are closed.

Previous post:

Next post: