How to be a good referee

by on October 24, 2006 at 9:54 am in Education | Permalink

Here are my tips:

1. Assume that no referee reports are truly anonymous.  It is fine to be critical but always be polite.

2. Unless it is immediate junk, read the paper once and return to it a week later with deeper thoughts and a fresh mind.

3. Your report should not assume that the editor has a working knowledge of the paper in his mind.

4. Respond within a month.  First it is considerate to the author.  Second, the less "fresh" the task is, the more painful it will be.

5. A properly critical and useful "accept" report is harder to write.  Don’t look for excuses to quickly reject a potentially good paper.

6. The editor might have chosen you as referee for a reason.  You need not go along with the editor’s grand plan or desired outcome, but be aware it may exist.

7. Don’t check the references to see if you are cited.

Here is a short article full of good advice.  Here is the longer piece (which I have not read) on how to publish in top journals.  Thanks to Elaine Hawley for the pointer.

Barkley Rosser, a frequent MR commentator who has sent me JEBO papers to referee, may comment on this post but not on my adherence to these standards, especially #4.

1 Jeff Smith October 24, 2006 at 3:03 pm

I like 1-6 but disagree with 7. Part of the referee’s job is to make sure that the paper properly references the literature. Given that referees are not assigned at random, the relevant literature likely includes one or more papers by the referee. Doing this part of the job is especially important in literatures that span disciplines or where strong “schools” exist within economics.

I would also add that you should not include your recommendation in the report itself. It should go only in the cover letter to the editor. This gives the editor the most flexibility. Being an editor is largely, though not entirely public goods provision; as such, editors deserve some flexibility.

Jeff

2 BillWallace October 24, 2006 at 4:33 pm

8) Any time Dwyane Wade misses a shot from within 10 feet, call a foul.

3 jim October 24, 2006 at 9:18 pm

Barley wrote:

“I think it is a bad social norm that has evolved over time. Other
disciplines do not have such long periods [for reviews]. I have
both published and refereed in physics, math, comp sci,
and biology. In all of those the norms are much shorter,
like two weeks. It is a major embarrassment, and I
regularly get berated by people from some of those
disciplines over our scandalous practices in this regard.”

I agree, but I’d take it a step further and call it, for lack of a better term, BULL****, given that there are sometimes impacts upon tenure decisions and promotions.

4 Barkley Rosser October 25, 2006 at 3:08 pm

jim,

I will stick to calling it scandalous.

albatross,

Excellent point.

5 jim October 25, 2006 at 5:41 pm

Barkley,

I agree with you, it IS scandalous (and it is worse considering the impacts upon people up for tenure and promotion)

6 jim October 25, 2006 at 10:02 pm

Barkely,

You are correct that there is a pick-up in the publication of empirical work and simulation; still there are trends in the top journals toward the usage of complex mathematical techniques. The SEJ paper I cited above provides the evidence. Additionally it shows that papers in top journals that use the more complex mathematics are: 1) less likely to themselves contain empiricism; and 2) less likely to be cited in a 5 year window following publication by any paper that presents empirical analysis.

Gordon’s paper was in 1955.

7 jim October 26, 2006 at 10:13 am

Barkely,

Another thing occurs to me: Deirdre McCloskey has been trying for many years to persuade economics to give up their fascination with existence proofs in particular and what she terms the “mathematics of the math department” in general; instead, she has been trying to persuade economists to apply the “mathematics of the engineering or physics departments” (that is driven by the operational concerns for whatever the purposes are at hand). I find her argument persuasive.

You are correct that economists are using simulations with greater frequency. But the simulations that one often finds in economics are not, in McCloskey’s words, “calibrated” so as to have any relvance to anything that we observe in this world. Because such simulations are unlike those typical of the engineering and physics departments (again that are driven by practical concerns regarding an issue at hand), McCloskey views them as largely inferior “works”. Again, I find her argument persuasive.

8 jim October 26, 2006 at 6:17 pm

Barkely,

I have another paper in the works with Coelho that presents empirical evidence over a longer period than in our SEJ article on the use of complex mathematics. So I’m very curious to learn what measures were used in Weintraub’s book (thanks for alerting me to it).

I agree with the statement “there is no going back to
the days of the non-mathematical Old Institutionalists (or hermeneutic Austrians)”. Mathematics is a valuable tool for the development of operational propositions in economics. I am decidedly not against the usage of mathematics in economics.

What Gordon did NOT say was that complexity rules out operationalism; his hypothesis is that mathematical complexity in economic theory introduces a bias against the generation of operational propositions. The evidence that Coelho and I present in the SEJ does not persuade us that there is any reason to discard Gordon’s hypothesis.

9 jim October 30, 2006 at 4:10 pm

Barkley,

Really nice chatting with you. The library I have immediate access to here at BSU does not have the book you recommended that I read (“How Economics Became a Mathematical Science”). Nevertheless, I have sent for it via interlibrary loan though; thanks again for alerting me to it.

10 SGVFOA August 5, 2008 at 2:59 pm

If you are interested in becoming a football referee, please visit http://www.sgvfoa.com/be-a-ref.htm

11 Anonymous November 24, 2008 at 7:02 am

long

12 sohbet odalari July 31, 2009 at 3:09 pm
13 AkguN September 24, 2009 at 1:46 pm

thank you very much….

14 su deposu November 20, 2009 at 9:03 am

thank you admin…

15 mardinkusu January 26, 2010 at 5:20 pm
16 air jordan April 21, 2010 at 8:55 pm

Jordan has more than once said,the jordan shoes is part of my clothing and my personality。 We can useMichael Jordan; to address the Jordan, shoe or call him both sometimes, just one. Jordan and the special relationship, the air jordan shoes is formed, it is can be different styles of 23 pairs of shoes and collision of nerve cells in the us. Jordan with elegant and complicated movements and the sixth championship rings to deepen our repeated the nike air jordan of memory. Every year, jordans continuously changing, but all the air jordan to me. Jordan said.

17 chanelsunglasses May 15, 2010 at 9:12 pm

I read post often, even on this side,I think it is great!

18 wholesale stockings November 17, 2010 at 3:30 am

Your site is a refreshing change from the majority of sites I have visited. When I first started visiting web sites I was excited by the potential of the internet as a resource and was very disappointed initially. You have restored my enthusiasm and I thank you for your efforts to share your insights and help the world become a better place.

19 beetle turbo January 26, 2011 at 11:28 am

I don’t know what to say. This blog is fantastic. That’s not really a really huge statement, but it’s all I could come up with after reading this. You know so much about this subject. So much so that you made me want to learn more about it. Your blog is my stepping stone, my friend. Thanks for the heads up on this subject.

20 chat kanalı February 9, 2011 at 8:26 am

Thanks admin very nice.

21 omegle February 11, 2011 at 6:57 am

Thanks admin very nice.

22 buy stromectol February 18, 2011 at 2:26 am

I really loved reading your blog. It was very well authored and easy to understand. Unlike additional blogs I have read which are really not tht good. I also found your posts very interesting. In fact after reading, I had to go show it to my friend and he enjoyed it as well!

Comments on this entry are closed.

Previous post:

Next post: