Referees get worse as they age

They titled the piece "Older but not Wiser."  Here is the summary result:

Michael Callaham, editor-in-chief of the Annals of Emergency Medicine in San Francisco, California, analysed the scores that editors at the journal had given more than 1,400 reviewers between 1994 and 2008. The journal routinely has its editors rate reviews on a scale of one to five, with one being unsatisfactory and five being exceptional. Ratings are based on whether the review contains constructive, professional comments on study design, writing and interpretation of results, providing useful context for the editor in deciding whether to accept the paper.

The average score stayed at roughly 3.6 throughout the entire period. The most surprising result, however, was how individual reviewers' scores changed over time: 93% of them went down, which was balanced by fresh young reviewers coming on board and keeping the average score up. The average decline was 0.04 points per year.

"I was hoping some would get better, and I could home in on them. But there weren't enough to study," says Callaham. Less than 1% improved at any significant rate, and even then it would take 25 years for the improvement to become valuable to the journal, he says.

I thank Michelle Dawson for the pointer; I wonder when the editor who ran the study, Callaham, thinks he should resign.  He's totally gray.