The smell test for an academic paper

As recently as the 1990s, you could pick up an academic paper in economics and by examining the techniques, the citations, how clearly the model was explained, and so on, you could arrive pretty quickly at a decent sense of how good a paper it was.

Today there are still many evidently bad papers, but also many more papers where “the bodies” are buried much more deeply.  There are many more credible “contender papers” where the mistakes and limitations are far from transparent and yet the paper is totally wrong or misguided.  For instance, it is easier to “produce” a novel and striking result with falsified privately-built data than with publicly available macro data, which already have been studied to death and do not yield new secrets easily if at all.

One implied prediction is that a small number of absolute frauds will do quite well professionally.  Another prediction is that having close (and reputable) associates to vouch for you will go up in value.  (How reliable a method of certification is that in fact?)  It may be harder for some outsiders to rise to the top, given the greater difficulty of those outsiders in obtaining credible personal certification.  What would you think of a new paper from Belarus, or how about Changchun, which appeared to overturn all previous results?

What else can we expect?

I do not think we are ready for an academic world where our smell test does not work very well.