Designing a statistics regime for selfish economists

As my colleague David Levy points out, the economics of economists is a much neglected topic.  But there is some action on the horizon:

The role that competition among scientists will have on researcher initiative bias was discussed by Tullock (1959) who argued that competition would counteract the version of publication bias that occurs when 20 researchers each use different data sets to run the 14 same experiment but when only the one significant result gets published.  Tullock argued that in this case the other 19 researchers would come forward and discuss their insignificant results.  The conclusion Tullock drew from this is that publication bias is more likely to occur in a situation where there is a single data and 20 possible explanatory variables.  In that case, there is no obvious refutation that could be published over the false positive.  The best that can be done is to publish articles emphasizing the number of potential explanatory variables in the data set (as in Sala-I-Martin, 1997) or the fragility of the results to alternative specifications (as in Levine and Renelt, 1992).

That is from a new paper by Ed Glaeser, highly recommended.  Hat tip to New Economist blog.

Comments

I'm not convinced that the best we can do is to publish the results of over a million false models
(a la Sala-i-Martin et al). Haven't read the paper yet, so may be taking things out of context, but
running millions of regressions on datasets riddled with data problems such as heteroskedasticity,
autocorrelation, multicollinearity and structural change means that many (if not all!) of the models
will give biased coefficients (omitted variable bias?). Rather, why not present a reasonably
simplified version of the 20 variable model, having ensured that both the original model and the final
model satisfy the assumptions that the model is based on, and that omitting any of the irrelevant variables
fails a t-test of significance by a comfortable margin?

"the economics of economists is a much neglected topic"

Are you serious!?!!?!!
Surely the most over-studied topic. There are at least three papers out there on collaboration networks, a ton on academic publishing (is it really first order, whether we read print or online), and even four or five different models of academic tenure.

Comments for this post are closed