The market for lemmas

by on August 22, 2007 at 11:24 am in Economics | Permalink

It’s not that big:

Of the sixty articles in AER, EJ, JPE, and QJE that had been cited more than 500 times, only one article contained an author written lemma.

Here is the paper, and thanks to Johan Almenberg for the pointer.  On p.6 you see a good graph, "Lemma frequency by decade."  Note also that a top paper in Econometrica is much more likely to have a lemma.

12 comments

Keith August 22, 2007 at 11:51 am

Lemmas are for lemmings.

Scipio August 22, 2007 at 1:08 pm

Well yeah, but according to your link there were only ca 17 journal articles per year with Lemmas 1970-1990, the period most of the cited papers are from (35 per year 1990-2000).

How many articles were published per year in those journals? 4-500? So you have 3-4% of articles with lemmas, and 1,7% of the top 60 citations? The under representation may be statistically significant, but not very impressive, especially in such a small sample. One more would have made Lemmas more likely to be cited†¦

Anyway I am not sure what the point is, except the obvious fact that there is a life cycle with important new theories. The first idea is more likely to be expressed verbally, subsequent authors write more rigorous versions.

Unless you are Stigler or Kahneman there is not much this tells you.

zbicyclist August 22, 2007 at 2:30 pm

Seems to be more proof that theoretical researchers don’t talk to academic empirical researchers (and do either dialogue much with practitioners?)

So, if a lemma never gets empirically tested, does it make a sound?

James August 23, 2007 at 4:12 am

Why in the world do people ever double space their papers? It always makes me think of freshman term papers. How painful.

AZ August 23, 2007 at 10:21 am

How about this: Papers without lemmas are accessible by many, many people, and papers with lemmas are accessible by a much narrower set of people. Akerlof’s 1970 QJE on lemons markets is much more accessible to the layscholar than Charles Wilson’s 1980 Bell Journal on equilibrium in markets with adverse selection. So the layscholars skip Wilson’s piece because they can’t/don’t want to slog through it, but cite Akerlof’s piece because it’s an easier read.

In the econometrics literature, I wonder how many people have actually READ Hausman’s 1978 Econometrica on specification tests, as opposed to simply citing it because “it’s that specification test paper”.

conchis August 23, 2007 at 3:16 pm

jim,

Thanks for the reference. As, I said I’m inclined to believe the result. But I’ll stand by my claim that the “lemmas” paper shouldn’t shift one’s priors much, and to be honest, while the analysis in the SEJ paper is slightly better, it isn’t exactly great either.

The SEJ paper claims to find two things:

(a) Papers with lemmas (PWLs) are themselves less likely to have “empirical content” than papers without lemmas. This seems entirely unsurprising. It’s consistent with the Gordon hypothesis, but only in a way that makes it almost trivially true: complexity crowds out empirics because, by definition, you’ve spent your time generating lemmas rather than analysing data.

(b) PWLs have a lower proportion of citing articles with empirical content. This seems closer to what we want, but the paper unfortunately doesn’t define it’s comparison group adequately. PWLs could generate a smaller proportion of empirical papers either: (i) because the Gordon hypothesis is true and more complex theory papers generate fewer empirical citations than simpler theory papers; or (ii) simply because (per (a)) PWLs are more likely to be be theory papers rather than purely empirical papers, and theory papers tend to generate a higher proportion of theory papers than do purely empirical papers. It should have been pretty easy to rule out the second explanation, but they didn’t try to.

(As an aside, I also suspect that the authors increase the significance of their results by artificially inflating the sample size (they count citing articles rather than cited articles) but I’m not sure that would affect the result much.)

conchis August 23, 2007 at 7:12 pm

jim, i don’t think i’m quibbling with nuances. i’m saying that the studies don’t adequately test the hypotheses they say they do. consequently, they fail to refute the null that the Gordon hypothesis is false. by your reasoning, this would seem to imply that “as the anti-Gordon hypothesis fails to be refuted by test after alternative test its merits are becoming more and more clear.”

to put it more bluntly: a bunch of crap studies don’t necessarily add up to convincing evidence. they may just add up to a bunch of crap.

i find this particularly annoying because, as i said, i’m a priori inclined to believe the hypothesis, and some really simple checks in both papers should have fixed the problems i’ve pointed out. when people don’t bother to report really obvious checks like this, there are two potential reasons: (i) they didn’t think to run some really obvious tests; or (ii) they actually ran the checks, but didn’t the results they wanted. neither of these possibilities should exactly incline one to revise one’s priors upwards.

if you want to argue that the Gordon hypothesis is probably true, i’ll concur. if you want to argue that these studies contribute anything substantive to proving that, you’ll find serious disagreement.

conchis August 23, 2007 at 9:35 pm

Jim, Explanations in science win out because they do a better job of predicting the evidence than alternative explanations. The problem with these papers is that they present evidence that one would place a high prior probability on observing even if Gordon’s hypothesis is false. They consequently provide little reason for updating our beliefs about the hypothesis.

Feel free to keep handwaving about how this is a mere quibble, peer review is infallible, and that the fact that Gordon’s hypothesis hasn’t been contradicted means that it must be true. Repeating it won’t make it any more correct. I think I’ve wasted more than enough time on this already, particularly given that you seem unwilling to engage with substantive argument.

MP August 24, 2007 at 5:05 pm

indiana jim,

Why is it important that economic theories must lead to “operational statements” that can be tested? Haven’t you read Rubinstein’s 2006 Econometrica paper (http://arielrubinstein.tau.ac.il/papers/74.pdf)?

无缝管 November 13, 2007 at 8:05 pm

Hi Best wishes。Allow me to offer my heartiest wishes.xicao loves-流水线娱乐博客常年提供高、中、低压锅炉钢管、流体钢管、结构钢管、化肥专用钢管、石油裂化钢管、地质钢管、液压支柱钢管及合金钢管-无缝管-无缝钢管等论文发表资讯/刊物信息，协助客户制定论文发表方案

翻译公司 February 13, 2008 at 7:16 am

翻译公司|深圳翻译good深圳翻译公司|同声传译

花蓮旅遊 June 4, 2009 at 9:51 am

花蓮,旅遊租車,花蓮旅遊,花蓮租車,花東旅遊,花蓮租車,花蓮租車,花蓮旅遊,租車,花蓮旅行社,花蓮旅遊景點,花蓮旅遊行程,花蓮旅遊地圖,花蓮一日遊,花蓮租車,花蓮租車旅遊網,花蓮租車,花蓮租車,花蓮租車,花東旅遊景點,租車,花蓮旅遊,花東旅遊行程,花東旅遊地圖,花蓮租車公司,花蓮租車,花蓮旅遊租車,花蓮租車,花蓮旅遊,花蓮賞鯨,花蓮旅遊,花蓮旅遊,租車,花蓮租車,花蓮租車 ,花蓮 租車,花蓮,花蓮旅遊網,花蓮租車網,花蓮,租車,花東 旅遊,花蓮 租車,花蓮,旅遊,租車公司,花蓮,花蓮旅遊,花東旅遊,花蓮地圖,包車,花蓮,旅遊租車,花蓮 租車,租車,花蓮租車資訊網,花蓮旅遊,租車,花東,花東地圖,租車公司,租車網,花蓮租車旅遊,租車,花蓮,賞鯨,花蓮旅遊租車,花東旅遊,租車網,花蓮海洋公園,租車 ,花蓮 租車,花蓮,花蓮旅遊,花蓮租車公司,租車花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,花蓮旅遊,花蓮租車,花蓮旅行社,花東旅遊,花蓮包車,租車,花蓮旅遊,花蓮租車,花蓮一日遊,租車服務,花蓮租車公司,花蓮包車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮租車,租車網,花蓮租車公司,花蓮旅遊,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮租車公司,花蓮一日遊,租車,租車服務,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮旅遊,花蓮賞鯨,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,租車花蓮,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,租車花蓮,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,租車公司,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮旅遊,花蓮旅遊租車,花蓮租車網,花蓮租車,花蓮一日遊,租車花蓮,花蓮租車,花蓮旅遊租車,花蓮租車,花蓮租車旅遊,花蓮租車,花蓮旅遊,花蓮旅遊,花蓮包車,花蓮溯溪,花蓮泛舟,花蓮溯溪,花蓮旅遊,花蓮旅遊,花蓮租車,租車公司,花蓮旅遊租車,花蓮租車,租車,花蓮旅遊,花蓮租車,花東旅遊,花蓮賞鯨,花蓮旅遊,花蓮泛舟,花蓮賞鯨,花蓮溯溪,花蓮泛舟,花蓮泛舟,花蓮溯溪,花蓮旅遊,花蓮旅遊,花蓮租車,花東旅遊,花蓮,花東,花蓮旅遊,花東旅遊,花蓮租車,花蓮,花東,花蓮旅遊,花蓮租車,花東旅遊,花蓮旅遊,花蓮租車,租車,花蓮旅遊,花蓮租車,花蓮旅遊租車,花蓮旅遊,花蓮租車,花蓮,花東旅遊萬事通,花蓮旅遊,租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車,花蓮租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車,花蓮包車,花蓮旅遊,花蓮租車,花蓮太魯閣,花蓮包車,花東旅遊,花蓮旅遊行程,花蓮旅遊,花蓮 租車,花蓮租車,花蓮租車旅遊,花蓮旅遊租車,租車,花蓮旅遊推薦,花蓮旅遊包車,花蓮租車,花蓮,花蓮租車,花蓮地圖,花蓮旅遊,花蓮旅遊資訊網,花蓮旅遊景點,賞鯨,花蓮旅遊行程,花蓮旅遊,花蓮旅遊租車,花東旅遊景點,花東旅遊行程,花蓮旅遊,花蓮租車,租車,花東旅遊,花蓮旅遊,花蓮租車,花蓮,旅遊達人,旅遊達人blog,花蓮租車旅遊資訊網,花蓮,租車,花蓮,花東旅遊,地圖,租車,賞鯨泛舟溯溪,租車,[ 芝麻店家 ] 花蓮租車旅遊資訊網,花蓮租車-行易花蓮旅遊租車資訊網 ,旅遊網,旅遊景點,花蓮行程,花蓮,花東,旅遊租車,旅遊,花蓮,租車,花東旅遊,花蓮租車旅遊,行易旅遊民宿資訊網,花蓮,旅遊,花蓮,一日遊,花蓮好玩的地方,花蓮,,一日遊,花東,租車,旅遊,花蓮旅遊,花東旅遊,花蓮租車,花蓮租車,花蓮旅遊-花東旅遊萬事通

Comments on this entry are closed.

Previous post:

Next post: