“If only there were a vast empirical literature…”

by on May 19, 2017 at 12:37 am in Economics, Education, Science, Uncategorized | Permalink

Paul Krugman blogged on that, with initial impetus from Noah Smith.  Here is Noah:

If you and your buddies have a political argument, a vast literature can help you defend your argument even if it’s filled with vague theory, sloppy bad empirics, arguments from authority, and other crap. If someone smart comes along and tries to tell you you’re wrong about something, just demand huffily that she go read the vast literature before she presumes to get involved in the debate. Chances are she’ll just quit the argument and go home, unwilling to pay the effort cost of wading through dozens of crappy papers. And if she persists in the argument without reading the vast literature, you can just denounce her as uninformed and willfully ignorant. Even if she does decide to pay the cost and read the crappy vast literature, you have extra time to make your arguments while she’s so occupied. And you can also bog her down in arguments over the minute details of this or that crappy paper while you continue to advance your overall thesis to the masses.

…My solution to this problem is what I call the Two Paper Rule. If you want me to read the vast literature, cite me two papers that are exemplars and paragons of that literature. Foundational papers, key recent innovations – whatever you like (but no review papers or summaries). Just two. I will read them.

If these two papers are full of mistakes and bad reasoning, I will feel free to skip the rest of the vast literature. Because if that’s the best you can do, I’ve seen enough.

Those are both interesting posts, but my perspective is different, probably more as a matter of temperament than thinking they are objectively wrong.  Here are a few comments:

1. The best two papers on ethics are not very convincing.  Nonetheless people who have worked their way through a good amount of that literature are much better at ethical reasoning than those who have not.

2. The best two papers on global warming are not very convincing.  What is convincing is how many different perspectives and how many different branches of science point toward broadly similar conclusions.  In fact the aggregate effect here is quite overwhelming (don’t debate gw in the comments, not today; I’ll delete).  It is a question of many moats, not all of them being entirely muddy.

3. I see the Smith-Krugman standard as fairly economistic, and fairly MIT-late 20th century at that.  It is one vision of what a good literature looks like, and a fairly narrow one.  It will elevate simple answers in status, whether or not that is deserved.  It discriminates against dialogic knowledge, book-based knowledge, historical knowledge, and knowledge when the answers and methods are not very exact.  There is the risk of ending up too certain about one’s knowledge.

That all said, I do understand that specialized top researchers, including Nobel Laureates, often may do better holding relatively narrow methodological visions.  Look at all the Nobel Prizes that have been awarded to Chicago.  It might be entirely correct to insist that Becker’s treatise on the family pay more attention to anthropology, but that doesn’t mean he should have followed that advicee.

4. The standard seems to discourage reading, and I would not want to teach it to my students.  I teach something more like “always read more, unless you are writing or doing relevant quantitative work.  And one reason you write is to improve the quality of your reading.  Read more and write more, all the time.”  I still think that is better advice for most (not all) people.

5. Isn’t there a lot to be said for deferring to the opinions of those who have read through the “muddy moat”?  By no means are they all partisans, and the non-partisan ones care most of all about the truth.  After all, they did all that reading!  Defer, rather than trust so much in your ability to pick you the right two papers, or have someone pick them out for you.  I have a much more positive view of survey articles than does Noah, while understanding they do often leave you fairly agnostic on major issues.

6. If the truth of the matter is in fact muddy, you may need to dip into the muddy moat to learn that.

7. The difference between total value and marginal value may be relevant.  You might conclude a field literature has low total value, but the marginal value of learning more about that area still could be quite high.  That is in part because muddy fields and results don’t spread so readily, and so dipping into the muck can yield some revelations.  That is another reason why I would not offer the “two paper standard” as practical advice.

8. If anything, I would put the reading pressure on the other side, namely more rather than less.  Rather than encouraging readers to dismiss or downgrade fields, I would urge them to consult different disciplines altogether, including political science, sociology, and anthropology, others too.  This is much easier to do if you take a more positive attitude toward survey articles.

9. This is quite a subjective impression, but I worry that the dogmatic will use the two paper standard to dismiss or downgrade particular lines of investigation.

10. I don’t know if Noah and Paul were referring to my colleague Garett Jones, who frequently tweets “…if only there were a vast empirical literature” when he sees claims that he regards as empirically false.  Now, I am not the Garett Jones oracle, but I always took his use of the word “vast” to be slightly sarcastic.  Usually these are cases where even a fairly cursory knowledge of the literature in question would indicate something is wrong with the claim at hand.  In my view, Garett is not demanding “vastness” of effort, rather he is criticizing those who don’t grasp what the effort space looks like in the first place.

1 derek May 19, 2017 at 12:54 am

It depends on whether you want to win arguments or actually figure out what is going on.


2 GoneWithTheWind May 19, 2017 at 9:49 am

Exactly. As for the statement to not debate GW in the comments; think about that. Why? Because there is in fact no consensus on GW contrary to the claim. Not all of the scientists, not all of the professors and most certainly not all of the citizens agree on the AGW theory. Therefore it must be censored. And THAT folks is the “ethical” standard for debate today. Shut the other side down.


3 The Other Jim May 19, 2017 at 11:50 am

And with Krugman, it is always the former.

[Did he actually denounce “appeals to authority”? Hilarious!]


4 Troll Me May 20, 2017 at 11:21 am

99 is not 0.

Also, the range inclusive from 95 to 100 is also not 0.

Yes, there are different opinions in the field.


5 Troll Me May 20, 2017 at 11:20 am

People who aim to win arguments more so than learn are mainly experts in self delusion (or in following influences which direct them to similarly deluded perspectives of “winning”).


6 first impression May 19, 2017 at 12:55 am

1. wrong. 2. wrong. 3. right. 4. right 5. MPAI. 6. Right. 7. Very right. 8. Naive, very naive – you could not have meant to say what you seem to say. I must have misread. 9. Nobody cares about the dogmatic. 10. sarcasm is difficult.


7 Earnest Hemingway May 19, 2017 at 7:09 am
8 first impression May 22, 2017 at 11:55 pm

Thanks, E.H. Maybe, like you, I post comments here – at a site run by an extremely knowledgeable guy – more for the amusement of 2 or 3 specific people, and a bunch more reading this stuff 20 years from now with AI help, than out of any aspiration that anyone outside that group cares what I say. For the record, though – my confidence levels on my takes on 1, 5, 6, and 7 are very high, and thanks for the research on 1. T.C. wrote 2 very badly – I wish he would buy a leading volcano textbook (generally they retail for around 60 bucks) – understanding reality is hard but understanding our limits of understanding of reality is not all that hard – harder, for sure, than a summer at computer camp in 1993, but not as hard as 6 months as an infantry officer in and around the rice paddies of Nam in 1968. Let’s not set our standards too low.


9 Thanatos Savehn May 19, 2017 at 12:56 am

There is but, per P E Meehl, it was long ago rendered “well nigh uninterpretable” thanks to academics engaged in the null hypothesis racket.

P.S. Krugman used the Econ lit pond for years as his personal septic lagoon. Yes, he was an early exploiter of faulty statistical tools and used them to fap his personal biases until they were seemingly confirmed. But he got rich doing it so that makes it ok to the progressive Left.

What a world.


10 Saint-Frusquin May 19, 2017 at 12:57 am

” What is convincing is how many different perspectives and how many different branches of science point toward broadly similar conclusions. ”

English is a foreign language for me but this is what I call complacency.


11 prior_test2 May 19, 2017 at 4:24 am

Well, Prof. Cowen has apparently zero interest in anything that might seriously perturb any of the current underpinnings of the apparently natural order of the rich getting richer.

Such as any even seeming awareness of the GPL, and the role it has played in such trivial things as the Internet or smart phones.


12 Troll Me May 20, 2017 at 11:23 am

When numerous things point in a similar direction, the lack of being identical is equivalent to them all pointing in the opposite direction that they in fact point.

Or … what else am I supposed to take from what you just said?


13 A.G.McDowell May 19, 2017 at 1:10 am

Among laymen – this is why we have a representative democracy, not a series of referendums.

Among supposed experts – agree on the design of an experiment to resolve the question at issue and then either execute it, or produce a paper from the literature from somebody who has already done it.

Amongst those developing computer systems, there has been a popular movement in the past decade or more away from producing large amounts of detailed documentation to producing working code early that the customer can test, precisely because large amounts of detailed documentation are rarely helpful. Perhaps instead of constructing large repositories of literature, other subjects should start investing more time in producing agreed and tested computer models. Then instead of a literature search you run an experiment on the model. The climate change argument certainly seems to centre around the credibility of climate change models and perhaps of associated economic models.


14 Bob May 19, 2017 at 1:26 am

It’s not just GW: A fun equivalent that today is less controversial is the theory of evolution. The different ways that scientist can use to reach the very same set of nested hierarchies is outstanding: From the original anatomic comparisons to da variety genetic markers.

There’s a lot anyone, will find valuable by looking at our research inside of a cell. For example, I’ve seen biologists getting interested in economics by seeing parallels between how some regulating mechanisms inside cells work and markets!

Almost any field will have a lot more than two papers that are worthwhile for those interested in the topic.

OK, maybe not cryptocurrencies.


15 Noah Smith May 19, 2017 at 2:06 am

1. Hmm, ethics is an interesting example. I’ll have to think about that one.

2. C’mon, I bet there are far more than two methodologically great papers on climate change, even if they don’t come to firm conclusions on their own. 🙂

3. I accept the compliment. 😉

4. That’s a good point. I guess I kind of assume that everyone else, like me, is constantly reading *something*…

5. The Two Paper Rule is supposed to tell me whether or not I should defer to the people who’ve read far more than two papers. If I read two papers and they’re obviously crap, I’m less inclined to defer to those who have read 200…

6. No doubt. But some moats are far more worth dipping into than others…

8. Yeah, that’s a good point.

10. I believe my post was inspired by someone telling me to go read the vast literature on cultural appropriation and social power dynamics…


16 The Cuckmeister-General May 19, 2017 at 2:41 am

Climat Change is a hoax you cuck


17 Axa May 19, 2017 at 5:05 am

Mr. Smith, I’m afraid you’re disguising your lack of interest in certain topics as the “effort cost of wading through dozens of crappy papers”.

Any individual working on research has to read crappy articles. It happens, either by being fooled by a misguiding abstract or because the reasoning behind the article is faulty.

The Two Paper rule is an idealization of science. How great would be if reading two articles gave a general understanding of a topic. But could you give an example of that rule? 1 topic, 2 articles.


18 Ricardo May 19, 2017 at 6:35 am

But he didn’t say that reading two papers would give one a general understanding of a topic. Think of it as like a Turing test. If someone who is considered an expert in a topic cites two papers he or she regards as high-quality but that are clearly full of errors and ignorance of relevant facts, then it is not clear why that area should be taken seriously by outsiders. If the papers do pass the test, then may there is something there. A fair-minded and curious person would be obligated to investigate further and, yes, possibly run into crappy papers over the course of the investigation.

More generally, a rule like this is necessary in order to make any intellectual progress at all. Otherwise, we are stuck in a permanent debate over whether the moon landing was faked, whether gravity is socially constructed or whatever. The nicest possible response to people who make claims like this is to ask for their best work, perhaps give that best work a genuine couple of hours of your time and then move on. Everyone has a BS filter whether they acknowledge it or not. An explicit filter that one has thought through provides the best chance of being open to new ideas but without wasting time on garbage and pseudo-science.


19 Noah Smith May 19, 2017 at 12:18 pm

“Mr. Smith, I’m afraid you’re disguising your lack of interest in certain topics as the “effort cost of wading through dozens of crappy papers”.

Any individual working on research has to read crappy articles. It happens, either by being fooled by a misguiding abstract or because the reasoning behind the article is faulty.”

Oh sure, but I’m not talking about research. Suppose I want to talk about cultural appropriation. But then someone comes and tells me that before I give my thoughts, I ought to read the vast literature on cultural appropriation that has been written by critical theorists and quasi-sociologists. Do I oblige them? I don’t want to be a researcher in that field. But I also doubt that the researchers in that field have expertise that I should defer to. What do I do? Just shut up and let people who have spent hundreds of hours immersed in that stuff have the floor to themselves??


20 RafaelR May 19, 2017 at 9:54 pm

You should always respect the knowledge of people who specialize in a field that you do not. Thinking that the field is garbage from the impression you might get from two papers is due to the lack of training in the field and not because the papers are indeed garbage.

People shouldn’t have strong opinions on 99.9% of the possible topics because outside their field of specialization they are indeed idiots. This applies to anyone: Albert Einstein for instance was defending socialism back in the 1940s. Ignorance is the natural state of the human mind, one shouldn’t have opinions on anything outside their field of expertise: judging fields one doesn’t know is just evidence of being an idiot.

21 Troll Me May 20, 2017 at 11:34 am

It sounds like you’re more concerned with the valuation or devaluation of opinions you pre-disagree with.

You pre-state that you a) don’t intend to read them and thus b) do not know what they might have said. But regardless of not having read them and not intending to read them, you claim to know that they are not worth paying attention to.

On the specific subject, I think I’d agree. If someone wants to use Viking helmets for artistic works in Africa or Nigerian forms for artistic works in Europe, I don’t see why that should be a big deal. You’d think that people would be honoured to have their culture copied or reinterpreted. And, while I’d be very interested to hear what the fruit seller or farmer of the relevant regions might have to say on the matter, I really don’t care what academic works on the subject have to say.

So it seems we agree. But we don’t. Because I’ve extended the situation a little to explain why I think this can be reasonable for the subject of cultural appropriation. Whereas it sounds like you’re just refusing to read stuff that would be painful for you to read because you will disagree with them a lot. I really just don’t care what they have to say on the specific subject, and have no qualms about proclaiming academic ignorance on a subject for which I think the perspectives of fruit sellers and farmers is more relevant.

22 Dan May 19, 2017 at 2:30 am

I think you’re misunderstanding Noah’s advice. The two papers don’t have to be “convincing” of the field’s bottom-line conclusion, they just have to be well-done analyses of some of the pieces that point towards that conclusion. Noah’s standard is that they should not be “full of mistakes and bad reasoning”. That lets you see that there is some good work being done in the field, which makes it plausible that the vast literature might combine to provide strong evidence for a particular conclusion.

For example, maybe one climate paper is about the feedback effect of clouds and the other is about the ocean as a heat sink. No matter how well-argued they are, those two papers in combination are clearly insufficient for establishing a bottom-line conclusion about likely future temperatures. But they can provide evidence that (at least some) climate scientists are doing good research, on the sorts of subtopics that are relevant for predicting changes to the Earth’s temperature. That would imply that the climate literature is something to be engaged with if you want to understand global warming, not something to be dismissed out of hand.


23 FusterCluckold May 19, 2017 at 3:00 am

“1. The best two papers on ethics are not very convincing.”

Just because they prove contradictory theories does not mean they aren’t convincing. The best ethics paper will prove deontological (Kantian) ethics, and the next best paper will prove Rule-Utilitarian (Mill) ethics. Together they disprove each other (they are both inadequate as complete theories), but you could look at it the other way–that separately they are very convincing of their own theories as part of a complete theory.

It’s like telling me that reading the ninth and tenth amendment together just proves that individually they are both incomplete parts of a grand unified political system. But you could look at it the other way, that reading them together they are more than the Summum Bonum of their parts.


24 S-J-double-U May 19, 2017 at 3:16 am

“And if she persists in the argument without reading the vast literature, you can just denounce her as uninformed and willfully ignorant.”

And she will say that you’re man-splaining and a minister of the patriarchy, and will file a Title IX complaint against you.


25 Jacob Aaron Geller May 19, 2017 at 4:17 am

Why not pick a review/survey article? That is exactly the kind of paper I would reach for if someone (including myself) were interested in the quality of a vast empirical literature…

I might also reach for textbook chapters.

I don’t know what the two best papers on the benefits of free trade are, but they’re probably too methodologically narrow or interested only in a narrow (set of) question(s) to be very indicative of the broader literature. A survey/review article would be better (broader, more indicative of the quality of the literature, etc.), but still maybe a bit dense for anyone unaware of the literature. But even a half-decent textbook chapter on international trade will sketch a good basic outline of the literature, plus some stuff from the frontier, and I would imagine that the *best* textbook chapter on international trade will do better than the Best Two Papers, inclusive of survey/review articles or not.

I’d be interested to hear from Noah what he thinks is most convincing and/or informative to the lay person interested in the vast empirical literature on international trade: 1) The Top Two Papers survey-inclusive, 2) The Top Two Papers sans-surveys, or 3) Noah’s favorite textbook chapter on international trade. (And I’d like to know what papers/chapters those are!) 🙂


26 Jacob Aaron Geller May 19, 2017 at 4:20 am

PS — If Noah could ask his interlocutor the same question about cultural appropriation and social power dynamics (what top 2 papers? top 2 surveys? top 1 chapter?) I’d love to know their answer to that question as well.


27 Troll Me May 20, 2017 at 11:38 am

Probably Ricardo from 200 years ago, and then maybe something empirically demonstrating some instances where ISI was sensible?

Key being to avoid dogmatism while internalizing important lessons.


28 StateoftheObvious May 19, 2017 at 4:50 am

Perhaps what is being said is, if you believe there is vast literature of primary source material defending your position, then have a few names of authors and books to give people who are interested in delving deeper into that (that will allow them to see that your ideas are not just propped up by a house of cards) and don’t just tell them that if they read the wikipedia article on the topic they would see that you are obviously correct in all ways and forms. Of course, if the primary source material is mostly made up of higher-level math, then a “popularizer” source for laypeople will have to do.

For instance, if someone wants to know why you accept classical liberalism, don’t just tell them to re-read the definition in their political science textbook, tell them that you were inspired by Mill or Hayek or whomever.


29 prior_test2 May 19, 2017 at 5:05 am

Well, here is some literature, but facts rarely have any place when it comes to talking about the ‘best’ – http://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(17)30818-8.pdf


30 TMC May 19, 2017 at 11:30 am

You talk about fact, then link to an article from the Lancet. Best satire on the web.


31 Troll Me May 20, 2017 at 11:40 am

Can you cite a more highly reputed journal in medicine than the Lancet? You might argue there are one or two. Or none.


32 Todd K May 20, 2017 at 7:54 pm

The New England Jnl of Medicine, the British Medical Jnl and JAMA.


33 Daniel Silveyra May 19, 2017 at 5:31 am

Tyler, I’m surprised you (nor the other commenters) addressed the time-saving benefit of the two-paper rule, which seems to be the main reason to favour it. It is better to read more of the literature on any given topic than less, no one would dispute that; Noah appears to read much more deeply that two papers into the literature of subjects he is interested in.

Another benefit of the approach is that it narrows the field of discussion to a concrete set of claims (vs. the entire universe of unstated potential claims) and thus enables some progress.

I’d be interested in views on this: what is a better approach (vs. the two-paper one) to evaluate the state of knowledge on a particular question and inform truth-seeking discussion*?

*Yeah, I know most discussion isn’t truth seeking but there’s a already vast literature on that


34 rayward May 19, 2017 at 6:19 am

“4. The standard seems to discourage reading, and I would not want to teach it to my students. I teach something more like “always read more, unless you are writing or doing relevant quantitative work. And one reason you write is to improve the quality of your reading. Read more and write more, all the time.” I still think that is better advice for most (not all) people.” Just when I have doubts, Cowen writes something like this, and restores my faith. Narrowly, nothing focuses the mind quite like writing a good sentence. And nothing prepares one for writing a good sentence more than reading. An athlete doesn’t prepare for a marathon by watching the Boston Marathon. A scholar doesn’t find knowledge by following the Two Paper Rule. Insight requires a broad base, a foundation – the broader the base, the stronger the foundation, the greater the insight. It’s the foundation from which knowledge and progress flows.


35 Evans_KY May 19, 2017 at 6:24 am

Science is facing is similar issues. We have polished and educated charlatans who co-opt our methods to argue in the negative without understanding simple statistics. We have publications failing to properly peer review submissions.

I am constantly amazed at articles that cite graphs, methods, and confuse general terms such as consensus. If 9 of 10 doctors say you have cancer and it was caused by smoking, do you believe the doctor who disagrees?

National Review has a fine example between John Cook and Oren Cass. I didn’t say it!

Slate published an interesting take this week that I had not considered. https://redux.slate.com/cover-stories/2017/05/daryl-bem-proved-esp-is-real-showed-science-is-broken.html

No wonder most Americans are confused about what is true and what is fake. Trust but verify is more difficult when TV can so effectively numb the senses.


36 TMC May 19, 2017 at 11:35 am

I am constantly amazed at articles that cite graphs, methods, and confuse general terms such as consensus. If 9 of 10 doctors say you have cancer and it was caused by smoking, do you believe the doctor who disagrees?

If this is your understanding of the subject, or science in general, you should definitely read more as Tyler suggests.


37 Evans_KY May 19, 2017 at 1:27 pm

Ooohh. Aggression is a perfect way to dismiss someone.


38 dearieme May 19, 2017 at 6:38 am

“The best two papers on ethics are not very convincing.” How do you know: do you read Ancient Greek?

“The best two papers on global warming are not very convincing.” I am surprised to discover that you consider yourself equipped to judge which are the two best papers on this topic. You must know far more about several different sciences than I would have guessed. How did you come by your expertise on the mathematical modelling of physicochemical phenomena, of fluid mechanics, of heat transfer, temperature measurement, thermodynamics, and so forth?

You are so little a scholar that you refer to the Swedish Central Banks Prize as a Nobel Prize, yet you ask us to accept your expertise on science and ethics. Rum do.


39 prior_test2 May 19, 2017 at 6:48 am

It is unfair to mention that ethics is not based on ‘papers’ and pointing out that Greek philosophers were discussing this subject 2500 years ago. Somewhat like saying that the top two papers on tragedies would only give a muddled perspective on someone like Sophocles or Shakespeare.


40 Edward Measure May 19, 2017 at 6:42 am

As usual, Cowen avoids or misses the point. The standard is not “the two best papers,” it is two ‘papers’ that are exemplars. For global warming, try Arrhenius (1896) and Keeling’s work in the 1960’s. For ethics, try the Sermon on the Mount and Aristotle – or any of hundreds of others.


41 Thiago Ribeiro May 19, 2017 at 6:43 am

So it has come to it.


42 prior_test2 May 19, 2017 at 6:50 am

‘don’t debate gw in the comments, not today; I’ll delete

From occasional reader to vigilant monitor of proper commenting decorum. One wonders what admission about how this place works will come next.


43 Todd K May 19, 2017 at 7:19 am

My guess: AlexT has been on a studio set in Hollywood the past few months faking the mission to India. It is amazing how real those photos look.


44 Scott May 19, 2017 at 7:22 am

How do we measure some people being much better at ethical reasoning than others? Two papers please.


45 john May 19, 2017 at 8:12 am

Re 1 — if the state of the literature is such that even the two best papers are not that convencing then perhaps the best paper to suggest reading by an opponent (if the tactic is “go get familiar” before saying anything) would be the one or two best literature survey peices. If that doesn’t exist then some grad student probably just found a $50 bill on the ground.

“And you can also bog her down in arguments over the minute details of this or that crappy paper while you continue to advance your overall thesis to the masses.”, well depending on just who “the masses” are that probably doesn’t really work. The masses tend to free-ride (or think they are) on the experts. Anyone who can then start poking holes in another persons argument (which may not mean having read everything but simply keeps demaning the proof be provided not asserted) may well start winning the debate in the masses eyes. Trump anyone?


46 Todd K May 19, 2017 at 8:39 am

2. “The best two papers on global warming are not very convincing. What is convincing is how many different perspectives and how many different branches of science point toward broadly similar conclusions. In fact the aggregate effect here is quite overwhelming…”

Notice that Cowen doesn’t say what the broadly similar conclusions actually are.

Here is an example of what I’m getting at. Last fall, Scott Sumner wrote a post on climate change. The opening of his first point was: “1. Global warming is a crisis for our planet,…”

In comments he also wrote: “Everyone, I’ve read many scientific papers on global warming, and view the science as being pretty solid.” Sumner does not have a science background so how would he know if these technical papers were solid? They could not have pointed to his statement that global warming is a crisis.


47 Troll Me May 20, 2017 at 11:45 am

“what the broadly similar conclusions actually are” to the effect of the “A” belonging in “AGW”, and the “W” in “AGW” being an accurate reflection of reality.


48 Todd K May 20, 2017 at 7:41 pm

You can’t seem to state what the “broadly similar conclusions” are either. I see one conclusion: Most of the warming of the past few decades, when not in a fairly long pause, has been due to an increase in industrialization that has increased the CO2 in the atmosphere.

That’s it – the one consensus.


49 Bill May 19, 2017 at 9:32 am

You can get both sides very easily

On Fox News

Because it is Fair and Balanced.


50 Barkley Rosser May 19, 2017 at 9:57 am

I’ll make a point here that I made to Noah on his blog. One should supplement looking at the supposedly top two papers (how that is determined has remained unclear) with that one should also look at a good review paper of the lit. I think that would catch the issue on climate change.

Noah put up a response as an addendum to the post arguing against looking at review papers because he says that he is interested in the methodology of the papers and the literature. But certain literatures in fact have competing methodologies, sometimes more than just two, and sometimes review essays actually do pick up on this.


51 Troll Me May 20, 2017 at 11:47 am

If a review paper doesn’t compare methods, then it wouldn’t be more than just a list of compared/contrasted opinions.


52 Troll Me May 20, 2017 at 11:48 am

At least, in more scientific stuff.

Such thinking could be irrationally logical in less empirical work. I.e., maybe not in ethics, which is where he started.


53 nigel May 19, 2017 at 10:49 am

Tyler is optimistic. I seriously doubt most people who claim to be familiar with the vast literature have actually read most of the papers, or even the abstracts. Tyler is a good reader and an honest scholar but I suspect many don’t measure up to the standards he charitably assumes everyone follows. That’s the real question. The muddier the moat, the less believable it is that people claiming the vast literature mantle aren’t just bluffing. Close reading and original thought are quite time consuming.


54 Jonathan May 19, 2017 at 10:52 am

My actual method of wading into any literature is to try and find the two or three most contemporary papers on the topic and wade back via things that strike me as interesting/relevant/important in the references. This method has usually worked so long as at least one of the articles is well written and well referenced. Where it works least well is when the subject itself is clearly more ideological than scholarly while written is a scholarly rubric, and I betray perhaps more than a little condescension to say that a literature on “cultural appropriation” would probably fall in that bucket. Taking Noah at face value, that may be what he meant as well.


55 TMC May 19, 2017 at 11:42 am

Noah’s suggestion may not be the best way to learn more about, but it will do a good job evaluating the person he’s talking to making the claims, which seems to be his objective.


56 Harald Korneliussen May 19, 2017 at 11:43 am

“Nonetheless people who have worked their way through a good amount of that literature are much better at ethical reasoning than those who have not.”

Are they?

I have a name for people who are good at robust ethical reasoning. I call them saints. Saints are in short supply. I don’t know of any ethics professors I would classify as one.


57 Borjigid May 19, 2017 at 2:17 pm

Surely there is some space between ‘average person’ and ‘saint’ which (some) well-read people might occupy?


58 Troll Me May 20, 2017 at 11:51 am

Do you know any ethics professors?

I don’t know any Martian experts in ethics. Having never met a Martian, of course.


59 Rayward the teacher May 19, 2017 at 1:57 pm


The rate of life expentancy increase from 1850-1950=1950-pressent day, roughly, accounting for Global War.

In 1920, the rate of infectious disease dropped precipitously. Cardio disease dropped but not by nearly, nearly the same rate.

Cardio disease death rates dropped since the 60’s but can we attribute this to good doctoring? Or reaping the gains from the 19th century. Plus simple and obvious choice making like decline in smoking.


60 Eric Rasmusen May 19, 2017 at 2:24 pm

A really good post. I’ll have to come back and reread it and read the comments. I think the Two Paper rule is not that you only have to read two papers, but that if there aren’t even two good papers then the rest of the literature is rubbish. Or, perhaps, that if you don’t get a hint of insight from the vast literature in the best two papers, then the rest of the vast literature must be rubbish. A good paper takes you to read other papers too, so it isn’t the MIT Simple Answer idea (tho I am MIT ’84 myself and very sympathetic).

It really is true that there are vast literatures that are entirely garbage. These must be distinguished from vast literatures that are 90% garbage. So it is definitely worth asking which papers are best.


61 uair01 May 20, 2017 at 10:40 am

I’m trying to read Heidegger’s “Sein und Zeit” right now. I’m getting more and more convinced that the secondary literature about that book (Dreyfus, Carman, Harman, etc.) is much better and more fun to read than the original. So maybe we should change the rule to: Show me two good books summarizing the field (secondary) – rather than – Show me two good articles (primary)


Leave a Comment

Previous post:

Next post: