Do scholars produce too much and revise too little?

by on January 30, 2014 at 7:52 am in Books, Economics, Education, Music, Science | Permalink

Art Carden asks me:

…do you think scholars spend too much time producing new content and too little time revising and refining their arguments? I’m thinking about the Scholars of Old (e.g. Smith) taking their work through multiple editions. Thomas Sowell is good about producing revised versions of some of his books, but a glance at my shelf makes me wonder if Eminent Scholar should’ve revised his/her first book instead of writing a second or third or fourth.Do you think books go through the optimal number of revisions? Is the editing process so good today that the first edition usually should be the last?

In music, Brahms is notorious for having spent a lot of time revising his drafts.  Pierre Boulez explicitly revised and improved many of his pieces, after adding in years of extra work.   Stravinsky’s later 1947 version of Petrouchka is much sharper and cleaner than his 1911 release.  In all of these cases the revisions are worthwhile, because these works were very special and worth improving.  Brahms’s first symphony might have done better with some further revisions yet.

When it comes to economics, individual works are less and less special all the time, unlike in the days of Adam Smith.  Smith waited, more or less perfected his book, and in the meantime he was not really scooped.  Today it is the collective body of work which carries the force.  We also live in the age of the working paper, where it is the first released draft which matters most.  There is an institutional failure that first released drafts are released too soon, for the purposes of receiving attention and credit.  Yet I don’t think there is a corresponding problem of too few editions of the working paper.  Ideally there should be no more than two.  A first “here I am” version, to stimulate discussion and feedback, and eventually a final version which maximizes accuracy.

In fact if scholars had to commit to only two versions of the paper, they might be induced to make the early release more accurate in advance, knowing they cannot magically revise away errors a week later with the magic of word processing and web re-posting.  This move toward fewer editions would offset some of the costs of premature release.  Again, we do not see a case where a greater number of revisions would be better.

The scarcity of attention is the key reason why a very small number of accuracy-enhancing revisions is more or less optimal.

Addendum: I heard once from a random beggar on the street that economics textbooks reach their peak quality in their second or third editions, not so much later on.  They can become too baroque and too overloaded, and the original structure of the book, which hangs over subsequent revisions like a heavy skeleton, can prevent the text from incorporating new ideas and methods of presentation as it ought to.  In this case market incentives may create too much revising, not too little, and capping the number of revisions would lead to increased entry, albeit more spending on fixed costs, higher faculty switching costs, and lower prices for students.  Of course he was a crank.

Michael January 30, 2014 at 8:03 am

I didn’t work in economics, but at least in the humanities, I think the answer is yes. There are a couple of political forces behind this:

1. Fear of looking wrong. Revising and refinement imply not getting it completely right the first time. This isn’t a bad thing in itself, but academic careers are made on a paradoxic mix of agreeing with the old guard and disagreeing with some oppositional thinker. This adversarial approach to knowledge building disincentives any activity that could help people say, “hey, you didn’t get it right the first time!” I think in finance and the hard sciences, the incentives are different, so admitting error there is easier.

2. The pretense of wisdom. If you revise a theory, that means you’re using trial-and-error to get closer to the truth. Again, this is how it works in the sciences, but not so much in the humanities and softer social sciences, where you begin with an ideological paradigm and you use something approximating logic and inductive/deductive reasoning to confirm the validity of the ideological paradigm. The better social scientists shy away from this approach; in the humanities almost no one avoids doing this, which is one big reason why I left academia altogether.

Michael January 30, 2014 at 8:14 am

I forgot to mention–the focus on production also has to do with the need to publish more, and journals and book publishers are much more interested in publishing fresh, new “groundbreaking” research than the duller “we need to rethink this old idea” research. Arguably, the latter is much more important, but is very overlooked.

Thor January 30, 2014 at 11:02 am

Very good points. My understanding is that in the Humanities you start from your ideological position — your “politics” — and then you defend this position by citing the great thinkers in your camp. (Of course, almost no one cites Hayek or Smith, but they do cite Marx, Hegel, Nietzsche and Freud.)

Nathan W January 31, 2014 at 1:24 am

perhaps hayek and smith did not say much of interest to people in the humanities? Smith, after all, spent most of his life focused on moral philosophy, and understanding how and why we do (or do not) account for the situations of others is a central consideration. Given that his “hidden hand” statement has been misquoted and misused so many times in justification of unchecked free market, the philosophical advances made in his works are generally too tarnished to make constructive use of without a) continuing the adulteration of his work or b) having to extensively explain how most readers will misunderstand references to Smith as a result of regular selective misquotation of his thought.

I’ve never quite understood the fascination with Hayek, but I’m willing to leave it at “I’ve never quite understood”.

Albigensian January 30, 2014 at 3:09 pm

Umm, not “trial and error” so much as successively better approximations of perfection, or some other form of continuous improvement.

Surely no scholar is so arrogant as to believe any work is so perfect that it can’t be improved?

Nathan W January 31, 2014 at 1:07 am

I think the need to publish (for tenureship reasons) is a large part of the reason that we find too many scholars focusing, on average, on quantity rather than quality. Teaching-oriented professors should be evaluated based on teaching and research-oriented professors should be evaluated based on publications, and if the school doesn’t make the distinction between the two, then my guess is that the teaching quality is very low in undergrad but that high quality of students and competition may make up for potentially low quality teaching.

Thomas January 31, 2014 at 2:34 am

Is it really the case that tenure committees consider the quantity of published works but not the quality? I’m not in academia but I seem to recall that there are ranking systems in place that should more accurately describe quality, such as how often an author is cited and weighing publications against the importance of the journal. If it were the case the quantity is, in fact, dominating quality, wouldn’t the root of the problem rest at the journals themselves?

Dan Cole January 30, 2014 at 8:18 am

Alex:

I wonder if your random street beggar had been laid off from an academic publishing company. After all, they’re the forces behind more or less constant revisions of textbooks, not for the sake of improvement, but for pure marketing purposes (to curtail the trade in used textbooks that eats into their profits). Many textbooks that might be subject to revision every 10 years or so, if ever, now appear in new editions every 3-4 years. The publishers have positive incentives to enforce this shorter cycle; but the authors may not get so much out of it, which might explain why value-added might deteriorate after the second or third edition.

Best.

Dan

Nick_L January 30, 2014 at 10:02 am

Given that Modern Principles is now in its second edition (working on a third?), I suspect that Alex’s ‘beggar’ is perhaps having a little subtle fun here.

sunbomb January 30, 2014 at 1:06 pm

Tyler wrote this.

Nick_L January 31, 2014 at 10:21 am

I sit corrected, tks.

Squarely Rooted January 30, 2014 at 8:26 am

The best advice from experience I would give to any higher education student is to always buy a textbook two editions previous to the current edition barring other exigent circumstances. If the current edition is going for $150, the previous edition is usually going for $40-60 and the edition prior to that is going for $10. At one book per course, three courses per semester, this can save hundreds of dollars a year at little cost otherwise. If your professor insists only the latest edition will suffice, be not afraid to ask for a justification.

The best professor I ever had in this regard assigned her own book as the sole book for the course, and provided us both with the manuscript pdf (not a scanned copy but the original file) as well as copies of the book gratis. Given her prior professional experience, I expect she was especially attuned to the market failure she could potentially exploit and made a point of doing otherwise.

Non Papa February 2, 2014 at 1:32 pm

Exactly right. Even if the average professor isn’t as conscientious as your example, you can always find a classmate who will let you photocopy the problem sets, which might have changed, from the latest version.

Angus January 30, 2014 at 8:27 am

Was his name Tyrone by any chance?

Warren January 30, 2014 at 10:04 am

That was my first thought.

liberty January 31, 2014 at 8:28 am

doesn’t the comment say “her”?

Vivian Darkbloom January 30, 2014 at 8:39 am

Only ambitious non-entities and hearty mediocrities exhibit their rough drafts. It is like passing around samples of one’s sputum.

dan1111 January 30, 2014 at 11:47 am

I would think that willingness to expose one’s unfinished work to the criticism of others is a good thing.

Vivian Darkbloom January 30, 2014 at 12:09 pm

Well, I guess you didn’t like the prior quote. But, since you raised the issue of critics and
since the post discussed Brahms, Stravinsky, Boulez, Mahler, you might appreciate this:

“Critics make pipi on music and they think they help it grow”

André Gédalge,

dan1111 January 31, 2014 at 5:57 am

Who wouldn’t appreciate that?

James January 30, 2014 at 8:44 am

Looking at the “our books” list rigth here, we can be pretty sure that people are publishing too much and not revising a lot.

MD2 January 30, 2014 at 12:35 pm

Haha, awesome. Are you saying you haven’t pre-ordered a new edition of “Markets and Cultural Voices: Liberty vs. Power in the Lives of Mexican Amate Painters” on Amazon?

Norman Pfyster January 30, 2014 at 9:00 am

From the outside, it looks like most academics publish the same thing over and over with minor revisions (thinking mainly about articles). From that perspective, mostly what they publish are revisions.

Cervantes January 30, 2014 at 9:17 am

Must be the incentive structure, right?
Regarding journal articles, which is what matters for promotion in economics academia, once an academic paper gets the OK by the journal editor, there is no more incentive to revise further. Time gets allocated to the next work in progress.
Now, prior to submission to the journal, there are strong incentives to revise because higher ranked journals have higher quality thresholds. So if I look in a my computer in the folder for a paper that eventually got published, there are a bunch of versions (revisions).

dan1111 January 30, 2014 at 11:55 am

What percentage of academic output is actually ideas that are valuable enough to be worth revising?

I would say that only a very small percentage of work actually makes any difference to the debate in the field. This work does tend to be revised–if not directly then through further published work and responses from other scholars.

For the rest, getting it published is pretty much the beginning and end of its significance. Revising it would not be worth the time, since no one is paying attention. Not to be harsh here–a lot of it is surely good work. But the world of ideas is very crowded, and only a few voices break through to be heard.

Marie January 30, 2014 at 10:48 pm

I wonder if it would be useful for textbook writers and other publishing academics to put out their texts, then Cliff notes themselves.

I remember reading the Durant books and finding it very cool that they used two type sizes, one for the words that should probably be read and one for the words that were optional.

Nathan W January 31, 2014 at 1:09 am

I get the idea, and am inclined to agree.

If it’s not really worth anything, then why try to dress it up like it is? If it’s potentially worth something, then why not work it through until its value is more apparent?

Robert Sams January 30, 2014 at 9:58 am

“The scarcity of attention is the key reason why a very small number of accuracy-enhancing revisions is more or less optimal.” I’d say a large number of accuracy-enhancing revisions is optimal if they can be made by anyone. Attention is scarce, but the single-mind is also myopic.

The paper or book isn’t really the research but a summary of the research, which actually resides in code and data (formal or empirical work at least). If every working paper had to put those in a public repository and database, it would be far easier for others to replicate results, debate the implications, modify it, and make accuracy-enhancing
revisions. But then authorship would no longer have the central place it has, which might explain why we get over-production of written work instead. How much attention was spent over Reinhart and Rogoff before someone bothered to run their numbers?

dearieme January 30, 2014 at 10:06 am

I stopped the development of my only book after the first draft, when I decided that I’d already got as much intellectual satisfaction out of it as I’d hoped for, and actually publishing the ruddy thing would just be a hassle. If the bloke from OUP who’d commissioned it from me was shattered by disappointment, he hid it well.

Marie January 31, 2014 at 5:34 pm

Self-publshing on Kindle is coming into its own, we saw somebody (thanks to whoever it was) here recommend “Wool”, big hit.

You could start the trend for academic publishing, I’m sure that would go over well. No process necessary, just upload what you got.

Willitts January 30, 2014 at 10:18 am

Everyone knows that the only things that needs to be changed in a new textbook edition are the homework questions in the back.

Little wonder why he’s homeless.

Urso January 30, 2014 at 10:40 am

And what of blog posts? They are usually written in an hour or less, and there is an immediate (and hypercritical) feedback from peer, and not-so-peer, reviewers. Seems like they would be especially ripe for the kind of revision you’re talking about.

Yet a co-author here recently said “I don’t usually like to recycle old material.” Perhaps it would be a fun & productive activity for the MR authors to reconsider and revise some randomly selected posts from years past.

tt31 January 30, 2014 at 11:14 am

And to some extent some bloggers treat some of their posts as places to sound out and develop ideas – almost a pre-pre-first draft. I think an issue here is just the general increase in accessible medias for sharing thoughts. The two-draft idea pitched here would seem to me to incentivize more careful first drafts, but I would expect to see some shifting of the current less-careful first drafts to blog posts and other media.

Tom Chambers January 30, 2014 at 10:42 am

My understanding (as an occasional concert program annotator) was that Stravinsky issued two revisions of Firebird because he needed the money. The money came primarily from concert royalties, so he downsized the scoring from the initially rather extravagent version (16 woodwinds, 3 harps) to one suitable for lesser-endowed orchestras. I suspect the same happened to Petrouchka.

The message might be that in the performing arts, revisions might be ‘improvements’ but they might also be made to fill a practical need.

Thor January 30, 2014 at 11:04 am

The beggar was misquoted. He was talking about the “Harry Potter” books.

Let’s see, reach their peak in vol. two or three (check).

Baroque and overloaded in later books (check).

Pierre Menard, Author of the Quixote January 30, 2014 at 12:45 pm

I see that the by line has changed from Alex to Tyler – a significant revision indeed, although to what extent does it really improve on the text?

David Bley January 30, 2014 at 3:20 pm

If books don’t move from theory to practice, the validity of the publication is untested, no matter how many revisions.

Casey Reeder January 30, 2014 at 3:32 pm

Derek Parfit seems to be a scholar who still works according to the older model, releasing just a handful of books during his lifetime so far, which he had worked on and previewed for several decades before they got published. So its not like that kind of work has completely vanished.

Ramagopal January 30, 2014 at 8:58 pm

I recollect reading somewhere that Alfred Marshall was also obsessed with revising his classic textbook, often agonizing over minor revision of passages not worth the time and effort.

I agree with Tyler’s beggar that “economics textbooks reach their peak quality in their second or third editions, not so much later on”. Just 2 examples. When I was a student a popular macroeconomics textbook in my campus was the one authored by Edward Shapiro. I found that the early edition I had was far more lucid than the later editions. A review of a new edition concluded that successive editions are marked by diminishing returns in lucidity. In my opinion the same applies to the latest edition of Paul Heyne’s The Economic Way of Thinking ( with apologies to your GMU colleague)

Pearl Yonick January 30, 2014 at 10:15 pm

George’s Lucas’s revisions of Star Wars were awful.

Nathan W January 31, 2014 at 1:04 am

As a part of revision processes, I do language editing (or translation if the client prefers) of scholarly research, generally in economics and in development. I.e., revision processes are my bread and butter.

I firmly believe that many scholars produce too much garbage and revise too little. Too many good ideas are embarked upon and then discarded before they are developed to a point of usefulness, only in order to shovel out more realms of garbage.

Apologies for the disrespectful tone. Did I make my point? I have every bit of respect for the genius that can flow from immediate inspiration, and entirely encourage everyone to risk looking like an idiot when putting out lots of new ideas, but if it’s worth doing a half decent job then it’s worth doing a great job, imo.

Ged Parker January 31, 2014 at 3:18 am

When I was an undergraduate decades ago I quickly learnt that before publishing their tomes academics had produced much more succinct articles in learned journals; more than sufficient for my assignment purposes

liberty January 31, 2014 at 7:39 am

As brilliant as he is, I think Tyler is the wrong person to ask this question of — he is the kind of person who reads as much as possible rather than closely studying and analyzing a smaller number of sources. I am the latter, and I think that there absolutely is a problem here, and although writing many books and papers is great, scholars should be encouraged to return to works and revise them, and they should also not be allowed to be considered experts in a area in which they have only done research for a single paper…

Marie January 31, 2014 at 2:42 pm

Should they return to published works and revise them so much as do serious and intense revisions before publications, then leave them be unless something new comes up?

Marie January 31, 2014 at 2:44 pm

I mean “before publication”, not “before publications.” There, I’m revised!

Peter Gerdes February 1, 2014 at 9:26 pm

I don’t think that academics spend to little time revising their OWN work. There is a limit to your ability to improve on what you already wrote, after all if you knew how to make your own work better you would have published a better book/paper the first or second time.

What is unfortunate is the lack of effort spent improving the works of others. The closer one moves towards literature (so the humanities are particularly guilty) the more people tend to buy into irrational reverence for the choices of the original author.

For instance, while in the sciences any work that is so compelling that we are still reading it hundreds or thousands of years latter has been continually reworked to isolate the good ideas, eliminate the confusions and present it in a more approachable fashion. We’ve taken the best insights of gallileo, newton and the like and tossed the more convoluted or outright confused (galileo had a bunch of bad reasons for believing in his model of the solar system).

Now it would be crazy to believe that if even in science and mathematics where we have access to the most rigorous standards and thus people have the greatest opportunity to excise mistakes from their own work we still find that much of it is best eliminated and the rest can be simplified hugely. Surely then we must accept that Plato, Kant, or even Shakespeare made a similar number of poor (or at least non-optimal) choices and just as with classical mechanics we could save students much of the time they waste struggling with Plato’s confusions and odd writing style and let them spend that time thinking about the good ideas not all the bad ones. We could even make shakespeare’s plays even better (indeed this is true of any book…no one is so good a writer that others can’t improve on it).

Unfortunately, people are horribly resistant to this and when the suggestion is raised they try and argue that the unity of the original would be ruined or that it would compromise their voice. This argument can’t be right. The very reason that we claim Plato is worth reading is not the historical insight it gives into what one greek dude thought, that would be a historical oddity nothing more. We read plato because we think his work is so insightful that it is still a better use of our time than reading modern philosophy. So to the extent we have evidence plato is worth reading we should be able to recognize when we have successfully extracted that value from Plato.

Unfortunately, humans are simply too skilled at seeing patterns in random data, especially when they have been told it is deep and meaningful. This is why so many people think the bible is worthwhile as a literary work (not merely a biased historical record) even if they don’t believe in it’s god or believe and claim to find hidden codes. As a result we have many people who are convinced of the importance of works like plato simply because they were able to read their own insights into his vague statements.

Comments on this entry are closed.

Previous post:

Next post: