One further thought on the Reinhart and Rogoff fracas

by on April 17, 2013 at 12:41 pm in Economics, Science | Permalink

There is a genuine tension between becoming (and staying) “famous” and expressing all the appropriate levels of agnosticism on issues, which fairly often ought deserve quite an extreme agnosticism (see Mark Thoma on this).  It is hard to do both, and you can see this tension in the writings of most if not all well-known economists, at least in their more public pronouncements.  In the “good old days” that tension could be elided.  Academic discourse took place at relatively closed seminars, no quick responses were required, word traveled slowly, back and forth was much less rapid, and in general transparency was lower all around.

I’ve seen the Reinhart and Rogoff book in airports around the world, even though it is to most people unreadable or at best boring.  Could they have still made a splash if they had changed the title to This Time is Different: Why Inference from Macroeconomic Data is Really, Really Hard?  I don’t think so.

Enter the internet and the blogosphere.  Someone criticizes your work, in this case a body of work which has become very famous and made you very famous.  Do you respond by trying to defend the “fameworthiness” of the work, in which case a gross “rightness” might suffice, or at the very least you will try to outline the defensibility of your position.  Or do you respond by spelling out all of the reasons why one might be agnostic about a difficult issue?

I predict that most famous people will respond by trying to defend the fameworthiness of their work.

We as readers then respond by taking media which produce both fame and transparency — the internet and the economics blogosphere and Twitter — and suddenly wielding them as a weapon for transparency alone.  Obviously something won’t look right.  I don’t want to conclude “the fault is ours,” but it is still worth noting the tension between the mediums we patronize and what they are, to the broader world, actually good for.  It’s as if you showed up to Justin Bieber’s birthday party and started complaining that not everyone in the room deserves to be there.  They probably don’t, and their presence at the party should not cause you to overlook their shortcomings.  Still, it’s also good to be self-aware about one’s own role in uttering such a complaint about the quality of the party.

If you are receiving any public recognition at all, choosing how to present your material is one of the most difficult decisions.

Justin Fox has very good related comments.

GIVCO April 17, 2013 at 12:55 pm

An odd addendum to your opening statement is that, “There is a genuine tension between political appointment and expressing all the appropriate levels of consistency to your academic conclusions.”

Had R&R obtained political appointments in the next GOP administration, an administration that continued deficit spending, R&R would no forget about the 90% debt conclusion. (This is a bipartisan point, e.g., Christina Romer on stimulus, Goolsbee on unemployment benefits.)

GIVCO April 17, 2013 at 12:56 pm

Er, “would no doubtforget about the 90% debt conclusion”.

Widmerpool April 17, 2013 at 12:58 pm

Well sure, you’ll never be a best-selling author (or, say, a famous New York Times columnist) unless you dispense with all the on-the-one hand and on-the-other-hand business. Even dimwitted types like me take that into account all the time. So if you’re saying the best policy advice is perhaps not found in best-selling books or opinion columns of major newspapers, well, yeah.

billyblog April 19, 2013 at 3:15 pm

@widmerpol

This seems like a drive-by shot at Paul Krugman.

Cowen is, of course, doing the same thing, but at least he tries to keep it at dog whistle level — probably for fear of having his head handed to him by Krugman — again.

In any event, I invite Widmerpol to put up evidence of where Krugman has not gone deeply into the reasoning of people with whom he disagrees, often explaining their positions more clearly than they have.

Or shut up.

Jonathan Finegold April 17, 2013 at 1:04 pm

Regarding R&R’s This Time is Different, I didn’t find it “unreadable” and/or “boring.” There are much drier economics books out there, despite its length R&R’s book is not that long (big text and a lot of graphics), and the arguments are pretty general. I guess you have to care about data, because there’s no much of a story in it, but otherwise the book is pretty easy to read and pretty interesting.

Tyler Cowen April 17, 2013 at 1:13 pm

For MR readers, yes, absolutely. For the rest of the world walking through airports…?

Urso April 17, 2013 at 1:05 pm

“It’s as if you showed up to Justin Bieber’s birthday party…”

MIE – Metaphors in Everything

Ritwik April 17, 2013 at 1:25 pm

But Tyler, this is not *that* subtle.

The relevant questions are :

1) Is it ‘weaseling’ to present debt/GDP figures as potential leading indicators of slowing growth and then going back to saying that we’ve only show correlation? YES. After all, showing correlation these days takes one Excel command and cannot possibly lead to global influence.

2) Is it shoddy to commit a coding error when working with an astonishingly tame data set? YES.

3) Is it shoddy to make weighting choices without at least engaging in a reasonably deep meditation on how/why you made those particular choices, what your main alternatives were, and why you made the choices you did? YES.

4) Even if *some* shoddiness could to be excused given the nature of the meta-game at stake, is the leeway smaller for an ex IMF chief economist/ student of Dornbusch-Fischer/top 50 policy economist in the world + a co-author who is recognised as a top 10 leading authority on sovereign debt/ top 50 policy economist in the world? Have R&R violated this leeway? YES & YES.

R&R have flunked the test on all counts that matter, it’s as simple as that. You’re treating their meta-game as if they were purely members of the popular commentariat, ala Jonah Leherer or something. Influence and popularity are different things, and Ken Rogoff doesn’t need to play the latter to ensure the former.

So, is it possible to be deep & solid without being wholly agnostic? Holy god, yes. In macroeconomics, for R&R? Yes! You yourself manage it, almost on a weekly basis.

Your meta-dichotomy simply doesn’t apply.

Millian April 17, 2013 at 5:05 pm

Tyler doesn’t produce macro research results, so of course he can manage it. His role is more that of the people he’s reflecting on in this post: very important in the modern economics world, as is clear from the reach of this blog.

Arthur April 17, 2013 at 1:49 pm

I think you’re defensive and suffering from moody affiliation because I never see you saying something close to this in relation to Krugman.

When Krugman does this you bash him because he’s wrong.

When conservatives do this you take a more subtle approach, studying the incentives, and do not address the factual claim, the substance of the argument at hand.

Do we had any reason to update our priors on debt and growth after seeing the RR paper, even if we did not know the recent disclosures mistakes?

Andrew' April 17, 2013 at 2:32 pm

Wow. TC has done at least 2 nearly identically themed posts on Krugman. When TC thinks Krugman is wrong he points that out. That is not bashing. What a lot of people are doing (and calling for more of) to R&R is on the bashing end of the spectrum.

I thought this is where I could come to get away from the irrationalities and misperceptions of joinerism. It’s worse than anywhere.

Arthur April 17, 2013 at 2:58 pm

Couldn’t find any. Could you provide some links?

This seens bashing to me:

“Let’s put it this way. Paul Krugman is a great economist. But of all the people in my RSS feed, in terms of his quality and skill as a reader, he is not in the top 90 percent.”

Or this:

” He could reinvent himself again — in a truly Humean direction — and become the most important American public intellectual — and perhaps intellectual — of his time. Or he could keep his current status as a sharp and brilliant someone who has an enormous number of followers but relatively little influence over actual events, and perhaps, like most of us, won’t be read much fifty years from now.”

Note that there’s no mentiong of “Krugman is famous so he has bad incentives” in any of this posts.

I think Tyler is right in bashing Krugman. He is wrong frequently. He misrepresents views he disagree with. He’s not the best exemple of public intelectual.

I just wished Tyler held RR to the same standards.

Claudia April 17, 2013 at 8:37 pm

Here’s your link: http://marginalrevolution.com/marginalrevolution/2012/01/krugmans-response-to-alex.html This is not bashing but it’s tussling with Krugman. And there’s some asymmetry with this post on R&R. If you follow the Krugman link there’s an ironic comment about Rogoff too. Small world of economics.

dirk April 17, 2013 at 1:53 pm

We need a new medium for data-driven “books” in this digital age in which many readers are statistically literate. Don’t just show me your charts, give me your source data and let me play with it myself.

That’s my takeaway from this fracas.

Andrew April 17, 2013 at 7:18 pm

This has been a growing demand in the life sciences (and probably physical sciences) for some time now. ‘Reproducible research’ is the new mantra, accompanying the supplementals section in many journals.

Orange14 April 17, 2013 at 1:53 pm

In the field of drug safety we struggle with similar problems and have much more data than macroeconomists do yet our tools are imperfect (false negatives and positives abound and the needle in a haystack problem often interferes). I’ve been disappointed with what I’ve read from R&R (the book was interesting but the core paper at question here is not) and agree that their defensiveness is more a product of the times than one that ought to be subject to so much sturm und drang. One of the things that we did within the pharmaceutical industry was to acknowledge that we had imperfect tools and figure out a way collectively along with academicians and the FDA to improve them. One of my major projects prior to retirement was setting up a not-for-profit effort to do just this (see: http://omop.fnih.org/ ). It takes time and money to solve problems and the economics profession needs to put much more effort into this and spend less time blogging and defending models that don’t work.

Mark Thoma’s cautionary column should be mandatory reading for all who enter the profession.

Brian Donohue April 17, 2013 at 2:04 pm

Great link from Thoma. Macro…yeah.

I wonder what would happen if a country had debt of 500% of GDP. Sadly, there are no data. Agnosticism is the only option.

henge April 17, 2013 at 2:19 pm

I consult Marginal Revolution as a perspective check on other more progressive news sources I tend to read. I checked MR hoping to find out whether R-R had screwed up as badly as Leftish sources had (apparently reasonably) claimed. Instead, there’s a post suggesting it’s just human nature for people to be weasley. Not what I was looking for.

Ritwik’s right; this not a hard case. I don’t do economic research for a living, but I do process complicated information for others, and if I presented to a client a report with a major outright error and some very dubious assumptions that I did not disclose, I would expect to get fired when those facts came to light. I wouldn’t expect a client to say, ‘Hey, I understand you were worried about your reputation and didn’t want to look dumb, so it’s OK’. Why this hand wringing for R-R? They screwed up on an important topic, and they kept silent for as long as they could, even when others were using their very questionable report as the justification for producing policy.

Andrew' April 17, 2013 at 2:43 pm

What NUMBERS do those leftish sites claim?

That is all that matters in how bad a mistake is.

henge April 17, 2013 at 3:58 pm

The NYT quotes from the Herndon et al article:
‘According to their rerunning of the figures, “the average real G.D.P. growth rate for countries carrying a public debt-to-G.D.P. ratio of over 90 percent is actually 2.2 percent, not –0.1 percent,” they write.’
Quite a difference.
http://economix.blogs.nytimes.com/2013/04/16/flaws-are-cited-in-a-landmark-study-on-debt-and-growth/

Krugman produces a chart based on OECD data showing no cliff at 90% debt levels. Again, quite a difference.
http://krugman.blogs.nytimes.com/2013/04/17/further-further-thoughts-on-death-by-excel/

And I do not agree that the only measure of the error is in the numbers. Since I lack the time, inclination and technical ability to independently assess the numbers, in determining whether I am willing to give someone the benefit of the doubt, I consider how the process looks. Here, it looks bad for R-R, so why should I give them the benefit of the doubt?

Andrew' April 17, 2013 at 4:16 pm

That is the total difference which is [error + methodological choice]. Why are the leftists choice of methodology the right one?

I’m not suggesting anyone believe anyone implicitly. I’m suggesting everyone distrust everyone with equanimity. Same as ever.

henge April 17, 2013 at 4:59 pm

I don’t know if the leftists methodology is correct. I do know they exposed a blatant error that R-R had both failed detect and failed to provide others with the information so they could correct it.

If by equanimity, which means calmness and poise, you intend to refer to some concept of equality, then I disagree. If you know nothing of two individuals, A and B, except that one of those individuals, A, failed to catch a blatant error, failed to provide info to others so they could correct the error, and failed to state material, controversial assumptions that supported his position, who would you trust more, A or B? No question, I’d trust B more.

Robert W. April 18, 2013 at 2:56 pm

Andrew,
If you look at the actual spreadsheet you would see that it has nothing to do with ‘leftist’ choices, it has to do with making *any* other choice other than the one that R-R chose. Reinhart/Rogoff’s dramatic results stem almost entirely from including New Zealand from 1951 on when they apparently had a severe recession and excluding New Zealand from 1946 to 1950. Now, I have no idea what was happening in New Zealand post WWII however I do know that it is silly make a generalized rule about the impact debt/gdp ratios on future growth based on data that is wildly skewed by 1 years worth of data from one minuscule economy. That’s essentially what R-R have done. If you slice the data any other way, the 90% ‘tipping point’ disappears. Does that seem reasonable to you?

Steko April 17, 2013 at 8:24 pm

Sorry you don’t get to decide for everyone what really matters.

What really matters to me is that the entire thing was exposed as an elaborate exercise in bullshit. If the best case for austerity you can make is bad cherry picking, bad statistics and “sloppy” excel formulae then (here comes the science) there’s no reason anyone should buy what they’re selling.

If you want to spin some good news out of it I guess now there’s room for another picture of little Tina on the mantle.

Ricardo April 17, 2013 at 10:30 pm

Your standards for quality are appalling. This agnostic position Tyler is taking and you so forcefully defend is completely ridiculous. I can only conclude that you are approaching this debt/GDP issue ideologically instead of with a critical mind. What you call R&R’s “methodological choice” would never have made it past a journal referee had it been disclosed, but they did not say anything.

Let me ask a very simple question: will you at least condemn the fact they were not transparent in their “methodological choices”?

LarryM April 17, 2013 at 6:29 pm

See, here’s part of the problem.

For people who know Tyler, reading his posts on the topics, he IS saying that R-R screwed up as badly as leftish sources claimed. But he insists upon … not so much holding people on his left to a different standard, but he uses different rhetorical tools when addressing such claims on the left. More plain speaking, as it were. Whereas for people on the right, he relies to a larger extent upon subtlety and relying upon he audience to be careful readers.

LarryM April 17, 2013 at 6:45 pm

Not to put too fine a point on it, but R&R themselves essentially admit that their critics got it right. Their response would have been embarrassing coming from PhD candidates, let alone economists of their stature. “Yeah, we screwed up in the very basic ways that our critics indicated, and yes it’s true that, despite public statements from us which ignore this important distinction, our study doesn’t have anything to say about the direction of causation, but our study is still important, because even if you remove the glaring errors that an undergraduate likely wouldn’t make, there’s still a strong correlation.”

Heck, given the huge problems with causation in the first place, you can almost argue that the mistakes don’t matter. Which was Tyler’s initial point. But that’s damning with faint praise.

Brandon April 17, 2013 at 8:13 pm

They didn’t keep quiet; they published op-eds and gave congressional testimony.

Rahul April 17, 2013 at 2:19 pm

I think Tyler’s over-complicating matters and needlessly being defensive. Yes, publicity is a whore but each one of us has to use his own conscience. If there are reasons to be agnostic, say so clearly. Don’t take indefensibly certain positions if they are not justified. Separate your analysis from your ideology. Don’t hunt for data that selectively supports your pet thesis.

If you are going for sensational snippets you are a mercenary media pundit not a serious academic.

They made a mistake. A pretty bad one. And they should graciously admit that they did. The matter really isn’t very nuanced.

Andrew' April 17, 2013 at 2:36 pm

Are people reading some secret TC blog somewhere and then commenting here?

Anyway, they made a bad mistake?

How bad is their mistake?

Rahul April 17, 2013 at 3:11 pm

“On the first point, we reiterate that Herndon, Ash and Pollin accurately point out the coding error that omits several countries from the averages in figure 2. Full stop. HAP are on point. Nevertheless, the mistake in figure 2 resulting from the coding error is a significant lapse on our part”

How bad? “Significant lapse”-bad. BTW, I think overall RR are doing the right thing. What’s annoying is Tyler’s current post. What’s his point?

wrparks April 17, 2013 at 3:30 pm

“Significant lapse” bad –

And in research, as we all know, significance is all that really matters.

Andrew' April 17, 2013 at 4:14 pm

That’s a mistake. And it is as bad as the impact on the number. It sounds like that is 0.3. The other 1.9 apparently comes from methodology differences. Who knows what is the right methodology? That may not even be all that significant. It sounds large, but all it probably does is shift the 90% inflection point to the right. What matters is the shape of the curve.

Popeye April 17, 2013 at 11:38 pm

If the methodology makes a 2-point difference, then the scientific answer is “we don’t know, but like lots of other people we have opinions.” Economists have a professional responsibility to be honest about the robustness of their results.

Ah, who am I kidding, the whole enterprise is a thin layer of rational bullshit over a mountain of ideology.

CG April 17, 2013 at 2:44 pm

Why on earth would this ever be a question for a serious economist or a serious academic of any sort? Priority #1 is accuracy. As soon as you distort the quality of your work for fame, popularity, or selling books, you’ve lost credibility. It should never be a difficult decision for someone who is seriously concerned with the truth. Truth and accuracy should be absolutely above any considerations of making the work seem interesting to the masses. That’s what we have the media for. It’s no surprise that when academics become a member of the media, the quality of their scholarship declines (exhibit A, Paul Krugman, among countless others).

Andrew' April 17, 2013 at 3:20 pm

1. The granting system. 2. See #1.

Andrew' April 17, 2013 at 2:54 pm

Sounds like there is a relatively small error and then a methodology difference that does not provide discrepancies nearly as large in magnitude compared to all the “multipliers” we heard about. Noone got flayed alive over the fact that they were all wrong.

Andrew' April 17, 2013 at 3:20 pm

http://en.wikipedia.org/wiki/List_of_countries_by_public_debt

People (see Justin Fox) keep referring to Europe having trouble because of Austerity and assuming we aren’t because we aren’t doing Austerity. But a gander at the public debt, it seems like that’s a pretty good indicator for who is in trouble. Did anyone ever say it was the only variable?

F.F. Wiley April 17, 2013 at 3:13 pm

This is interesting but I’m not convinced that fame vs. empirical caution helps us to understand this particular case. It seems to me that R&R genuinely and strongly believe in a causal relationship between debt and growth, on conceptual grounds, and that they surely haven’t changed their beliefs on the basis of a small calc error on one metric over one time period in one paper (among the variety of metrics, time periods, and studies that they’ve used to back up their arguments).

If it weren’t for the Excel error, which was unfortunate but didn’t have a significant effect on the results, would anyone even be paying attention? Take away the error, and all that’s left is an academic debate about weighting schemes and data sets (and imo R&R had a solid answer for the omitted data) that doesn’t even change the slope of the line.

I guess what I’m saying is that I never saw R&R as fame-seekers, nor did I see their work as skewed one way or the other – they merely reported on some means and medians on data that we all have access to and can check for ourselves. And they acknowledged that correlation isn’t causation. IMO, those who understood the paper never saw 90% as a magic number – it was the arbitrary upper-bound on their last bucket and seemed a reasonable level to flag given the results and the fact that the >90% bucket showed lower growth (and still does).

Eli Rabett April 18, 2013 at 12:18 am

The Excel error and the single year NZ data point had significant effects

F.F. Wiley April 18, 2013 at 3:58 pm

ER,
Per the paper, the Excel error had an effect of 0.3%. My point about the omissions is that R&R had a solid answer in that they hadn’t completed the data set (I know that the N.Z. data, in particular, could have been a problem because I’ve checked a bunch of sources for my research and they don’t line up). I do agree, though, that the 1951 N.Z. growth figure was given too much weight with the other years omitted and this moved the average by more than a percent.

Hazel Meade April 17, 2013 at 3:22 pm

Economics needs to split into two fields.

Microeconomics is useful, interesting and thoroughly grounded in psychology and neuroscience.

It’s the macroeconomics part that is dismal. Spin that off and let them team up with astrologers and climate scientists.

Arthur April 17, 2013 at 4:35 pm

“thoroughly grounded in psychology and neuroscience”?

The Original D April 17, 2013 at 10:10 pm

Marketers make use of behavioral and micro every day. Businesses are built on understanding the psychology of persuasion.

Can you imagine if every decision businesses make was subject to a point-counterpoint argument on Fox News and CNN? That’s basically how governments do it.

Claudia April 17, 2013 at 3:24 pm

I am disappointed in the post. I don’t think R&R should be bashed, as I think they simply remind us of a set of socially bad incentives in economics. Shouldn’t we think about how to change these incentives, so we don’t make life so tough for famous economists?

Rahul April 17, 2013 at 3:51 pm

Is this sarcasm? If not then what is this ” set of socially bad incentives in economics”?

Claudia April 17, 2013 at 4:06 pm

Here’s a related twitter conversation I had on this today: https://twitter.com/ErikAngner/status/324518496060977152

And a good set of ideas from Cathy O’Neil: http://mathbabe.org/2013/04/17/global-move-to-austerity-based-on-mistake-in-excel/#comment-25492

Also see my comment in yesterday’s post. This is pervasive issue with empirical work that can be better addressed.

PS by socially bad incentives … I mean everyone acting according to their personal incentives ends up creating a bad outcome for many.

prior_approval April 17, 2013 at 4:15 pm

‘I mean everyone acting according to their personal incentives ends up creating a bad outcome for many.’

Clearly, you will never be offered a nice position at a GMU affiliated econ policy think tank with that sort of attitude.

Rahul April 17, 2013 at 4:21 pm

I again think you are over-complicating the issue just like Tyler is.

Claudia April 17, 2013 at 4:38 pm

I disagree, Rahul. This is not about R&R. There are lots of big name economists that carry with them data transgressions (almost always the RA’s fault or not a big deal). What do you think happens in a profession that puts almost zero weight on replication? I learned about a procedure in grad school and was told on the side you had to have a certain set of last names to get it to work. Hmm. Or that rewards people for finding proprietary data sets that no one else can access. R&R were a lot more open than many economists…can you guess how the incentives will naturally shift in their example? Or that lets papers circulate to the popular press for years without peer review? And I think the blogs are making this all worse not for the transparency but for encouraging over-reaching, over-simplified research findings. Why aren’t blogs a part of the critical review process and not just a cheap conduit? I write a fair number of referee reports and I suspect someone here could bang out a few instructive paragraphs pretty fast. Posting abstracts that oversell is not helping the incentives. R&R is not a surprise and it’s not unique and that’s a problem.

Ritwik April 17, 2013 at 6:05 pm

Claudia

Here’s the reason this time is different. Reinhart and, expecially, Rogoff, are not people who have to accept the profession’s default incentives. They are not price-takers in this market. Rogoff was the IMF chief economist. It is very difficult to argue that Rogoff and Reinhart have to be evaluated by a set of standards which seem more applicable to a graduate student desperate to finish his/her PhD.

We don’t need to go meta here and think about systemic changes and the like. Who is this ‘we’? Not all responsibilities have to collapse into collective action problems. Of all the people who can conceivably change/ignore without much risk the modal reward system in eco, wouldn’t Rogoff and Reinhart rank near the very top? Why such diminished expectatiosn from our elite?

Further, even if the ultimate solution is to change the reward system, within the context of that reward system, do you not agree that R&R choices smack of non-robust and very questionable intellecutal choices? If so, they fully deserve the barrage of ridicule coming their way.

If you like, you can think of this public ridicule as an essential step in the process of ‘we’ changing the reward system in eco. Referee process changes may be too slow. Maybe next time, the fear of public ridicule will keep researchers honest to god.

Claudia April 17, 2013 at 6:42 pm

Ritwik, you’re right. This is about R&R. They are uniquely bad in their analysis and their public flogging will lead to more open, honest scholarship.

(Rahul, that’s my sarcasm.)

RZ0 April 17, 2013 at 3:25 pm

As a spreadsheet modeler, I question the chops of anyone who makes an error as trivial as this and leaves it uncommented on for three years. I question why R-R didn’t double check their work when others struggled to replicate it. I wonder who would dare to engage them on substantive work in the future.

prior_approval April 17, 2013 at 4:13 pm

A bit of the touchiness in the post might be explained by old web links like this –

‘Mercatus Center Mercatus Center ‏@mercatus

Our nation’s high levels of debt are crowding out private investment, raising costs to business and stifling growth http://bit.ly/PQGF05

1:33 PM – 30 Aug 12′ https://twitter.com/mercatus/status/241272369052807168

The bit.ly link is to a site which is filtered here, but the title of the publication is ‘Governing to Win: Enhancing National Competitiveness Through New Policy and Operating Approaches’ with a chapter by Jason J. Fichtner.

The comment which follows is this one – ‘Daryl Montgomery ‏@nyinvesting 30 Aug

@mercatus Reinhart and Rogoff put the stiffling cutoff at 90% of debt to GDP. The US is now over 100%. Growth will be difficult from now on.’

Basically, a lot of snake oil has been sold, and right now, the glare of publicity reveals just what the salesman was selling – factually incorrect data which had been seemingly manipulated.

Of course everyone involved in the same business is concerned, but to publicly acknowledge that concern could be a bit damaging when it comes to remaining competitive in the snake oil marketplace.

(Or from another perspective, perhaps Prof. Cowen has always been aware of how the game is played, it is just he never expected such a public factual repudation of a tenet which is part of the broad current he has participated in framing. A current framed only for the public good, of course – shame that it turned out to be wrong, but mistakes happen. Luckily, at least one author at this site has already said that facts aren’t important, so he is covered in this case, tool. Well, except from the spillover of public ridicule. Unlike the case of Blahous’s invented ‘double counting’, where the ridicule came from people who were already familiar with the issue.)

prior_approval April 17, 2013 at 4:43 pm

Sorry for the typo – ‘too,’ not ‘tool.’

mobile April 17, 2013 at 4:25 pm

My reaction to this post was “Damn, guess I’ll have to start reading Krugman more charitably.”

prior_approval April 17, 2013 at 4:28 pm

Yeah, almost as if a The Sveriges Riksbank Prize in Economic Sciences in Memory of
Alfred Nobel winner might actually be worth listening to.

But hey, who are you going to believe – Krugman (while fully acknowledging he makes mistakes, generally in the full glare of publicity), or people who can’t get their math straight while apparently cherry picking their data. And no, I not referring to the current fracas.

prior_approval April 17, 2013 at 4:25 pm

Oops – I guess this paper needs some reworking too – ‘Projections of U.S. Public Debt Continue to Accelerate’ (sorry, filtered link) by Veronique de Rugy

‘This means that the U.S. is slipping down an unsustainable fiscal path at a much faster rate than before. This unforeseen acceleration in the public debt is important because high levels of debt can have a negative impact on the economy. Economists Carmen Reinhart and Ken Rogoff have shown that when a country’s level of debt reaches 90 percent of GDP, its economy could start shrinking by 1 percentage point every year. The U.S reached this milestone four years ago. One percentage point may not seem like a lot, but as Mercatus Senior Research Fellow Matthew Mitchell has shown, if the U.S. had reached this point in 1975, our standard of living could be 30 percent lower than it is today.’

Obviously, one will expect a prompt reworking of the math in this paper, so it at least reflects reality, instead of the current fantasy it projects.

Of course Prof. Cowen is concerned – the Mercatus Center faces a lot of work in ensuring that its scholarship retains the high standard for factual accuracy which it feels it deserves to be granted.

Wait, some of you don’t believe that the Mercatus Center is going to put any effort at all into correcting its work? And to think I used to be paid to write the PR to handle the concerns of such uncharitable cynics as yourselves otherwise.

Arthur April 17, 2013 at 4:41 pm

Talking about public economists facing bad incentives…

prior_approval April 17, 2013 at 4:32 pm

Just a test to see if Mercatus Center links are still filtered, like the Pirate Bay’s – http://mercatus.org/publication/projections-us-public-debt-continue-accelerate

prior_approval April 17, 2013 at 4:39 pm

Well, it is always good to test statements to see if they are still true. Especially because the Mercatus Center is a veritable font when it comes to quoting leading sources of information. Just searching for all the publications which mention this now false assumption about public debt should be revealing, especially when comparing a current list of googled results, and what changes in the next several months using the same saved query.

I’m sure someone at the Mercatus Center has already started on correcting them, of course. Or will get started soon. Or will start as soon as the project is properly defined within the framework of the center’s goals and requirements. And the necessary budget is approved – after going through the correct procedures, of course. Including assigning it the proper priority compared to all the other valuable work that the center performs.

No wonder this post tiptoes its way past the reality of what this ‘fracas’ reveals.

Arthur April 17, 2013 at 4:52 pm

“The ninety percent figure is important. At that point, according to economists Carmen Reinhart and Kenneth Rogoff, debt begins to
hamper economic growth.”

http://mercatus.org/sites/default/files/publication/A-Balanced-Budget-Amendment.pdf

“Past a certain point, the debt becomes so large that economic growth begins to slow—a fact that is well known among economists. Harvard economists Carmen M. Reinhart and Kenneth S. Rogoff made this relationship apparent in their 2010 study.”

http://mercatus.org/publication/spending-time-debt

“This is not just theory. Economists Carmen Reinhart and Kenneth Rogoff have examined 200-years’ worth of data from over 40 countries. They found that those nations with gross debt in excess of 90 percent of GDP tend to grow about 1 percentage point slower than otherwise (the U.S. gross debt-to-GDP ratio has been in excess of 90 percent since 2010)”

http://neighborhoodeffects.mercatus.org/2012/12/21/dont-like-the-fiscal-cliff-youll-hate-the-fiscal-future/

“Using a historical data set spanning forty-four countries and two hundred years, their findings are startling. Across wealthy and poor countries, the median growth rates for countries with publicly held debt exceeding 90 percent of gross domestic product are roughly one percent lower than they would be otherwise”

http://mercatus.org/sites/default/files/publication/de%20Rugy.house_.ways_.means_.3.30.11.pdf

“As shown in Figure 1, the Congressional Budget Office estimates that, absent fundamental changes, the
U.S. debt-to-GDP ratio will reach 90 percent within 7 years and be over 100 percent by 2021.5
These
figures are important. Examining 200-years’ worth of data from over 40 countries, Carmen Reinhart and Kenneth Rogoff have found that debt-to-GDP ratios in excess of 90 percent tend to be associated with economic growth that is about 1 percentage point slower than otherwise.”

http://mercatus.org/sites/default/files/publication/The_Real_Debt_Crisis_Mitchell_07_27_2011.pdf

Marc Kagan April 17, 2013 at 4:38 pm

Maybe you should focus on presenting it correctly. Unless I misunderstand, I thought that was the goal of academics – to properly explain the world.

prior_approval April 17, 2013 at 4:56 pm

Here’s another one needing correction, just clicking on the links from the last factually incorrect Mercatus paper to another by Veronique de Rugy – http://mercatus.org/expert_commentary/debt-ceiling-fight-club

‘Perhaps more important, economists have shown that in some countries (excluding Japan) very high levels of debt haven’t led to a systematically large increase in interest rates. In a recent National Bureau of Economics working paper called “Debt Overhangs: Past and Present,” economists Carmen Reinhart, Vincent Reinhart, and Kenneth Rogoff show that in 11 of the 26 cases in their sample, countries in which debt exceeded 90 percent of GDP for at least five years did not experience an increase in interest rates.

But stable interest rates are not a sign that these countries are in good shape. Economic growth in the 26 cases was 1.2 percentage points lower than in other periods, “the average duration of debt-overhang episodes is 23 years, and it produces a ‘massive’ shortfall in output that is almost one-quarter less, on average, than in low-debt periods.” In other words, the fact that bond markets are blasé about high levels of debt in countries perceived as safe tells you very little about how well they are doing. It certainly should not be mistaken for a signal that the government can borrow more without risk.’

One could quickly sense a theme, back in the fall of 2012 – though I’m sure that the Mercatus Center will expend as much energy correcting its flawed work as it did in created it in the first place. Anyone remember what was happening back in the fall of 2012? Were we having some sort of public debate about economic issues, maybe along the lines of an election?

prior_approval April 17, 2013 at 5:04 pm

One last one, from another but related address – (could be filtering still works on variants of the address – in this case, ‘neighborhoodeffects.’ goes in front of mercatus.org/2011/09/16/when-does-each-state’s-debt-reach-90-percent-of-gdp/ –

‘Few economic studies have received as much attention as Carmen Reinhart and Kenneth Rogoff’s “Growth in a Time of Debt.” The attention is well-deserved. Reinhart and Rogoff have painstakingly assembled data on debt from over 40 countries covering 200 years, making it the largest dataset of its kind. They then examine the way debt impacts economic growth and find that as nations’ debt-to-GDP ratios go from 30 to 90 percent, their growth rates decline markedly. In the best case, their real average annual growth rates decline by about 1 percentage point; in the worst case, the growth rate is halved.

To put this in perspective, if in 1975, the US growth rate had slowed by 1 percentage point, our current economy would be about 30 percent smaller than it actually is. And if our growth rate had been cut in half, then the current economy would be nearly 43 percent smaller than it actually is. In other words, an entire generation’s worth of economic growth would not have occurred.’

Almost like Ouroboros, with one Mercatus scholar quoting another, all based on factually incorrect information.

It shouldn’t be too hard to find others – it only took me a few minutes, after all.

DIS April 17, 2013 at 7:40 pm
John B. Chilton April 17, 2013 at 8:57 pm

Two nonrhetorical questions:

1. Do only unfamous economics list their AER Papers & Proceedings publications as AER Papers & Proceedings?
http://scholar.harvard.edu/rogoff/publications
http://scholar.harvard.edu/files/rogoff/files/cv_rogoff_040113.pdf
T&P committees certainly don’t want the ambiguity.
2. Does AERP&P have the same data sharing requirements as AER?

I didn’t see any indication in their response to HAP that what R&R call the AER paper is AERP&P.

perfectlyGoodInk April 17, 2013 at 11:35 pm

As Bergmann noted seven years ago, “Material gain and career advancement also come into play. Any particular economic policy holds the possibility of adding to the income and wealth of some people and subtracting from the income and wealth of others. Politicians beholden to particular groups in the population can pick and choose among economists. This gives an incentive for economists to divide themselves into two camps, so that whichever party comes to power will have a ready-made corps of economists will ing and ready to recommend policies congenial to the party’s constituency.”

Just look at the history of macro thought. Classical vs. Keynesian vs. Monetarist vs. New Classical vs. New Keynesian (plus the Austrians and Institutionalists). Macro is and always has been just one big political argument over whether or not there should be more or less government intervention in markets.

Eli Rabett April 18, 2013 at 12:04 am

So R&R point to another study they did with a third party as justification, without answering the question as to whether that data/software had the same problems. This is particularly worrying because the first cut is to compare the new stuff with the old stuff and if they agree to say, must be ok. There are also significant issues that were raised with the data itself that R&R have not answered.

Eli’s polite inquiries at the WSJ on these matters were tossed down the memory hole.

PS: Also worth noting that HAP only asked for the actual R&R spreadsheet after they could not get the R&R results using publicly accessible data.

Politics Debunked April 18, 2013 at 10:06 am

It is unfortunate that people focus on errors in this one paper as if it would have a huge policy impact, while other work isn’t being critiqued with far more policy implications (since in this case there are other papers say debt is bad, the details won’t make much of a difference policy wise). Of far more concern is that the government is basing its concrete planning on work that is far more questionable than the Reinhart paper, but people don’t bother looking into it. The Social Security administration has a history of being off on its forecasts, and the techniques it uses are very questionable. This page goes into gory detail, pointing out:

http://www.politicsdebunked.com/article-list/ssaestimates
“The 1950 annual SSA report projected US population in 2000 to be worst case 199 million, best case 173 million. In reality it turned out to be 282 million, 42% above their highest projection. Imagine if their real costs turn out to be 42% higher than their current projections”

before getting into concrete problems with its current approach For instance the “high cost scenario” is high cost in nominal dollars, but when adjusted for inflation it becomes the *low cost* scenario, indicating something is wrong, among many more concrete problems like its implication it can forecast GDP out to 2090 more accurately than other entities can for 2-5 years, in fact more accurately than it is measured (based on the GDI-GDP statistical discrepency).

Greg Ransom April 18, 2013 at 11:39 am

What?

biuro rachunkowe sosnowiec April 19, 2013 at 12:40 am

I have to thank you for the efforts you’ve put in writing this website. I am hoping to check out the same high-grade blog posts from you in the future as well. In fact, your creative writing abilities has inspired me to get my own, personal site now ;)

Chad April 19, 2013 at 1:58 pm

For a number of reasons including those outlined here, RR scandal is the best thing to happen in a long time (http://cbias.blogspot.com/2013/04/reinhartrogoff-best-thing-to-happen-to.html)

Getting Ezra Klein, Matt Yglesias, Tyler Cowen, Neil Irwin, Paul Krugman, et al to all start saying cogent, thoughtful things on economics studies and how they’re reported? We’re all better off for this having happened.

J. Hansen April 19, 2013 at 4:23 pm

Just to go on record here and say that, yes, while the tension you mention is real, that is not what’s going on with the R&R scandal. R&R did an extraordinarily sloppy statistical analysis in Excel – I’m not talking about the error. I’m talking about the averaging of values that were already previously averaged and represented different time-frame weights – New Zealand’s single year of -7.9% and the UK’s 19 years of 2.4% (on average). These two numbers were then simplistically averaged (with a few others with the same problem) to produce the now-infamous “-0.1%”. R&R still claim that they always preferred the “mean” from this same series of unweighted averages: 2.2%. But *both* of these numbers come from a series of values that are obviously of dubious comparable weights – at least, this should be obvious to anyone who understands much of anything about statistics – and it is intuitively obvious to everyone else. After all, just consider this bizarrely structured contest with an utterly simple question at the end:

10 people sit down to eat pies in a pie eating competition that will happen over 10 consecutive days. Each day, each person eats as much pie as they want – but the pies are huge, so the % of the pies eaten by each person are values like 2% at one sitting, and 2.5% at another sitting. But the organizers also kinda made a mistake and the size of each contestant’s pie vary from day to day – that is, on Day 1, contestant #1 gets a pie with a volume of 50 gallons, while contestant #2 gets a pie with a volume of 100 gallons. Then on Day 2, contestant #1 gets a pie with a volume of 40 gallons, while contestant #2 gets a pie with a volume of 130 gallons. At the end of each sitting, the people running the contest calculate the percentage that person ate that day, then throw out the un-eaten portion of pie. (It is a very wasteful contest.) Contestants are not required to show up each day and there was never a day when all the contestants ate pie that day. At the end of the contest, some contestants had eaten pie on just one day, while others had eaten pie all 10 days. The organizers averaged the percentages each contestant over the 10 days, so if that contestant only had 6 days of pie-eating, they added up the 6 values and divided by 6, while the contestant that only ate pie on one day only had one value, so there was nothing to average – that contestant’s “average” was just the one value. Then the organizers made a list of all 10 contestants, and computed the average and the mean for all 10 contestants’ average. The numbers ended up being 2% (average) and 4.5% (mean), and the organizers then announced the results, saying that, “When you have a huge pie and give any one, randomly chosen person off the street the opportunity to eat as much of it as they want, if any, on average they will eat 2% of the pie, but the mean amount of pie eaten will be 4.5%.”

And the question is, “Do you think this conclusion is correct?” IE, “Do you think this was a valid method for deciding how much of a huge pie a random person would eat?”

You don’t have to go to MIT or Harvard to intuitively grasp that this “study” was performed all wrong, and the concluding “statistics” are BOTH utterly meaningless. And if you can’t admit that when the problem is laid out for you, you don’t get to teach at MIT or Harvard, and you don’t even get to teach Economics – not even to people in High School. You are in the wrong profession.

Yet R&R have been sticking to their conclusions. And in 2011 they even provided a closed-door meeting with the US Congress where they essentially stated, when congress-people probed about how much pie people eat and what we should do about it, “Yes, people will eat a lot more pie if the pies you provide them are smaller.” And the congress-people believed them!

(if your head hurts as you try to figure out whether I’ve broken the analogy when equivocating between “Yes, reduce spending now to prevent poor growth later,” with the statement, “Yes, people will eat a lot more pie if the pies you provide them are smaller” – that’s the point. It hurts your brain because your brain can intuitively see that this whole “study” and the way they “analyzed” it is completely inept.)

Tom April 22, 2013 at 4:14 am

The desire of big-gov’t partisans like most Dems and Krugman is to spend more gov’t money, irrespective of the rates of return.
More wasted gov’t boondoggles like Solyndra, in order to “increase Aggregate Demand”.

The desire of smaller-gov’t partisans like many Reps is to reduce gov’t spending, despite possibly missing opportunities of higher growth, but using excessive debt as a reason to oppose increased gov’t spending.

Most macro talk attempts to avoid the political partisanship, but their work is inevitably evaluated based on partisan usefulness.

Krugman was not afraid to call out against Bush tax cuts — despite the reality of those cuts boosting Ag Demand after the dot.com bubble pop. Krugman was fairly quiet in 2002-2006:2008 about ways to reduce the housing bubble before the financial crisis.

Krugman favors gov’t spending over tax cuts. There should be more macro research clearly indicating how much more often tax cuts improve Ag demand, per level of increased debt, rather than gov’t spending.

Comments on this entry are closed.

Previous post:

Next post: