A more complete response from Reinhart and Rogoff

You can read it here, it is too long and the formatting is too complicated to reproduce in full on MR (which is what I ideally would like to do), so please do read it.


Incoherent misdirection. It's almost as if Tyler is secretly an austerian neanderthal.

For posting their response? You people are officially certifiable. Apologies if you are kidding.

Examine your priors Wu. You feel Keynesianism works, and therefore any attempt to rein in the state must be evil. So you protect your own bias to others, who may or may not share them. In fact TC is very even keel--he disagrees with the right as much as he does the left. We are in a new age now--the carnival of ideologies should be apparent to all by now (an excellent mural on this theme is found by this artist: José Clemente Orozco (1883-1949) "Carnival of the Ideologies," at the Palacio de Gobierno, Guadalajara, Mexico)

Let's keep it simple. TC is informing us of this debate, and by the way siding marginally against R&R, or at least slightly on the side of guilty until proven innocent. To do even this is to brand him an enemy by association by simply acting as a fair broker. That's why I leave open the possibility that Wu is satirizing.

The usefulness and relevance of empirical studies can vary a whole lot, and are critically dependent on the question being asked, how the data is organized, how the data is distributed, etc. I applaud Rogoff & Reinhart for trying to create an exhaustive empirical data-set and analysis of historical crises & debt, but they unfortunately fall short in two important ways: analysis & organization.

In their paper they try to confirm through empirical evidence something quite intuitive: too much debt is bad. But their methods end up being far too simplistic for what is a very complicated subject. They don't consider crucial factors such as whether or not a state is the sovereign issuer of their currency (the difference between eurozone countries & otherrs), owes debt denominated in a foreign currency, and even INTEREST RATES and MONETARY POLICY. These factors are mentioned very rarely in "This Time Is Different", and for the most part play no role in the filtering or organization of their data.

And of course this comment by Tyler in his earlier post is also relevant and related to those factors listed above:
7. My own view, as you can read in The Great Stagnation, is that the primary mechanism is slow growth causing high debt/gdp ratios, not vice versa. In any case this is by far the most important issue, whether or not you agree with my take on it.

My view is similar. I think their data set is valuable. I think the analysis they did with that data set is incredibly simplistic for a variety of reasons (some of which you list). Even if you fix the Excel error, so many things are missing that it's hard to ascribe any value to the numbers that pop out, no matter if they're R&R's numbers, or H, A, & P's. Personally, I think it's a horribly misspecified model. I wouldn't trust any number that comes out of it, no matter if it confirms or contradicted my personal beliefs (or my preferred theoretical model).

poster child for open access to data & models

should be the standard by default for papers

Ironically Rogoff et al. complained in their book TTID that governments did not supply them with data, and they had a hard time acquiring the data for their excellent book. Then they turn around and deny this data to researchers until pressured. I guess it's only natural to want to make data proprietary. There was a link the other day on BBC about how magicians tricks are kept proprietary and there's a "Code of Honor" to keep tricks secret.

"If you are good at something, never do it for free." The Joker.

No ungated link?

I guess they didn't learn their lesson.

Yglesias twittered this link. I think is the same response:


Can somebody please tell me where to find the Conference Board data? I've used the links to their site but I do not find any debt ratio data in any of the files. Plenty of GDP data, as well as productivity, etc. but nowhere do I see any debt data.

This is my third attempt to submit this comment (I click submit and then it appears to be submitted but the comment never appears). Anyway, I am looking for someone to show me where to get the data that Krugman used - I've scoured the Total Economy Database at the Conference Board and can find plenty of data on GDP growth, but I see nothing at all about debt ratios. Searching their website turns up nothing either. Please point me to the data on debt ratios.

To put it simple:

They admit the excel mistake, but say it don't affect the result.

They say they didn't exclude data. That the data points that were not considered were not in the database at the time of the paper. Or they still had compatibility issues with this data and the later years/ different country data.

And they say the weighting is fine.

They also say they didn't focus on the mean, they published all countries info and an almost correct median.

My take still is that this is all rubbish.

The groupings are arbitrary, there are no methodological discussion about the groupings. No discussion about reverse causality. No proper discussion of the weighting. No admition that even excluding all this the results are not robust to the inclusion of a few data points.

As I've said, it's a shame that a paper with so many methodological failures got so much attention. Even before these mistakes were discovered, we economists should have known better.

"They admit the excel mistake, but say it don’t affect the result." Jesu, just like Climate Scientists. Except they tend to deny the mistake for several years first.

I'm trying to care about this, but it's hard. I don't think the correlation is especially meaningful in the larger picture of things (ahem Acemoglu) except in the sense that one is a proxy for the other -- lack of growth and indebtedness have similar underlying causes, but you can't get drunk by throwing up.

They keep referring to their "AER" paper which is really a five-page note in the unrefereed Papers and Proceedings issue.

And to their paper in JEP which solicits manuscripts and again avoids a real refereeing process.

And of course their book also is not refereed.

Seems like this whole episode shows the value of real academic publications that have to survive the refereeing process.

Idk, I don't find this controversy so remarkable from a policy perspective. Even the "HAP" paper shows a clear inverse correlation between public debt and growth. One can observe that correlation is not causation. But there's nothing that shows the reverse, that taking on more debt is correlated with increased growth. So other than on a very sophomoric level of zero sum reasoning (my opponent made an error, so I win (even though I have no empirical evidence to support my position)). Maybe the high debt is caused by slow growth. But even if so, if you look at your economy and see it has slow growth, what is the cost / benefit of asking it to service more debt? And what is the degree of confidence in the estimate?

More specifically, what data robustly show that nations with a 90% or worse debt / GDP level can run a persistent, large fiscal deficit, as the Krugmanits cry out for, and achieve better than 2.2% real GDP growth over an extended period of time? Or, put another way, if we imagine Nation X at the beginning of year 1 with a 90% debt / GDP ratio and we model a Krugman - endorsed stimulus program being implemented in year 1, what does that nation's debt / GDP ratio look like in year 5 or 10? There's no multiplier analysis I know of that leads to a net reduction in the ratio. I've never seen Krugman do any such projection. And once a slow growth nation lets the ratio get over 100%, how does it hope to service that debt without fiscal contraction, except via inflation, financial repression or other value-destroying tactics?

Comments for this post are closed