Russ Roberts has a question

It's about the influence of empirical economics:

I'd like one example, please. One example, from either micro or macro
where people had to give up their prior beliefs about how the world
works because of some regression analysis, ideally usually instrumental
variables as that is the technique most used to clarify causation.

I will cite a few possible examples, although I won't stick with instrumental variables:

1. The interest-elasticity of investment is lower than people once thought.

2. We have a decent sense of the J Curve and why a devaluation or depreciation doesn't improve the trade balance for some while.

3. Dynamic revenue scoring tells us over what time horizon a tax cut is partially (or fully) self-financing.

4. Most resale price maintenance is not for goods and services involving significant ancillary services.

5. More policing can significantly lower the crime rate (that one does use instrumental variables).

6. The term structure of interest rates is whacky.

I see other examples but in general I agree with Russ's point that empirical work fails to settle a great number of important disputes, most disputes in fact.  Many of the examples I would cite turn out to involve an elasticity being lower than we had thought.  And many more involve macroeconomics (rather than micro) than you might expect.

What examples can you think of?


As of the early 1970s there were approximately 100 studies finding that profits were positively correlated with industry concentration. Many people thought this finding was evidence of explicit or tacit collusion and that it justified strong antitrust policy. Harold Demsetz's clever empirical work (JLE 1973) supported a different interpretation: the leading firms in concentrated industries were competitively superior. After a few years profits-concentration studies became, to use Leonard Weiss's word, "unpublishable".

Corporate Finance (Mergers and Acquisitions) example:
Campa and Kedia, Journal of Finance 2002, I think, used IV to show that diversifying mergers do add value, whereas most studies previously had trumpeted the "Diversification Discount" but when CK used IV (and some other more sophisticated techniques) they showed that not only does the "discount" disappear, but it may be a premium.
Primarily, they had to control for self-selection bias in that most mergers happen during industry shake-ups, such that previous work had benchmarked too high for the comparison.

VARs killed naive monetarism: the finding that money shocks only explained maybe 25% of output volatility set off a modest monetarist literature trying to show that was a mistake and that money really did cause most output volatility.

Of course once the monetarists lost they didn't say 'we lost.'. Who's dumb enough to say that? The tearful mea culpa is common in movies because it is so rare in real life.

All the same VARs played a major role in killing the k% growth rule.

And reappearance?


Can we get some citations for #4? I was under the impression this was still disputed.

I was unaware that anyone serious thought that tax cuts were anything other than partly self-financing. The only issue was that some, mostly non-economists, thought they were fully self-financing. And good empirical work should have put an end to that nonsense.

Oh and VARs are methodologically similar to IV.

Another example: Barro's 2 papers showing that money shocks influenced output for 2 years. Made it tough to believe in Lucas-style money surprise models; are people fooled for 2 years?. The lore of the profession is that this spurred RBC since the flexible price types couldn't embrace New Keynesian models.

The Lucas Revolution was cancelled by the empirical hump-shaped response of output to money shocks. Again, near-IV techniques were used.

Again, few recantations were issued: after all these people were busy building a new research agenda!

The visible implications of mind-changing are few.

The series of articles over the past 3 decades that show low (and declining) intergenerational mobility in the U.S.

Joseph Doyle's papers in the AER and JPE from 2007 and 2008 showing foster care causes worse outcomes in crime, employment, and teen pregnancy. IV is probably one of the only ways I would've believed any result regarding foster care's effect on outcomes since studying foster care outcomes has selection problems.

Seatbelt laws kill -- as well as other applications of the, so called, "Peltzman Effect."

Andy makes good points. More generally, sifting through data is a humbling task, as is sifting through empirical results. The easy response is to take sides, but the hard part is to try and jump into the study and understand its strengths and weaknesses.

Causal effects literature seems to be progressing at a lightning fast pace, so even when one feels really sure about a result, it seems like new studies come out which make you do a double take. Take the weak IV literature. IV, to be valid, must be exogenous and relevant. In the 1990s, almost everyone focused on exogeneity, with little to no attention to relevance. Now we're up to our necks in weak IV tests. Or take all the hoo-hah over clustering.

Even when I feel convinced by a good causal effects study, I'm no longer 100% convinced. I'm either mostly convinced or mostly non convinced. Observational data makes it very difficult, for me, to be as sure about findings.

I don't understand his question- any situation where the elasticity of an issue could be positive or negative would fall under this, and there are plenty of those issues. Or is he one of those fellows that thinks we can a priori know, regardless of the question, whether the income or substitution effect dominates?

Tom, how did you decide the data was weak? The data is weaker than it would be if everyone agreed, but that is obvious. My point is that not all data is created equal, and we shouldn't throw our hands up in the air merely because people disagree. We should evaluate people's statistical techniques, the plausibility of the exclusion restriction, and so forth. I'm not saying we shouldn't have a degree of skepticism and of course we should be willing to change our mind in the future, but it's obvious we can evaluate methodology when we evaluate researchers' results.

Personally, my prior on the abortion-crime link was that the theory made some sense, but that they were unlikely to find anything as the variation was too limiting and the effect was likely to be small. Reading the literature carefully changed my mind. Isn't that the point of this discussion and most research?

"The term structure of interest rates is whacky" does not exactly reflect great insight into how stuff works. Otherwise we might have to add the following insights:
- exchange rates are whacky
- capital flows are whacky
- consumers are whacky

The relationship between income and health is a good example. Everybody used to think that causality ran largely from income to health. But now a number of methods, including IV, have demonstrated that, at least in the industrialized world, income exerts no robust effect on health. The effect of health on labor income, on the other hand, is quite strong.

Of course, some ideologues (especially non-economists) have not changed their view that income uniformly improves health. But among reasonable researchers, the consensus has completely flipped.

Angrist's work on the effect of the draft lottery on future earnings.

But it seems a bit ironic that without some credible statistical methodology--regression analysis, anyone?--applied to a much more carefully defined hypothesis, Professor Roberts' question is unanswerable. Empirical economics at its best does seem to influence policy and perceptions, but usually as a literature not as single papers.

Emergency Room Economics - As an Emergency Room RN, I've never treated a cocane adict who actually paid for their first line of coke. Diminished bariers to entry works well in drug dealing.

Similarly I soon learned in Emergency Medicine, when a patient comes complaining of abdominal pain, you can't just give them Morphine thereby masking the pain that is trying to tell you something. In removing pain, you remove information and may as a result miss something that is life threatening. It was difficult watching patient writhing in pain as we saught to discover the source of the pain, but to do otherwise would be mal-practice.

Does this help?


Another point - tax cuts are indeed partially self-financing. If the state gives me $1 and I spend it, they get back 7 cents in revenue or perhaps it goes into someone's salary and the feds get back some 20% or so.

On the other hand, if the state raises my taxes $1 and gives it to someone to serve in the military or pave the highway by my house or whatever, that $1 also generates tax revenue.

So if we use dynamic scoring to try to mitigate the apparent effect of a tax cut, we should also note that a spending increase 'partially pays for itself' as surely as a tax cut 'partially pays for itself.'

The only TEMPORARY advantage is if the Government encourages a foreign investor to give us a dollar to distribute to taxpayers to spend. For a very brief period (think Bush2's early years) this makes the GDP look good, but eventually (think Obama's inherited situation) it comes back to haunt us.

Before Solow's empirical paper on the sources of economics growth most economists believed that the accumulation of factors was the key to growth. Solow showed how important technical change was to American economic growth. This changed the whole profession's view, on arguably the most important question in economics.

Foreign aid has no effect on growth of recipient countries:

Paldam, M., Doucouliagos, H. 2008, "Aid effectiveness on growth", European Journal of Political Economy, vol. 24 nr. 1, s. 1-24.Artikel peer reviewed

Fama and French on pricing stocks.

I agree with you

Comments for this post are closed