Russ Roberts has a question

by on June 12, 2009 at 7:26 am in Economics | Permalink

It's about the influence of empirical economics:

I'd like one example, please. One example, from either micro or macro
where people had to give up their prior beliefs about how the world
works because of some regression analysis, ideally usually instrumental
variables as that is the technique most used to clarify causation.

I will cite a few possible examples, although I won't stick with instrumental variables:

1. The interest-elasticity of investment is lower than people once thought.

2. We have a decent sense of the J Curve and why a devaluation or depreciation doesn't improve the trade balance for some while.

3. Dynamic revenue scoring tells us over what time horizon a tax cut is partially (or fully) self-financing.

4. Most resale price maintenance is not for goods and services involving significant ancillary services.

5. More policing can significantly lower the crime rate (that one does use instrumental variables).

6. The term structure of interest rates is whacky.

I see other examples but in general I agree with Russ's point that empirical work fails to settle a great number of important disputes, most disputes in fact.  Many of the examples I would cite turn out to involve an elasticity being lower than we had thought.  And many more involve macroeconomics (rather than micro) than you might expect.

What examples can you think of?

Craig M. Newmark June 12, 2009 at 8:03 am

As of the early 1970s there were approximately 100 studies finding that profits were positively correlated with industry concentration. Many people thought this finding was evidence of explicit or tacit collusion and that it justified strong antitrust policy. Harold Demsetz’s clever empirical work (JLE 1973) supported a different interpretation: the leading firms in concentrated industries were competitively superior. After a few years profits-concentration studies became, to use Leonard Weiss’s word, “unpublishable”.

Jesse Blocher June 12, 2009 at 8:27 am

Corporate Finance (Mergers and Acquisitions) example:
Campa and Kedia, Journal of Finance 2002, I think, used IV to show that diversifying mergers do add value, whereas most studies previously had trumpeted the “Diversification Discount” but when CK used IV (and some other more sophisticated techniques) they showed that not only does the “discount” disappear, but it may be a premium.
Primarily, they had to control for self-selection bias in that most mergers happen during industry shake-ups, such that previous work had benchmarked too high for the comparison.

Garett Jones June 12, 2009 at 8:47 am

VARs killed naive monetarism: the finding that money shocks only explained maybe 25% of output volatility set off a modest monetarist literature trying to show that was a mistake and that money really did cause most output volatility.

Of course once the monetarists lost they didn’t say ‘we lost.’. Who’s dumb enough to say that? The tearful mea culpa is common in movies because it is so rare in real life.

All the same VARs played a major role in killing the k% growth rule.

Tyler Cowen June 12, 2009 at 9:13 am

And reappearance?

ao June 12, 2009 at 9:20 am

Tyler,

Can we get some citations for #4? I was under the impression this was still disputed.

Mike June 12, 2009 at 9:25 am

I was unaware that anyone serious thought that tax cuts were anything other than partly self-financing. The only issue was that some, mostly non-economists, thought they were fully self-financing. And good empirical work should have put an end to that nonsense.

Garett Jones June 12, 2009 at 9:40 am

Oh and VARs are methodologically similar to IV.

Another example: Barro’s 2 papers showing that money shocks influenced output for 2 years. Made it tough to believe in Lucas-style money surprise models; are people fooled for 2 years?. The lore of the profession is that this spurred RBC since the flexible price types couldn’t embrace New Keynesian models.

The Lucas Revolution was cancelled by the empirical hump-shaped response of output to money shocks. Again, near-IV techniques were used.

Again, few recantations were issued: after all these people were busy building a new research agenda!

The visible implications of mind-changing are few.

GregN June 12, 2009 at 10:40 am

The series of articles over the past 3 decades that show low (and declining) intergenerational mobility in the U.S.

anony June 12, 2009 at 10:52 am

Joseph Doyle’s papers in the AER and JPE from 2007 and 2008 showing foster care causes worse outcomes in crime, employment, and teen pregnancy. IV is probably one of the only ways I would’ve believed any result regarding foster care’s effect on outcomes since studying foster care outcomes has selection problems.

MW June 12, 2009 at 10:57 am

Seatbelt laws kill — as well as other applications of the, so called, “Peltzman Effect.”

anony June 12, 2009 at 11:53 am

Andy makes good points. More generally, sifting through data is a humbling task, as is sifting through empirical results. The easy response is to take sides, but the hard part is to try and jump into the study and understand its strengths and weaknesses.

Causal effects literature seems to be progressing at a lightning fast pace, so even when one feels really sure about a result, it seems like new studies come out which make you do a double take. Take the weak IV literature. IV, to be valid, must be exogenous and relevant. In the 1990s, almost everyone focused on exogeneity, with little to no attention to relevance. Now we’re up to our necks in weak IV tests. Or take all the hoo-hah over clustering.

Even when I feel convinced by a good causal effects study, I’m no longer 100% convinced. I’m either mostly convinced or mostly non convinced. Observational data makes it very difficult, for me, to be as sure about findings.

confused June 12, 2009 at 12:23 pm

I don’t understand his question- any situation where the elasticity of an issue could be positive or negative would fall under this, and there are plenty of those issues. Or is he one of those fellows that thinks we can a priori know, regardless of the question, whether the income or substitution effect dominates?

Andy June 12, 2009 at 12:40 pm

Tom, how did you decide the data was weak? The data is weaker than it would be if everyone agreed, but that is obvious. My point is that not all data is created equal, and we shouldn’t throw our hands up in the air merely because people disagree. We should evaluate people’s statistical techniques, the plausibility of the exclusion restriction, and so forth. I’m not saying we shouldn’t have a degree of skepticism and of course we should be willing to change our mind in the future, but it’s obvious we can evaluate methodology when we evaluate researchers’ results.

Personally, my prior on the abortion-crime link was that the theory made some sense, but that they were unlikely to find anything as the variation was too limiting and the effect was likely to be small. Reading the literature carefully changed my mind. Isn’t that the point of this discussion and most research?

ft June 12, 2009 at 1:53 pm

“The term structure of interest rates is whacky” does not exactly reflect great insight into how stuff works. Otherwise we might have to add the following insights:
- exchange rates are whacky
- capital flows are whacky
- consumers are whacky

Tom June 12, 2009 at 2:48 pm

The relationship between income and health is a good example. Everybody used to think that causality ran largely from income to health. But now a number of methods, including IV, have demonstrated that, at least in the industrialized world, income exerts no robust effect on health. The effect of health on labor income, on the other hand, is quite strong.

Of course, some ideologues (especially non-economists) have not changed their view that income uniformly improves health. But among reasonable researchers, the consensus has completely flipped.

Mark June 12, 2009 at 3:35 pm

Angrist’s work on the effect of the draft lottery on future earnings.

But it seems a bit ironic that without some credible statistical methodology–regression analysis, anyone?–applied to a much more carefully defined hypothesis, Professor Roberts’ question is unanswerable. Empirical economics at its best does seem to influence policy and perceptions, but usually as a literature not as single papers.

Tom Coss June 12, 2009 at 7:42 pm

Emergency Room Economics – As an Emergency Room RN, I’ve never treated a cocane adict who actually paid for their first line of coke. Diminished bariers to entry works well in drug dealing.

Similarly I soon learned in Emergency Medicine, when a patient comes complaining of abdominal pain, you can’t just give them Morphine thereby masking the pain that is trying to tell you something. In removing pain, you remove information and may as a result miss something that is life threatening. It was difficult watching patient writhing in pain as we saught to discover the source of the pain, but to do otherwise would be mal-practice.

Does this help?

Tom

johnjay60 June 12, 2009 at 8:17 pm

Another point – tax cuts are indeed partially self-financing. If the state gives me $1 and I spend it, they get back 7 cents in revenue or perhaps it goes into someone’s salary and the feds get back some 20% or so.

On the other hand, if the state raises my taxes $1 and gives it to someone to serve in the military or pave the highway by my house or whatever, that $1 also generates tax revenue.

So if we use dynamic scoring to try to mitigate the apparent effect of a tax cut, we should also note that a spending increase ‘partially pays for itself’ as surely as a tax cut ‘partially pays for itself.’

The only TEMPORARY advantage is if the Government encourages a foreign investor to give us a dollar to distribute to taxpayers to spend. For a very brief period (think Bush2′s early years) this makes the GDP look good, but eventually (think Obama’s inherited situation) it comes back to haunt us.

Barry Ickes June 13, 2009 at 9:01 am

Before Solow’s empirical paper on the sources of economics growth most economists believed that the accumulation of factors was the key to growth. Solow showed how important technical change was to American economic growth. This changed the whole profession’s view, on arguably the most important question in economics.

Carsten Valgreen June 13, 2009 at 1:45 pm

Foreign aid has no effect on growth of recipient countries:

Paldam, M., Doucouliagos, H. 2008, “Aid effectiveness on growth”, European Journal of Political Economy, vol. 24 nr. 1, s. 1-24.Artikel peer reviewed

chrismealy June 13, 2009 at 5:20 pm

Fama and French on pricing stocks.

Laly streaming July 13, 2009 at 3:08 pm

I agree with you

Comments on this entry are closed.

Previous post:

Next post: