What is the critical view on the lead-crime correlation?

by on January 9, 2013 at 6:59 am in Law, Science | Permalink

Here is a report from Scott Firestone.  He does admit that much of the evidence carries some weight, but he is less persuaded when it comes to cohort studies:

It turns out there was in fact a prospective study done—but its implications for Drum’s argument are mixed. The study was a cohort study done by researchers at the University of Cincinnati. Between 1979 and 1984, 376 infants were recruited. Their parents consented to have lead levels in their blood tested over time; this was matched with records over subsequent decades of the individuals’ arrest records, and specifically arrest for violent crime. Ultimately, some of these individuals were dropped from the study; by the end, 250 were selected for the results.

The researchers found that for each increase of 5 micrograms of lead per deciliter of blood, there was a higher risk for being arrested for a violent crime, but a further look at the numbers shows a more mixed picture than they let on. In prenatal blood lead, this effect was not significant. If these infants were to have no additional risk over the median exposure level among all prenatal infants, the ratio would be 1.0. They found that for their cohort, the risk ratio was 1.34. However, the sample size was small enough that the confidence interval dipped as low as 0.88 (paradoxically indicating that additional 5 µg/dl during this period of development would actually be protective), and rose as high as 2.03. This is not very convincing data for the hypothesis.

For early childhood exposure, the risk is 1.30, but the sample size was higher, leading to a tighter confidence interval of 1.03-1.64. This range indicates it’s possible that the effect is as little as a 3% increase in violent crime arrests, but this is still statistically significant.

I don’t have any particular view on this matter, but if you wish can you read Drum’s response here.

Addendum: Andrew Gelman comments.

prior_approval January 9, 2013 at 7:29 am

Germany banned lead in gasoline sometime in the early 1990s – though wikipedia disagrees, my memory is leaded fuel was still considered normal in Germany in 1992 – least in terms of my wife’s Polo from that period.

(And why yes, I do seem to be one of the very few people affected by the switch to unleaded in the U.S. – my 1978 BMW R100/7 suffered from valve pitting when they were replaced in the mid-80s at Morton’s – tthough the valve work was certainly worthwhile, replacing the old valves and grinding the seats for better compression.)

However, the German stats don’t seem to follow the lead hypothesis – possibly because of the opening to the East, where lead was not as efficiently banned, or the Yuguslav wars, or the Russian mafia, or ….

Nick January 9, 2013 at 8:13 am

The effect of violence generated by excess lead might not be captured purely at the individual level. It could be that propensity to violence in a relatively small part of a cohort may also have effects on their peers, or produce an “availability cascade” that makes violence more common/acceptable in a given district.

Rahul January 9, 2013 at 10:31 am

Wonder if anyone tried something wacky, like feeding a colony of rats lead fumes and observing if they fight more? Or is that too much of a jump?

TallDave January 9, 2013 at 2:59 pm

http://www.eurojournals.com/ejsr_35_4_12.pdf

I think the effect they’re looking for a is a lot more subtle though — lower dosages, small percentage change.

Mark Thorson January 9, 2013 at 10:41 am

Maybe the Korea and Vietnam wars were ultimately caused by Washington decisionmakers exposure to lead? That would explain why the wars are getting smaller, and maybe they’ll disappear altogether in a few years, as the last lead-addled generation retires and dies.

TallDave January 9, 2013 at 3:02 pm

I’m guessing you don’t live in South Korea.

Willitts January 10, 2013 at 9:41 pm

Or Viet nam.

Ashok Rao January 9, 2013 at 8:14 am

I’m more interested from a systems perspective. While the effect might be “limited” to a still-quite-significant 3%, imagine in a city where many people with these “increased” levels of violence are interacting with each other, and their heightened sense of conflict will result in positive feedback, increasing greatly overall crime.

Andrew' January 9, 2013 at 8:28 am

What about FMJ?

Ray Lopez January 9, 2013 at 9:03 am

Scot FIRESTONE? Is he aware that the inventor of vulcanized rubber, Goodyear, who founded the eponymous competitor to Firestone, died early from complications due to lead poisoning (used in early experiments in vulcanization)? Funny how that works.

dearieme January 9, 2013 at 10:00 am

“the inventor of vulcanized rubber, Goodyear”: or maybe it was Hancock. Hard to say.

Matt January 9, 2013 at 9:04 am

The cited argument seems to be an error of confusing absence of evidence with evidence of absence. The size of the confidence interval is wide enough that we wouldn’t expect to see a statistically significant effect in the longer study. Violent crime has dropped roughly 40% from its peak (i.e., +67% trough to peak), but the confidence interval on the affect of 5ug of lead stretches from -12% to +103%. At the very most, we’d expect to see a barely statistically significant impact from this- The study could never be particularly convincing on its own. It’s that study along with a variety of others that makes the argument compelling.

Dan M January 9, 2013 at 10:12 am

Was going to make this point. In general, Mr. Firestone is doing a terrible job of interpreting confidence intervals by emphasizing the lower bound in both cases.

This seems like weak evidence in any case, but certainly not evidence against Drum’s point. Very cool that someone did a prospective study, however. And the fact that this offers mild evidence in favor of Drum’s argument makes me, if anything, believe the argument more.

ScottF January 9, 2013 at 11:24 am

Hey, I’m the author of the linked blog. I don’t disagree that emphasizing only the lower bounds is generally misleading. I was merely trying to illustrate the idea of statistical significance, and that one non-significant and one barely significant finding out of three, in the only cohort study cited, makes for pretty mixed evidence. Correlations from ecological data can often be very suggestive, but in my mind, they should be enhanced by data on individuals, which then should be replicated. I believe in systematic reviews and meta-analyses when it comes to public health. In some cases, like a second or third order effect from a dangerous environmental contaminant, I don’t think that bar is entirely necessary, but I do think there needs to be something more than what we have.

The lead/crime connection as laid out is definitely a compelling argument. I tried hard to make it clear that I’m really just suggesting how the story can go from “I think we might have something here” to “let’s fix this ASAP.” At any rate, I do appreciate your comment.

mavery January 9, 2013 at 2:49 pm

Making the point that the study you’re citing doesn’t provide anything close to definitive evidence is fine, but statements like “This range indicates it’s possible that the effect is as little as a 3% increase in violent crime arrests, but this is still statistically significant” belies a lack of technical statistical knowledge that undercuts your credibility. I mean, that statement is basically nonsense.

ScottF January 9, 2013 at 3:39 pm

As it turns out, that’s really the one sentence I wish I would have put more thought into how I worded it. I butchered the concept for the sake of laziness. Fair enough.

mw January 9, 2013 at 9:53 am

Oh we’re going to start criticizing the lower bound on confidence intervals now? Better throw away 80% of the econ papers cited on this blog, and tear up all the RCTs.

Scott Cunningham January 9, 2013 at 9:54 am

Jessica Reyes study on this published in the BE journal on economics and policy (I always forget the name of this journal even though I have study published there too) is IMO the best study on this relationship, and her interpretation is definitely more circumspect and cautious. There’s a lot of measurement issue challenges with getting lead over time by region, not to mention getting measures of individual exposure. There is also the challenges associated with exogenous variation in lead. She uses the clean air act for identification, but she notes that there are weak instrument problems that she runs into. My reading of her study is that more work needs to be done on this, but only for ones with credible causal inference. The world doesn’t need another correlational study of lead and Y (pick your outcome). But public health is a mixed bag when it comes to correlation/causation and research design so don’t hold your breath.

Tom T. January 10, 2013 at 5:57 am

Reyes notes, “This means that two children who are otherwise identical but whose lead levels differ by 15 μg/dL (approximately the decline in lead levels between 1976 and 1990) would exhibit an average IQ difference of 7.5 points.”

Has such an increase of IQ over that timespan been identified?

Varun Rao January 9, 2013 at 10:47 am

The magnitude of the lead – crime relationship should be very testable over the next few years. India, China (and a host of other countries) banned lead in petrol quite late (2000 – although there was some phase out). Given that automobile use in both of those countries increased steadily in the cities, the impact on urban crime ought to be easily visible between 2010 and 2020.

My guess is that, at least in developing countries, it will be hard to attribute any significant violent crime increase to lead over the next few years (this is partly because of all the other things that are improving at the same time).

My sense is that lead (and other environmental toxins) has an impact on crime rates, but its far too small to be a major contributor to the 80s crime spike in the US. You probably have to reach for a complex mix of social and environmental factors to explain its magnitude.

In general, what I find more shocking is how little crime there is in the US (and oddly enough in the developing world). My fear is that we’re actually in an unusual equilibrium right now, and crime rates (of all kinds) will rise over the next few decades.

Scott Cunningham January 9, 2013 at 11:02 am

So long as countries are changing lead regulations and nothing else, that is. Also, worth noting, if they are changing lead regulations for reasons correlated with potential crime outcomes, then any observed correlation is biased and may be bias in an unknown direction too depending on the selection. Such country-wide policy changes are almost always endogenous though.

Varun Rao January 9, 2013 at 3:51 pm

That’s not quite right is it? First, I believe most of the banning happened thanks to an international drive to get rid of lead (and to a lesser extent, thanks to the drop in price of catalytic converters thanks to research driven by the US). So the link is only loosely coupled with outcomes in the particular countries. Second, the rise in car volumes is exogenous. Finally, we’re looking for the spike in crime rates, not the drop off after 2000, since we, in effect, have a prospective experiment not a retrospective one requiring an IV variable.

Scott Cunningham January 9, 2013 at 7:00 pm

“That’s not quite right is it? First, I believe most of the banning happened thanks to an international drive to get rid of lead (and to a lesser extent, thanks to the drop in price of catalytic converters thanks to research driven by the US). So the link is only loosely coupled with outcomes in the particular countries.”

If they are assigning the treatment for reasons that are associated with the potential outcomes — that is, the actual levels of crime, the counterfactual levels of crime, etc — then the treatment is not ignorable, and it won’t matter what association you find. That association between lead and crime will be biased. Causal interpretations in this context are always about maintaining the ignorable treatment assumptions, at least conditionally.

“Second, the rise in car volumes is exogenous.”

Exogenous to what? To crime? Was it because of income growth possibly? Changes in trade restrictions? There’s a very long list of backdoor paths that plausibly connect car purchases in the aggregate with a variety of social indicators like crime. The passage of a law doesn’t have to be directly because of the potential outcomes iow. It is sufficient that its correlated through a back door path for the associations to be invalid (from the point of view of interpreting the association as a causal effect).

“Finally, we’re looking for the spike in crime rates, not the drop off after 2000, since we, in effect, have a prospective experiment not a retrospective one requiring an IV variable.”

It depends I suppose on whether randomization is essential to an experiment. For us to interpret associations between lead and crime *causally*, then we require that the treatment assignment be independent of crime levels in actual or counterfactuals.

Unless they are flipping coins when choosing their lead exposure levels, one always should be skeptical. The assumptions necessary under observational scenarios are non-trivial, and cannot be directly tested since potential outcomes by definition don’t exist. So maybe you’re right that the passage of lead exposure regulations at an international level can be conditionally independent, but its definitely a lot for someone to swallow.

Varun Rao January 10, 2013 at 9:01 am

Scott,

I actually think we are in some agreement here.

Yes the banning of lead in petrol in China, India etc. is somewhat coupled with crime related outcomes (in the US). Obviously there is some interaction between crime and the dramatic increase in cars and therefore the lead concentration. But clearly the linkages are much weaker than the equivalent linkages in the linked study and in similar US studies.

So while we can’t make surefire claims about causality (though I can point you to papers that use much worse variables as instruments and claim causality), we can use the data from India and China to refine claims about the magnitude of a possible causal link. That was my original statement. There is other medical research that more tightly suggests a causal link.

TallDave January 9, 2013 at 2:55 pm

Yep, Scott nails it.

FredR January 9, 2013 at 11:14 am

I just wish somebody in this debate would give Saul Bellow’s ‘The Dean’s December’ a shout-out.

mavery January 9, 2013 at 12:29 pm

That’s a horrible way to interpret a confidence interval.

The right way to do it is to say that with [whatever level they used, because the excerpt didn't say; let's hope 95?] percent confidence, folks with early childhood exposure to lead have a risk ratio greater than 1.

Reading the excerpt above indicates a pretty poor understanding of what the values estimated actually mean, which makes me less confident that the analysis was performed correctly in general. I bet this guy used SPSS.

brianS January 10, 2013 at 7:50 pm

mavery wins with the SPSS crack.

Floccina January 9, 2013 at 1:22 pm

If I could I would make a big bet that lead is not a significant cause of crime.

Rahul January 9, 2013 at 2:14 pm

I think, if you did, you’d win.

TallDave January 9, 2013 at 2:54 pm

You might lose even if you’re right, if lead ingestion and crime share common causes.

the commentariette January 9, 2013 at 2:22 pm

Correlation (to the extent to which it may exist) is not causality…

Children with lower lead levels may differ from children with higher lead levels in a number of ways — many of which would correlate with parents who are more knowledgeable about environmental risks, more proactive about child safety issues, and more willing/able financially to minimize them. And parental income, education and involvement are all very strongly negatively correlated with future criminality.

TallDave January 9, 2013 at 2:52 pm

It seems at least as likely that people who are exposed to lead also happen to be living in lower-trust communities.

This reminds me of the old ’60s canard that poverty caused crime. So we gave the poor lots of welfare money and crime didn’t get any better. Now it’s pretty well-established that the reverse is true: crime causes poverty. But we’re still struggling to find a way, absent massive coercion, to get societies to become more trustworthy, which is why there are still so many poor communities/countries.

Scott Cunningham January 9, 2013 at 7:04 pm

Crime didnt get better relative to what? Relative to what it would have been absent the income transfers? Or relative to what it was before the income transfers? Finding that crime goes up after an income transfer isn’t evidence of no effect if income transfers are made because of predicted increases in crime. It’s entirely possible that without said transfers, crime rates would’ve been even higher in other words.

Popeye January 9, 2013 at 3:01 pm

This reminds me of the old ’60s canard that poverty caused crime. So we gave the poor lots of welfare money and crime didn’t get any better. Now it’s pretty well-established that the reverse is true: crime causes poverty

So poverty rates have plummeted as crime has dramatically dropped?

TallDave January 9, 2013 at 3:15 pm

Yes, if crime levels are falling in an area economic activity is usually increasing.

Crime creates costs. If you’ve lived in high-crime areas, it’s hard to be unaware of some of them, but there are some less obvious ones too — I’ve talked to Nigerian immigrants who gave up on businesses in their country because employees had such a high propensity to steal. But it’s hard to change cultures.

Tusca January 11, 2013 at 5:02 am

No need to conjure up a hypothetical “if you’ve lived in high crime areas…”
We know that crime rates have dropped over the last 20 years. Has poverty rates dropped in the same way? In fact, we also know where crime rates have fallen more quickly, less quickly, and barely at all. Does that correlate with changes in poverty? If not, then your assertion is false.

Willitts January 9, 2013 at 3:37 pm

Who knew in all those trials I prosecuted, all I needed to do to prove or disprove willfulness or mens rea was to conduct a blood test.

This sounds like phrenology. Maybe we should cut open their skulls and see if they have dimples. Oh, that was just a movie.

So Much For Subtlety January 9, 2013 at 8:04 pm

Hey, you should count your blessings. At least their lawyers didn’t get to hear about this. If they did, you would be knee deep in acquittals. It would be the 21st century equivalent of the Twinkie defense.

Ricardo January 9, 2013 at 9:13 pm

And they say lawyers don’t understand statistics.

Willitts January 10, 2013 at 6:40 pm

A lawyer’s understanding of statistics is directly proportional to the contribution of the statistics to winning his case.

That said, judges have been smart enough to recognize the ecological fallacy.

Steve Sailer January 9, 2013 at 6:51 pm

Back in 2007, I looked into the research Drum relies upon today and found it interesting, more resilient to reality checks than Steven Levitt’s celebrated abortion-cut-crime theory. Nonetheless, I pointed out a number of anomalies that needed to be resolved, such as why in densely populated Japan, which had lots of lead spewing cars, was there never a rise in crime?

http://isteve.blogspot.com/2007/07/lead-poisoning-and-great-1960s-freakout.html

Steve Sailer January 9, 2013 at 6:58 pm

The highest homicide rates today are found in old industrial cities with large black populations. In particular, I would look at New Orleans (which industrialized rapidly during WWII), a perennial leader in murder rate, both before and after the hurricane, as a place where lead poisoning would plausibly be causing social problems: the notorious Lower Ninth Ward is below sea level so there’s no place for heavy metal toxins to wash away down to. Everywhere is up from the Lower Ninth, so lead particulates pile up.

Willitts January 10, 2013 at 7:41 pm

It is a sad fact that the proportion of blacks and population density are the chief predictors of a wide range of social ills. Of course, the moment you cite that fact you become a target for race baiters.

Somewhere around 96% of the victims of black crime are black. One would think that shedding light on the problem would benefit blacks, the majority of whom commit no crimes.

I was bewildered by the Levitt and Donohue paper which was big news in my home town. I’m more inclined to believe that the incarceration rate had more to do with the drop in crime than anything else. I found the L-D paper hard to argue with, and I’m glad that smarter people than me explored its shortcomings.

At first I brushed off the lead-crime connection, but I later recalled the stereotype of the Mad Hatter who went crazy using lead based products in his daily trade. So perhaps the idea is not so farfetched. I’m not sure I would enjoy seeing that used as a defense in court.

Your link to studies of lead exposure in Japan are an interesting counterexample. Perhaps looking at crime rates for workers exposed directly to lead would help.

ezra abrams January 9, 2013 at 8:50 pm

Brain Saver
Quote
Between 1979 and 1984, 376 infants were recruited.
Unquote
at that point, any sane person should have stopped paying attention, as even the most basic knowledge of social sci should tell you that the sample size is to small, by at least a 10X, to be meaningful
Further analysis was a total waste of time
There is one point – if you were thinking of doing a real study, with maybe 50,000 kids, this would be a good pilot, to figure out how to do assays correctly, how to keep track of people etc
other then that total waste of time

Ricardo January 10, 2013 at 12:38 am

I don’t agree “any sane person should have stopped paying attention” and your last sentence hedges this strident over-statement. The problem with having a small sample size is that it gives you weak power, e.g. a small probability of finding a statistically significant effect. Here, the study found a statistically significant effect for lead levels at six years of age. You might want to say they got lucky but they got lucky, so did all the other people publishing in this field who found similar results.

I do agree with your last sentence. The reality of social science research is that sample sizes are determined largely by budgets. A bunch of studies are providing suggestive evidence that lead exposure may increase criminality — that’s reason enough to spend money on a larger study compared to other more dubious interventions that don’t appear to work. The controversy will only be resolved by studies with large sample sizes and a good identification strategy.

rosacea symptoms January 13, 2013 at 12:48 am

It is perfect time to make a few plans for the future and it is time
to be happy. I have learn this post and if I may I desire to recommend you few attention-grabbing issues or tips.
Maybe you can write next articles relating to this article.
I want to read more things approximately it!

Comments on this entry are closed.

Previous post:

Next post: