Long-chain correlation: lead paint and crime

ChrisHibbert

A friend has been asking my views on the likelihood that there's anything to a correlation between changing levels of lead in paint (and automotive exhaust) and the levels of crime. He quoted from a Reason Blog:

So Nevin dove in further, digging up detailed data on lead emissions and crime rates to see if the similarity of the curves was as good as it seemed. It turned out to be even better: In a 2000 paper (PDF) he concluded that if you add a lag time of 23 years, lead emissions from automobiles explain 90 percent of the variation in violent crime in America. Toddlers who ingested high levels of lead in the '40s and '50s really were more likely to become violent criminals in the '60s, '70s, and '80s.

I responded with the following:

Sounds like a stretch to me. I'd want to hear that they didn't test more than 5 other hypothesis before coming to that conclusion, or the p value was far better than .05. I kind of doubt that either is the case.

He's apparently continued to pursue the question, and just forwarded these remarks from Steven Pinker that I thought were very illuminating, and probably deserve a place in this community's toolkit for skeptics. Pinker's main point is that the association between Lead and crime is a long tenuous chain of suppositions, and several of the intermediate points should be far easier to measure. Finding correlations at this distance is not very informative.

http://stevenpinker.com/files/pinker/files/pinker_comments_on_lead_removal_and_declining_crime.pdf

Does the phrase "long-chain correlation" stick in your head and make it easier to dismiss this kind of argument?

So Nevin dove in further, digging up detailed data on lead emissions and crime rates to see if the similarity of the curves was as good as it seemed. It turned out to be even better: In a 2000 paper (PDF) he concluded that if you add a lag time of 23 years, lead emissions from automobiles explain 90 percent of the variation in violent crime in America. Toddlers who ingested high levels of lead in the '40s and '50s really were more likely to become violent criminals in the '60s, '70s, and '80s.

I responded with the following:

Sounds like a stretch to me. I'd want to hear that they didn't test more than 5 other hypothesis before coming to that conclusion, or the p value was far better than .05. I kind of doubt that either is the case.

http://stevenpinker.com/files/pinker/files/pinker_comments_on_lead_removal_and_declining_crime.pdf

Does the phrase "long-chain correlation" stick in your head and make it easier to dismiss this kind of argument?

One form of more convincing evidence based on observational longitudinal data is using g-computation to adjust for the so called "time varying confounders" of lead exposure.

A classic paper on this from way back in 1986 is this: http://www.biostat.harvard.edu/robins/new-approach.pdf

The paper is 120 pages, but the short version is, in graphical terms all you do is pretend that you are interested in lead exposure interventions (via do(.)) at every time slice, and simply identify this causal effect from the observational data you have. The trick is you can't adjust for confounders as usual, because of this issue:

C -> A1 -> L -> A2 -> Y

Say A1, A2 are exposures to lead at two time slices, C is baseline confounders, L is an intermediate response, and Y is a final response. The issue is the usual adjustment here:

$p[y|do[a1,a2]]=\int_l\int_c p[y | a1,a2,c,l]p[c,l]dcdl$

is wrong. That's because C, L and Y are all confounded by things we aren't observing, and moreover if you condition on L, you open a path A1 -> L <-> Y via these unobserved confounders which you do not want open. Here L is the "time-varying confounder": for the purposes of A2 we want to adjust for it, but for the purposes of A1 we do not. This implies the above formula is actually wrong and will bias your estimate of the early lead exposure A1 on Y.

What we want to do instead is this:

$p[y|do[a1,a2]]=\int_l \int_c p[y | a1,a2,c,l] p[l | a1,c] p[c] dc dl$

The issue here is you still might not have all the confounders at every time slice. But this kind of evidence is still far better than nothing at all (e.g. reporting correlations across 23 years).

Prediction: if you did this analysis, you would find no statistically significant effect on any scale.