Defensive Gun Use and the Difficult Statistics of Rare Events

In the mid-1990s, Kleck and Gertz (1995) estimated that in a typical year about 1.3% of US adults used a gun for self-defense against another person. Kleck and Gertz’s estimate, which came from a survey of nearly 5000 people, implied that there were millions of defensive gun uses every year.

Following Kleck and Gertz’s 1995 paper, the CDC added a question about defensive gun use to their Behavioral Risk Factor Surveillance System (BRFSS). In 1996, 1997, and 1998 the CDC asked:

“During the last 12 months, have you confronted another person with a fire arm, even if you did not fire it, to protect yourself, your property, or someone else?”

But here is the surprise. The CDC buried the question and the results. Only recently was the data discovered and made public by Kleck in a new paper.*(see addendum) So what were the results? You will perhaps now not be too surprised that the CDC’s survey supports Kleck and Gertz’s original finding, about 1% of survey respondents reported a defensive gun use, implying millions of such uses over a year.

The case isn’t closed on defensive gun use, however, because of a statistical conundrum.

The CDC asked 12,870 individuals about defensive gun use over the three samples.That’s a relatively large sample but note that this means that just 117 people reported a defensive gun use, i.e. ~1%. In comparison, 12,656 people (98.33%) reported no use, 11 people (0.09%) said they didn’t know and 86 people (0.67%) refused to answer. People answering surveys can be mistaken and some lie and the reasons go both ways. Some people might be unwilling to answer because a defensive gun use might have been illegal (Would these people refuse to answer?). On the other hand, mischievous responders might report a defensive gun use just because that makes them sound cool.

The deep problem, however, is not miscodings per se but that miscodings of rare events are likely to be asymmetric. Since defensive gun use is relatively uncommon under any reasonable scenario there are many more opportunities to miscode in a way that inflates defensive gun use than there are ways to miscode in a way that deflates defensive gun use.

Imagine, for example, that the true rate of defensive gun use is not 1% but .1%. At the same time, imagine that 1% of all people are liars. Thus, in a survey of 10,000 people, there will be 100 liars. On average, 99.9 (~100) of the liars will say that they used a gun defensively when they did not and .1 of the liars will say that they did not use a gun defensively when they did. Of the 9900 people who report truthfully, approximately 10 will report a defensive gun use and 9890 will report no defensive gun use. Adding it up, the survey will find a defensive gun use rate of approximately (100+10)/10000=1.1%, i.e. more than ten times higher than the actual rate of .1%! Those numbers are, of course, approximately what the CDC survey found which doesn’t prove that Kleck’s interpretation is wrong only that very different interpretations are also plausible.

The bottom line is that it’s good to know that the original Kleck and Gertz survey replicated–approximately 1% of adult Americans did report a defensive gun use in the 1990s–but the real issue is the interpretation of the survey and for that a replication doesn’t help.

Addendum: The paper has since been taken down perhaps because in addition to the issue of interpretation that I raised the survey may not have been national. Robert VerBruggen has further details.

Comments

Comments for this post are closed