The New York Times published an 11-minute documentary in June titled “‘It Was Really a Love Story.’ How an N.R.A. Ally Became a Gun Safety Advocate,” which tells a heartwarming story of how friendship transcended political differences and convinced a right-wing partisan to come to terms with the truth about firearms.
The film stars a couple of improbable friends: Dr. Mark L. Rosenberg, who for many years oversaw research on gun violence at the Centers for Disease Control and Prevention (CDC) as the director of its National Center for Injury Prevention and Control, and “NRA Pointman” Rep. Jay Dickey (R–Ark.), who was the author of an amendment inserted into a 1996 spending bill that prohibited the CDC from using federal funds to advocate for gun control.
The story is also framed by the findings of a famous (or infamous) 1993 CDC-funded study, which was “the first piece that we funded by external scientists,” Rosenberg recounts. It allegedly showed that owning guns made Americans overwhelmingly less safe. According to the film, the National Rifle Association (NRA) lobbied for the Dickey Amendment because of the 1993 study’s damning results. The organization “didn’t think it would be good for business,” Rosenberg says, “and they went to Congress, and they said, ‘You have got to stop this research because it’s going to result in all Americans losing their right to have a gun in their homes.'”
Dickey and Rosenberg started out as “mortal enemies,” but after making small talk about their kids during a chance conversation they developed “an incredible friendship,” as Rosenberg recounts. Years later, they were habitually ending their conversations by telling each other “I love you,” and “we really meant it,” Rosenberg says. Through the power of this human connection, Dickey ends up seeing reason and changing his mind. He comes to believe that the amendment bearing his name was a mistake.
It’s a story of redemption through friendship that’s well-tuned to our own hyperpolarized times. The lesson is that if blind partisans aren’t swayed by empirical evidence, human connection might just do the trick. “Underneath what people think are such opposing forces are some very important shared values,” Rosenberg says.
Although the moral of the documentary is undoubtedly true, every other detail is wrong. The takeaway from the story of Dickey, Rosenberg, and the 1993 gun study at the center of the piece is that the congressman was correct to begin with. The CDC shouldn’t be studying gun violence.
Titled “Gun Ownership as a Risk Factor for Homicide in the Home” and published in The New England Journal of Medicine, the 1993 study looked at 388 people who had been killed in their homes and matched them to 388 neighbors of similar age, sex, and race. One hundred and seventy-four of the victims lived in houses where at least one gun was present versus only 139 of the matched controls.
With scary music and breathless claims, the video tells viewers that if you had a gun in your house, you were 200 percent more likely to be killed with a gun in your home and 400 percent more likely to kill yourself.
These are both exaggerations and misstatements of the study results. It didn’t address suicide risk at all, nor gun homicides. It found households in which a resident had been murdered at home by any means had a 25 percent greater frequency of having a gun, not 200 percent. But this doesn’t mean owning a gun increases your risk of being killed by 25 percent.
This is a classic statistical error known as the “base rate fallacy” and is particularly important when studying rare events, like people murdered in their homes. Suppose 10 people are murdered in their homes, and five of those homes had guns. A matched set of 10 people who were not murdered in their homes found only four homes had guns. So there are 25 percent more guns in the homes of murder victims than matched nonmurder victims (Five vs. four).
But what if you put those 20 people in the context of another million, none of whom were murdered in their homes, half of whom had guns in their homes and half of whom didn’t. The rate for gun owners to be murdered at home becomes five out of 500,009, while the rate for non-gun owners becomes five out of 500,011. So now we find that the risk is 0.0004 percent higher.
In other words, being murdered in your home means you have a 25 percent higher chance of having a gun, but having a gun means you have only a 0.0004 percent greater chance of being murdered in your home. Those are not the same thing.
The finding that owning a gun made study subjects less safe was also a conclusion selected from much stronger statistical results that didn’t fit the authors’ political views and, thus, weren’t mentioned in the study. Yes, 25 percent more victims’ homes had guns than control homes, but 38 percent more victims had controlled security access to their property. Why not lobby against gates as a public health matter? Twenty times as many victims had gotten in trouble at work because of drinking, so why worry about guns when drinking at work is two orders of magnitude more dangerous? Renting and living alone were far more dangerous than having a gun. Victims were less likely than controls to own a rifle or a shotgun, so why not a government program to trade in handguns for long guns?
Another issue is the study design. Comparing victims to nonvictims can make sense in medicine. When a new disease appears, for example, researchers compare victims to outwardly similar nonvictims to get clues about the cause. In the early days of AIDS, the CDC identified a “4H club” of Haitians, homosexuals, hemophiliacs, and heroin addicts. Most of the sick fell under one of those four categories, whereas non-“4H” members were seldom infected. While the first H turned out to be a false clue, the last three suggested correctly a blood-to-blood or blood-to-mucous membrane transmission.
With a new disease, we don’t know much about transmission. But with gun ownership, we can identify obvious mechanisms, such as the gun owner shooting himself, or an attacker stealing the gun and shooting the gun owner. But if the gun in the house wasn’t the murder weapon, that homicide shouldn’t be included in the study.
Therefore, for guns, it’s silly to lump all deaths together. The majority of the deaths in the households included in the study were not caused by guns, and when a gun was used, the researchers did not determine who owned it. We also don’t care equally about all deaths. If a woman kills a violent attacker threatening her family, that’s a bad thing only to a tunnel-vision public health researcher adding up total deaths.
A more technical criticism is that the study uses flawed statistical methodology. Most journals require research to demonstrate the results are unlikely to result from random chance. Overall, 40.6 percent of subjects and controls had guns in the house. If gun ownership has no effect on homicide rates, there’s a 4 percent chance that 174 or more of the subjects would own guns. That’s less than the arbitrary 5 percent threshold many journals use, so the results are deemed statistically significant. The researchers did a more complex statistical analysis to arrive at the same 4 percent significance.
The issue with both my simple calculation and the researchers’ more complex one is they are only accurate if there are no errors in the data, the precise effect of all controls is known, and all relevant controls are included. We know there are significant errors in this kind of data, we have little idea about the effects of controls, and we know there are many relevant factors that were not measured.
A good rule of thumb many practicing statisticians use is to ignore any odds ratio below three—that is, you’d want to see three times as many victims as controls owning guns before you took the result seriously. A 25 percent difference can easily be attributed to random chance.
Another technical criticism is the 1993 study ignores the vast literature by psychologists, sociologists, criminologists, and statisticians on crime and violence. It cites only public health studies and general references. This is a general problem with most public health literature. It exists in a closed world that does not learn from other fields, nor do researchers in other fields build on it.
Despite the study’s problems, which have been written about widely (see here, here, here, and here), Rosenberg attributes all of the criticism to gun manufacturers concerned about potential loss of sales. Though the Dickey amendment prohibited the CDC’s Injury Center from spending money on gun control promotion and advocacy, Rosenberg blames it for shutting off all research into gun violence.
Rosenberg sums up the Dickey amendment as follows: “If you do research in this area, we will harass you.” An example of the harassment Rosenberg gives is “the threat of congressional inquiries” that can “wreak havoc with your research.”
Why does Rosenberg think that taxpayer-funded research shouldn’t be subject to congressional inquiry? Rather than stating that he was willing to answer sensible and relevant questions, Rosenberg wanted to be shielded from congressional Republicans like Dickey, who he deemed ignorant and evil.
“Worse than not understanding,” Rosenberg says, “he doesn’t care.”
The Dickey amendment didn’t prevent the CDC from gathering and analyzing data on gun injuries and deaths. Many gun control researchers rely on CDC data. But gun control is part of a much larger issue of crime, violence, rights, and policy effects; it’s not something that can be studied usefully with only infectious disease models, methods, and data.
Much of the screen time of the video is devoted to entirely irrelevant scary scenes of shooting and shells with threatening graphics. It ends with text on screen noting that the $25 million restored to the CDC for gun control research is only a fraction of a percent of its budget. (Gun homicides are also a fraction of a percent of total U.S. deaths.)
Dickey, who died of Parkinson’s disease in 2017, may have come to regret pushing for the amendment that bears his name, but he remained concerned about the 1993 study, which would have complicated The New York Times‘ narrative and which the film left out. It uses a clip from CNN in which Dickey says he had a “change of heart” because of “the weight of all of the incidents that have occurred…. The kids and the innocent people who are being killed deserve our attention.”
What’s not included is a moment from that same clip in which Dickey states his ongoing concerns about the 1993 study and how it was carried out. “We wanted research done for gun violence, and that’s what the money was paid for,” he says. “But we found out that as we went along, that not only was the research being done just to support gun control, but we weren’t even given access to what the collected data was. So it was clear that we needed to do something and to stop what was being done.”
Dickey was right. Guns are not a pathogen and violence is not a disease to be controlled. What is needed is not propagandistic films or sloppy research. It’s cross-disciplinary research on what drives human beings to violence, whether with a gun or without one.
Written by Aaron Brown and John Osterhoudt; edited by Osterhoudt; graphics by Adani Samat; additional graphics by Regan Taylor and Isaac Reese.
Photos: Associated Press; Cottonbro/Pexels; Roll Call/Newscom
The post Guns Aren’t a Public Health Issue appeared first on Reason.com.
