How Industry Weaponizes Science and Sows Doubt to Serve Their Agenda
We may think of science as our foremost producer of knowledge, but for the past decade, science has also been studied as an important source of ignorance. The historian of science Robert Proctor has coined the term agnotology to refer to the study of ignorance, and much of the ignorance studied in this new area is produced by science. Whether an active or passive construct, intended or unintended, this ignorance is, in Proctor’s words, “made, maintained, and manipulated” by science.
In the conversation that follows, excerpted from a longer discussion published in the volume “Science and the Production of Ignorance,” Proctor and historian and philosopher of science Peter Galison describe some of the most important strategies used by industry over the years to produce this ignorance. Some of the well-documented tactics used by the tobacco industry, for example, included funding decoy research to distract from critical questions; organizing “friendly research” for publication in popular magazines; establishing scientific front organizations; and forever calling for more research and more evidence, setting standards for proof so high that nothing could ever satisfy them.
“It’s really quite brilliant,” says Proctor. “If you don’t like the science that’s out there, create some of your own. And then claim ‘we need more research.’ And then label your opposition as a bunch of close-minded fanatics.” The Proctor-Galison dialogue provides a historical context for today’s explorations of agnotology and makes abundantly clear why the rest of us need to be as concerned as they are with such ignorance and the new area of its investigation.
—The Editors
Peter Galison: How did you get interested in industrial science, and in particular tobacco, and the issues surrounding it?
Robert Proctor: I’d been exploring since my undergrad days the social and political causes of cancer; I’d worked to get a ban on smoking in my sophomore dorm at Indiana University (in 1973), after I realized that about a third of all cancers are caused by cigarettes. Other cancers were caused by what we eat or breathe, the radiation we’re exposed to, etc. — topics generally ignored in Richard Nixon’s War on Cancer, which focused on cures rather than causes and ignored cigarettes. I realized pretty early on that there were powerful industrial interests trying to shape what we know and don’t know about these topics, and incorporated these into our Bio 106 topics at Harvard. I discovered there were about 1,500 trade associations whose business was just to protect a particular substance against claims it was causing harm. So there was the Asbestos Information Association defending asbestos, the Global Climate Coalition denying global warming, the Methyl Butyl Ether Task Force defending (guess what) — I’m talking about the late 1980s now — plus of course the Tobacco Institute defending tobacco. Most of these trade associations were denying there was “sufficient evidence” to convict a particular compound of causing harm. And this was a relatively unstudied social aspect of science, although Harvard did have a radical medical student newsletter cautioning that “Harvard may be hazardous to your health!” — in 1973, thanks to payoffs from the Tobacco Institute. What I found remarkable about these trade associations was how they were using (or creating) science to create ignorance — partly by funding what I like to call “distraction science” or “red herring research.” Science was effectively being supported as part of an effort to disguise harms.
Science was effectively being supported as part of an effort to disguise harms.
Recall that agnotology is both the study of ignorance and the studied social production of ignorance — just as English is both a language and the disciplined study of that language. What’s brilliant about the industrial production of ignorance is that when it was developed in the tobacco context in the early 1950s (following earlier efforts to defend lead and sugar, inter alia), they could actually claim to be acting in the name of science when they called for “more research.” The call for more research was an effective legal-savvy form of denial, expressed in a manner that effectively captured the allegiance of universities (and the high rhetorical ground of open-mindedness) while retaining plausible deniability in court (“we never said cigarettes are safe!”). Science could also be supported in such a way as to distract from the evidence that was accumulating that cigarettes were in fact causing cancer.
PG: One of the most striking features of governmental secrecy is its extent. For fiscal year 2014, for example, the government itself estimated that the cost of secrecy runs, for the government as a whole, about $15 billion. That is bigger than the fiscal year 2016 budget for the Department of Labor ($13.2 billion) or the Environmental Protection Agency ($8.6 billion). The effort involves tens of millions of pages of classified material that must be sorted, classified, and guarded. Recognizing that the scope of this effort is important — scale matters, and tracking the structure and effects of this system is part of what I’ve tried to address not only in my written work but also in my film (with Robb Moss), “Secrecy” (2008). Of course, even these staggering numbers don’t include the full range of outsourced materials, much less the secrecy that has its origin and maintenance in the private sphere. Which leads me to ask, How big is the production of distraction or decoy science in the industrial realm? Can you give a sense of the amounts of money involved, or the number of people, or any measure of how widespread it is?
RP: Well, in tobacco alone, between 1954 and the 1990s, $450 million were funneled through just the Council for Tobacco Research (CTR), only one of several arms of the cigarette conspiracy. Most of that went to distraction or decoy research, mainly into basic virology, genetics, biochemistry, immunology, etc. — any kind of agent that was not nicotinic. Over 7,000 papers were published with support from the CTR, the principal arm of the cigarette conspiracy. CTR-funded scholars went on to win at least ten Nobel Prizes — so we’re talking about some pretty solid research, albeit “harmless” from the industry’s point of view. The problem is not really visible from looking at any one published paper; from a micro level, the conspiracy, the intent, is invisible. The pernicious intent is really only visible when you look at the research funded in the aggregate—and fortunately we have the industry’s own secret documents, which talk about the CTR as a “front,” a “shield,” and “a successful defensive operation.” Again, the intent was to distract from cigarettes as a cause of harm, especially by focusing on proximate rather than ultimate causes. CTR research can be considered a kind of data chaff, jamming the scientific airwaves with noise. The CTR also funded a good deal of basic research, along with research looking at harms from things like carpet fumes, radon, occupational exposures, genetic predispositions — all of which are respectable topics, but which in aggregate create an impression that something other than tobacco is causing harm. And that’s why you can’t see the bias in any one publication. You have to look at macro bias rather than micro bias, because it turns out that if you actually control for industry funding — when looking at, say, the hazards of secondhand smoke — you get a very different result than if you just take the aggregate of all research that’s published.
Over 7,000 papers were published with support from the Council for Tobacco Research, the principal arm of the cigarette conspiracy.
PG: I remember one example you once told me about, where it was discovered that miners in coal country got lung cancer or black lung disease or other diseases of the lung in large numbers if they smoked and were miners. The response was not to improve mining conditions, it was instead to ban smokers from some of the mining tasks. I might not have that story quite right, but could you say what happened, and how it fits into this idea of distraction or chaff science?
RP: Cancer hazards are often multiplicative, synergistic — which makes sense if the disease is caused by the accumulation of mutations. So if you are exposed to asbestos, you increase your risk of lung cancer by, say, fivefold, and if you smoke you increase it by fivefold. But if you smoke and are exposed to asbestos, you increase your risk by fiftyfold; there’s a nonlinear, disproportionate augmentation. What’s interesting about that is how different industries strategize to blame something other than themselves. Asbestos will blame tobacco, and tobacco will blame asbestos, but what’s important is that each of those groups can create and then rely on particular types of science as weapons to be deployed in court. And a lot of these strategies do arise out of litigation — one of the three pillars of the cigarette conspiracy (the others being legislation and public opinion). Expertise is important in court, and cigarette makers have been able to create what they call a “stable” of expert witnesses to defend themselves and their products. And the role of science in this? As Imperial Tobacco once said, “Research must go on and on.” Support for science is used as a way to prevent certain kinds of questions from ever becoming “closed.” In the tobacco case this was called the “open controversy”: The idea was that by continuing to support science, they could use this very fact (“we need more research”) to claim that harms had not been definitively proven.
It is important to appreciate the scale and scope of this campaign. CTR research alone resulted in over 7,000 peer-reviewed publications. The collaboration with the American Medical Association produced hundreds of others — and I mentioned the 10 Nobel Prizes. At least 26 Nobel laureates have taken money from the cigarette conspiracy. R. J. Reynolds funded almost all of Stanley Prusiner’s research into prions, for which he was awarded the Nobel Prize in 1997. Again good research, but part of the cigarette makers’ interest in funding “small virus” research to distract from cigarette causation. Cigarette makers helped found the field of behavioral genetics; they funded leading scholars, like Hans Selye and Ancel Keys, publishing on the role of stress or cholesterol in causing heart disease. When people today think of heart disease as caused by stress or cholesterol, that is in no small part because cigarette makers encouraged this kind of research. Keywords here are corruption and monopoly (they tried to monopolize certain kinds of expertise), but also alternative causation and open controversy.
PG: You mean that the opposition in a sense would have to take a position of illiberalism? They would have to say, “We are against further research”?
RP: That’s right.
PG: And that’s a bad rhetorical spot to be in; by making doubt into a product, the industry has hidden perfectly well-established science. Demonstrated health risks became uncertain topics for research; inquiry became the opposite of understanding. Here is a particularly damaging form of knowledge manipulation and an important result of focusing on how ignorance is produced — agnotology at work.
RP: That’s right. Tobacco and other industries were able to use their support for science as a central pillar of their conspiracy, which was that we don’t really know whether cigarettes cause cancer. They were able to use the openness of the question and their support for research as a defense of their legal stance and propaganda position. So it’s a brilliant example of using the liberal rhetorics of science to defend what’s essentially a criminal enterprise: a conspiracy to hide the hazards of smoking. It’s really quite brilliant because it captured the authority of science and allegiance of scientists, made the tobacco industry seem open-minded, and made public health advocates seem like close-minded fanatics.
PG: And scientists who wanted to take the industry’s money would then be able to say to themselves and their peers that they were involved in open-ended research, that as long as they were doing research they weren’t guaranteed to produce what the tobacco industry hoped they would. They were simply adding to a general, broad and open-ended debate.
RP: That’s right. I think of it as something like an army of idiot miners, paid to look for gold where the funders know there’s no gold to be found. And then you (the paymaster) say, “Oh look, there’s no gold.” Cigarette makers loved funding research topics that posed no threat to their business. So most of the research funded by the industry had nothing to do with tobacco, and certainly nothing to do with that product causing harm (with a few exceptions, which I call “leakage” — since the conspiracy was not perfect).
I think of it as something like an army of idiot miners, paid to look for gold where the funders know there’s no gold to be found. And then you (the paymaster) say, “Oh look, there’s no gold.”
PG: It’s significant too that these techniques of ignorance production have multiplied over many different domains. Naomi Oreskes and Erik Conway’s (2011) “Merchants of Doubt” shows how these same techniques, and indeed same public relations firms, even many of the same prominent scientists, went on to argue that climate change was “doubtful” and in need of more research. Once again, an “openness” toward scientific research could, in the event, serve short-term industrial gain, block regulatory and political action, and scramble public debate, even when the science was clear.
RP: It’s really quite brilliant: If you don’t like the science that’s out there, create some of your own. And then claim “we need more research.” And then label your opposition as a bunch of close-minded fanatics. Later this became more subtle, with the industry claiming that knowledge of cigarettes causing cancer was “common knowledge,” and had been so for hundreds of years. This was part of the industry’s “assumption of risk” defense: you were fully informed of the nature of the hazard, information on the hazard was “available,” you have only yourself to blame for whatever harms you may have suffered. The common knowledge defense is now deployed in every tobacco trial. So first they falsify science, now they falsify history.
As for science, though, the deceit was not what we normally think about in terms of research misconduct — falsification or fabrication of data, for example. The bias was further upstream than we imagine, and the fact is that once funded, the industry generally did not influence the science funded.
PG: You mean if they accepted money for research from the Tobacco Institute or CTR?
RP: Right. The CTR would say, “Publish whatever you want.” But the bias was built in to the selection of problems in the first place. And that’s a general principle that historians and philosophers need to pay more attention to: Problem selection and funding shape what kind of science gets done. One of the more general points about agnotology is that there are infinitely many things you might know, and that whatever in fact becomes known is only a tiny sliver of what might be known — infinitesimal really. What this means is that when you’re shining a light on something, almost everything else remains in the dark. And sometimes that darkness is deliberately kept dark; the darkness itself may be created, maintained, exaggerated, inflated, and reinforced, sometimes even by the very power of the light itself (think flashy fish lures or Donald Trump). I think there’s an assumption in a lot of thinking about science that there is some finite quantum of knowledge humans might acquire. Maybe we’ll never get it all, but at least we’re moving forward, vanquishing the darkness. But darkness has many friends, and often deep pockets as well. And darkness can easily grow as fast as (or faster than) the light. So it’s much more a constructive or organic metaphor that we need.
PG: Of course, the tobacco industry employed other strategies of ignorance production as well. One technique that was often used was to mention things out of proportion. For example, 90 percent of lung cancer is caused by smoking. But it’s true that people sometimes will know somebody who got lung cancer but who didn’t smoke. If you obscure the fact that 90 percent of lung cancer comes from smoking by mentioning twelve other possible ways to get lung cancer — or even having in that 10 percent some that are completely of unknown origin — then you’ve actually created ignorance.
“The darkness itself may be created, maintained, exaggerated, inflated, and reinforced, sometimes even by the very power of the light itself.”
RP: Yes. I like to ask my World History of Science students, “Are you convinced that smoking is the leading cause of lung cancer?” only about two-thirds will say yes. If you ask, “Do you think smoking causes cancer?” a much higher proportion will answer in the affirmative. And if you ask, “Have you heard that smoking causes cancer?” an even higher proportion will say yes. So much depends on how the question is asked, and this has often been exploited by the tobacco industry in court because it’ll say, “Look, in the 1950s, 90 percent of Americans had heard that smoking causes lung cancer.” But they’d also heard that aliens were being held in Area 51. There is a big difference between awareness and belief.
PG: Certainly if you are aware of a debate, you could say, “Are you aware that there’s an argument for increasing taxation? Are you aware there’s an argument for decreasing taxation?” People would with high probability recognize both of those.
RP: Right. The industry is able to use this proximity of ideas in deception. Merely by pointing to someone who smoked and lived a long time — George Burns lived to be 100 — the industry creates the impression that smoking doesn’t always causes cancer, which of course is true. But you also have to remember that if you ask ordinary people, ordinary smokers, “Do you think smoke will kill you?” the most common answer will be, “Well, if God wants me to die, I will die.” We tend to forget that most people are still monotheists and think their fate lies in God’s hands. That has agnotological implications.
PG: The everyday epistemology is really strange to me, given that we don’t actually have experience with absolute causality in our daily lives. Wearing a seat belt while driving does not make you invincible against any possible car crash, it just makes you more likely to survive or to survive with lesser injuries than had you not worn it. But when we’re talking about a public, disputed, economically powerful area and whether something causes something else, as a society we often go back to saying, “If there is an exception, then the thing isn’t true.” That’s strange, right? Because we know someone who lived to be 100 and smoked, we think smoking really shouldn’t cause cancer, even though most of the causal things we have any experience with are probabilistic. And in science, our most basic account of the world, quantum mechanics, tells us that when one elementary particle hits another, the outcome can be predicted only probabilistically. So whether we are deciding about whether to wear a bicycle helmet or how deep-inelastic electron scattering will proceed, we use, formally or informally, a notion of probabilistic causation. It’s really stunning that the industry can get away with saying that causes inevitably either lead to a particular outcome . . . or else they are not causes at all. Strange reasoning, and stranger still that it worked (somewhat).
RP: Yeah, it’s called sophistry! But it seems to work. One of the brilliant things cigarette makers were able to do was to say that unless every person who smokes gets cancer, and unless every cancer is in someone who smoked, then smoking doesn’t cause cancer! By that same logic you’d say that drunk driving can’t cause traffic accidents, because some people who have accidents weren’t drinking and not everyone who drinks has an accident. So cigarette makers created an impossibly high bar for inferring causality. And for decades they were successful using this argument — by essentially redefining causality. They actually funded a lot of scholars, people like Alvan Feinstein at Yale, an early CTR special projects operative and one of the founders of evidence-based medicine, to create such a high bar for evidentiary proof that nothing could jump over it! So there’s a long litany of actual harms he was able to deny because they didn’t meet his criteria for causality. The industry was able to manipulate these philosophical ideas to its benefit because of how much money it’s got.
Peter Galison is an American historian and philosopher of science. He is the Joseph Pellegrino University Professor in history of science and physics at Harvard University.
Robert Proctor is an American historian of science and Professor of the History of Science at Stanford University, where he is also Professor by courtesy of Pulmonary Medicine.
This discussion is excerpted from the book “Science and the Production of Ignorance.”
