The OPERA Experiment and the Value of High-Profile Scientific Blunders

One of the most exciting physics stories of 2011 was the September announcement that an experiment in Italy had observed neutrinos traveling faster than the speed of light. If true, this would directly contradict one of the cornerstone predictions of the theory of relativity. A comprehensive revision of modern physics would be needed.

The excitement meeting the initial announcement was quickly matched by skepticism from many parts of the physics community. And a few months later, in March 2012, the results were traced to a bad fiber-optic connection in the timing system. A new and improved measurement a year later confirmed that neutrinos do, in fact, obey the conventional laws of physics.

The faster-than-light neutrino saga evolved very rapidly, with the whole issue completely resolved within nine months. It was also extensively documented at every step, making it an excellent demonstration of the trajectory of a high-profile scientific failure.

Timeline

The Oscillation Project with Emulsion-tRacking Apparatus (OPERA) collaboration involves around 180 physicists from 28 institutions, mostly in Europe. Its goal is to study how the fundamental particles called neutrinos transform as they travel from an accelerator at CERN outside Geneva to a detector at Gran Sasso in Italy.

In 2011, while analyzing data from its first three years of operation, the OPERA team spotted an anomaly. The neutrino beam produced at CERN is pulsed on and off in a regular way, and neutrinos are detected in real time at Gran Sasso, enabling a measurement of how much time the particles take to travel the 730 kilometers between accelerator and detector. The OPERA data showed particles being detected some 60 nanoseconds sooner than would be expected if they were moving at the speed of light. This would directly contradict Einstein’s theory of relativity, which forbids matter or information being transmitted at speeds greater than that of light.

Detailed analysis of the data within the OPERA collaboration provided a very good measurement of this anomaly: (60.7 ± 6.9 ± 7.4) ns (the two quoted uncertainties are statistical and systematic, respectively), corresponding to a speed of 1.0000248 ± 0.0000028 ± 0.000030 times the speed of light. Despite extensive checks of the data, they were unable to find an explanation of this anomaly; everything they could readily test was consistent with a faster-than-light speed for the neutrinos in their particle beam.

Given the potential importance of the result, and their inability to find an error, the OPERA collaboration put together a preprint of a paper describing their experiment, and scheduled a seminar at CERN for September 23, 2011. Rumors circulated on social media before the seminar created a huge sensation surrounding the announcement, which was widely reported in international news media.

The initial public claims of the OPERA collaboration were relatively restrained. The technical article posted to the Arxiv preprint server at the time of the announcement referred only to an “anomaly” and explicitly disavowed “any theoretical or phenomenological interpretation of the results.” At the CERN seminar, collaboration coordinator Antonio Ereditato emphasized the preliminary nature of the result Collaboration coordinator Antonio Ereditato emphasized the preliminary nature of the result. “We made a measurement and we believe our measurement is sound. Now it is up to the community to scrutinize it,” he said. “We are not in a hurry. We are saying, tell us what we did wrong, redo the measurement if you can.” He also noted that “a result is never a discovery until other people confirm it.”

Initial reactions from the physics community were generally positive, with physicists who watched the initial presentation praising it as “very carefully done” and “a very serious job.” But the general sense of excitement was tempered with scientific skepticism. For example, Susan Cartwright, a particle physicist at the University of Sheffield was quoted saying, “Neutrino experimental results are not historically all that reliable, so the words ‘Don’t hold your breath’ do spring to mind,” and Lisa Randall, a prominent Harvard physicist, told the Washington Post, “If you had to bet, you’d bet on some experimental error.” The most colorful expression of doubt came in the form of an actual bet, when Jim Al-Khalili, a theoretical physicist at the University of Surrey, stated that if the OPERA result held up he would eat his boxer shorts on live TV.

The OPERA result also generated a furious burst of activity in the technical literature, with more than 40 articles citing the OPERA preprint posted to the Arxiv in the first week after the announcement, and around 180 by the end of 2011 (according to the INSPIRE database tracking citations of high-energy physics papers). These featured a mix of theoretical papers offering mechanisms by which neutrinos might violate the relativistic speed limit (or at least appear to do so), theoretical papers demonstrating the impossibility of faster-than-light neutrinos, and attempts to find errors in the OPERA collaboration’s analysis of their anomalous data.

The various attempts to analyze the experiment from a distance were mostly unimpressive — as John Learned of the University of Hawaii presciently noted, “If a screw-up, it is probably in the details not accessible to outsiders” — but two theoretical papers quickly emerged as a solid foundation for opposition to the result. The first, by Gian Giudice of CERN and two colleagues, was posted to the Arxiv on September 26 (just three days after the initial announcement). The paper argued that the speed measured by OPERA would imply other violations of relativity at a level ruled out by the previous experiment. The second, by Andrew Cohen and Sheldon Glashow just a few days later, described a mechanism by which the OPERA neutrinos should have rapidly lost energy and slowed to speeds below that of light. These were widely taken by theorists as proof that OPERA’s initial results must be an error, and stronger public complaints began to appear. For example, University of Arizona physicist Lawrence Krauss worried that the whole issue “could wind up embarrassing the profession.”

On the experimental side, the late months of 2011 were a mixed bag for the OPERA collaboration, with a follow-up experiment using shorter pulses of neutrinos (one of the potential systematic problems frequently raised by critics) giving results consistent with their initial measurement. However, the ICARUS collaboration, which operates another detector in the same neutrino beam, analyzed its own data to show that there was no sign of the energy loss required by the Cohen and Glashow mechanism, a significant blow to the experimental finding.

In early 2012, the OPERA finding unraveled completely. While the results were being debated in the literature, the experimenters were going over every inch of their apparatus looking for possible sources of error. In late February, they announced a devastating discovery: One of the fiber-optic cables carrying the time signal from a GPS receiver on the surface to the underground laboratory was not correctly connected. The misconnected cable combined with their timing electronics to delay their clock by an amount that was almost identical to the anomaly they originally reported. In mid-March, the ICARUS experiment reported on its own measurement of the velocity of neutrinos from CERN to Gran Sasso, finding a value completely consistent with the speed of light.

Finally, in June the OPERA collaboration completed their own follow-up measurement with the fiber-optic connection fixed. The team found a speed consistent with both the light-speed limit and results from three other experiments using the same neutrino beam. From start to finish, the whole story took a bit under nine months, at the end of which physics was returned to the status quo.

Commentary

The unraveling of the faster-than-light neutrino anomaly was unquestionably an embarrassment for the OPERA collaboration, with the initial burst of excitement and international media attention vanishing in a wave of recrimination. Collaboration spokesperson Antonio Ereditato and physics coordinator Dario Auterio resigned their positions at the end of March 2012, both blaming media hype for their downfall.

Plenty of other commentators blamed OPERA for the unfolding of events. From very early in the story, Lawrence Krauss publicly called the incident “an embarrassment,” and worried that it was “very unfortunate — for CERN and for science.” And Nima Arkani-Hamed of Princeton declared the whole thing a waste of time, as “No really decent theoretical physicist took this seriously from the very start.”

While it is easy for disappointed physicists to point fingers, closer inspection suggests that the OPERA collaboration did little that was wrong. The team’s result appeared statistically sound and had some support from an earlier measurement by the MINOS experiment at Fermilab, which had seen a similar timing anomaly in 2007 but without enough statistical power to claim it was a genuine measurement. And the experimental flaw, when it was uncovered after several months of extensive searching, turned out to be much subtler than the “loose cable” that was often reported in the media. A small gap in a fiber-optic line would only be expected to attenuate the timing signal, not delay it by 60 nanoseconds. The delay came from an unfortunate combination of the attenuation and the particular electronics used; in the words of Belgian physicist Pierre Vilain, “only a very deep knowledge of the electronic chip which treats the signal can explain why it leads to such a long and fatal delay.”

In fact, the superluminal neutrino saga stands as an excellent demonstration of the process of science. Throughout the process, the OPERA team was careful to emphasize the preliminary nature of their results, and they were very forthright about the measurements they had made and the procedures they followed. They were accused of “science by press conference,” but the media attention their anomaly generated was largely a function of a changed media landscape, with reporters and bloggers closely monitoring the Arxiv and CERN calendar for rumors. The common suggestion that they could have quietly sought outside assistance displayed what Dennis Overbye drily called “a quaint faith in the ability of the Twitterverse to keep secrets.”

The superluminal neutrino story also demonstrates the deep engagement with the scientific community that characterizes genuine science and distinguishes it from pseudoscience. In the wake of the initial announcement, there was a real dialogue carried out within the literature. The original report by the OPERA team has been cited in more than 300 papers on the Arxiv and in physics journals, both by theorists proposing models for superluminal neutrinos, and others attacking the original analysis. The Cohen and Glashow paper has itself been cited more than 150 times, and not only by papers arguing against superluminal neutrinos. Many of those citations are in articles proposing new physics, attempting to work around or explain away the mechanism identified by Cohen and Glashow.

Finally, the ultimate resolution of the issue demonstrates the self-correction that is the hallmark of genuine science. The discovery of the faulty connection was publicly announced by OPERA, and the final nail in the coffin was a corrected measurement by the very same collaboration that kicked the whole business off nine months earlier.

In the end, perhaps the best summation of the whole issue was given by Sergio Bertolucci of CERN in announcing the combined results in June 2012:

Although this result isn’t as exciting as some would have liked, it is what we all expected deep down. The story captured the public imagination, and has given people the opportunity to see the scientific method in action—an unexpected result was put up for scrutiny, thoroughly investigated and resolved in part thanks to collaboration between normally competing experiments. That’s how science moves forward.

Chad Orzel is a professor at Union College, a blogger at Forbes, and the author of four popular science books about physics, most recently “Breakfast With Einstein” (BenBella Books). This article is excerpted from “Pseudoscience: The Conspiracy Against Science.”