How NASA didn’t discover the hole in the ozone layer

Date

29 September 2023

Topics

anomalies

I am writing a new textbook on anomaly detection. It probably won’t be finished for at least a year, but here is an excerpt.

There is a widespread myth that NASA did not discover the hole in the ozone layer above the Antarctic because they had been throwing away anomalous data that would have revealed it. This is not true, but the real story is also instructive (Pukelsheim 1990; Christie 2001, 2004).

NASA had been collecting satellite data on Antarctic ozone levels using a Total Ozone Mapping Spectrometer (TOMS) since 1979, while British scientists had collected ozone data using ground sensors at the Halley Research Station, on the edge of the Brunt Ice Shelf in Antarctica, since 1957. Figure 1 shows average daily values from the NASA measurements in blue, and from the British observations in orange. There is a clear downward trend in the British data, especially from the late 1970s, which is confirmed with the NASA data. So why wasn’t the “ozone hole” discovered until 1985?

Figure 1: Observations of Antarctic ozone levels since 1957, measured in Dobson units (DU). Observations are mean daily values from October each year. Ground observations are from Halley Research Station, while satellite observations were obtained using a Total Ozone Mapping Spectrometer (TOMS). The satellite data were obtained from Leslie R Lait (NASA), while the Halley ground observations were obtained from Jonathan Shanklin (British Antarctic Survey). The horizontal line shows the threshold of 180 DU, used by NASA to determine when the ozone level was unusually low.

The British scientists had noticed the low ozone values as early as 1981, but it took a few years for the scientists to be convinced that the low values were real and not due to instrument problems, and then there were the usual publication delays. Eventually, the results were published in Farman, Gardiner, and Shanklin (1985).

Meanwhile, NASA was flagging observations as anomalous when they were below 180 DU (shown as a horizontal line in Figure 1). As is clear from the figure, this is much lower than any of the plotted points before the early 1980s. However, the 180 threshold was used for the daily measurements, which are much more variable than the monthly averages that are plotted. Occasionally daily observations did fall below 180, and so it was a reasonable threshold for the purpose of identifying instrument problems.

In fact, NASA had checked the unusually low TOMS values obtained before 1985 by comparing them against other available data. But the other data available to them showed ozone values of about 300 DU, so it was assumed that the satellite sensor was malfunctioning. The British Halley data were not available to them, and only after the publication of Farman, Gardiner, and Shanklin (1985) did the NASA scientists realise that the TOMS results were accurate.

In 1986, NASA scientists were able to confirm the British finding, also demonstrating that the ozone hole was widespread across the Antarctic (Stolarski et al. 1986).

This example reveals some lessons about anomaly detection:

The NASA threshold of 180 was based on daily data, and was designed to identify instrument problems, not genuine systematic changes in ozone levels. The implicit assumption was that ozone levels varied seasonally, but that otherwise the distribution of observations was stable. All anomaly detection involves some implicit assumptions like this, and it is well to be aware of them.
Sometimes what we think are anomalies are not really anomalies, but the result of incorrect assumptions.
Often smoothing or averaging data will help to reveal issues that are not so obvious from the original data. This reduces the variation in the data, and allows more systematic variation to be uncovered.
Always plot the data. In this case, a graph such as Figure 1 would have revealed the problem in the late 1970s, but it seems no-one was producing plots like this.

References

Christie, M. 2001. The Ozone Layer: A Philosophy of Science Perspective. Cambridge, UK: Cambridge University Press.

———. 2004. “Data Collection and the Ozone Hole: Too Much of a Good Thing?” History of Meteorology 1: 99–105.

Farman, J C, B G Gardiner, and J D Shanklin. 1985. “Large Losses of Total Ozone in Antarctica Reveal Seasonal ClO_x/NO_x Interaction.” Nature 315 (6016): 207–10.

Pukelsheim, F. 1990. “Robustness of Statistical Gossip and the Antarctic Ozone Hole.” The IMS Bulletin 19 (4): 540–45.

Stolarski, R S, A J Krueger, M R Schoeberl, R D McPeters, P A Newman, and J C Alpert. 1986. “Nimbus 7 Satellite Measurements of the Springtime Antarctic Ozone Decrease.” Nature 322 (6082): 808–11.