Marine Data Decimal Frequency Oddities

Meteorological data transmitted operationally (e.g., across the GTS) to global archives (e.g., ICOADS) has had various formats over time. Some of these restricted the number precision of the value reported to one or no decimal places. There may have been difference practices used to reduce any measurements that originally had higher precision to a whole number before reporting the observation. A value may have been rounded or truncated. Such widespread changes over time can have consequences for long term records of climate data, even in the global average. This issue has been noted previously in the ICOADS dew point temperature data (http://icoads.noaa.gov/corrections.html) and I thought it might be a contributing factor to the pre-1982 moist bias in marine relative humidity.

My recent work to produce a marine surface humidity record from ship and buoy data has included exploring the decimal frequency of the air temperature and dew point temperature data from the ICOADS 3.0.0 and 3.0.1 database. Figures 1 and 2 show the time series for each decimal place from 1973 to 2017 for all (1) and ship only (2) observations passing a set of quality control tests.

Figure 1 Changes in decimal frequency over time from the ICOADS 3.0.0 and 3.0.1 database for all observations passing a set of quality control tests.

Figure 2 Changes in decimal frequency over time from the ICOADS 3.0.0 and 3.0.1 database for all SHIP observations passing a set of quality control tests.

These figures highlight several interesting things. For air temperature (plot a), whole numbers are far more common than they should be throughout and this decreases percentage wise considerably in the late 1990s when moored buoys are included (Figure 1) and again from around 2008 for both ship only and all observations (ship and moored buoys) (Figures 1 and 2). The late 1990s decrease in Figure 1 (all obs) could be due to the increase in moored buoys around that time. There is more consistency in .0s frequency over time in the ship only time series (Figure 2). The frequency of .5s in the database is also disproportionately high. This could be due to a rounding practice or the recording precision of the instrument used. This declines very sharply in 1974 and then more gently over time. There is a curious blip in 1997. This is present in both figures but far smaller when moored buoys are not included (Figure 2). I have looked at this in more detail. It is caused by a drop in the frequency of .2s and .7s and an increase in the number of .3s and .8s. This appears to be coming from deck 893 (moored buoys) and possibly 892. Deck 926 may contribute to the increase in .8s. Figures 3 (all) and 4 (ship only) show the break down of decimal frequency by deck for 1997. This could be an error originating from when this deck was first integrated into the ICOADS archive.

Figure 3 Decimal frequency for all air temperature observations in 1997 for each deck present (coloured lines) and all observations (black). Only observations passing a set of quality control tests are included. For each deck number the percentage of observations with a whole number flag detected during quality control is shown with the percentage that this contributes to the total number of observations in parentheses. The quality control whole number flag is set when an observation is a whole number and part of a track containing a high proportion of whole numbers.

Figure 4 Decimal frequency for all SHIP air temperature observations in 1997 for each deck present (coloured lines) and all observations (black). Only observations passing a set of quality control tests are included. For each deck number the percentage of observations with a whole number flag detected during quality control is shown with the percentage that this contributes to the total number of observations in parentheses. The quality control whole number flag is set when an observation is a whole number and part of a track containing a high proportion of whole numbers.

For dew point temperature (plot b), the pattern of decimal frequencies is very different. The frequency of .5s is very consistent with all other non-zero decimals. The frequency of .0s shows two distinct peaks and declines further from around 2008 (similar to air temperature). As for air temperature the ship only time series (Figure 2) are more consistent over time than when moored buoys are included (Figure 1) suggesting that the increase in number of moored buoys over time brings the frequency of whole numbers down. The first peak occurs in the 1970s and appears consistent with the change in transmission format to allow a decimal place for dew point temperature from January 1982. However, the steepest decrease in .0s occurs in 1980 which is a little early. There is a second peak in the mid- to late 1990s.

David Berry (NOCS) has done some digging around and found a references on the ICOADS website (icoads.noaa.gov/e-doc/other/dupelim_1980 and icoads.noaa.gov/real-time.html - I'll double check these when the site comes back online again) relevant to the 1997 issue in air temperature. These suggest that the issue may be caused by conversion to and from Kelvin in early BUFR data with differing conversion factors and storage precision of one decimal place. For deck 893 specifically, there is a significant reduction in .2s and .7s and increase in .3s and .8s from April to December. This is documented to also affect the SST directly and also the dew point temperature and dew point depression indirectly when these are reconstructed. Storage precision in BUFR was extended to two decimal places at around 17th February 1999.