Another 2022 update from HadISDH.marine.1.5.0.2022f

NOTE: Late this year I discovered a bug in the gridding code for the marine data. This has now been fixed. There are resulting changes to the gridbox values but over the large-scale mean and long-term trends these are negligible in that they do not change the overall direction of the trends. This new version will not be formally released as we are so close to the end-of-year update but this blog post gives the details of the changes in comparison to version 1.4.1.2022f.

1) How does the new version compare?

v1.5.0.2022f is virtually identical to v1.4.1.2022f in both coverage, large scale mean timeseries, and decadal trends. This bug fix does not change the overall story that we conclude from HadISDH.marine. This is that the amount of water vapour near the surface over oceans is increasing - the specific humidity (q) is increasing. Simultaneously, the relative humidity (RH) is decreasing, meaning that the air is apparently becoming less saturated. 

The re-gridding has resulted in slight changes to the long-term mean spatial coverage but no overall decrease or increase in gridboxes. There are 33 gridboxes that have been lost (pink boxes on panel a) of the two figures below) and 33 gridboxes that have been gained elsewhere (yellow boxes on panel a) of the two figures below). Panel b) in the two figures below shows the change in number of gridboxes per latitude band over time. This shows that the bug fix has resulted in changes to spatial coverage over time with more losses in the Tropics and Southern Hemispehere and more gains in the Northern Hemisphere. As explained below, the small changes to the gridding assignments will both directly affect gridbox values through their presence or absence but also indirectly affect the gridbox values through their influence on the climatology values and coverage, which are used to quality control the observations and calculate anomalies.

The the long-term trends in q for v1.5.0.2022f compared to v1.4.1.2022f are identical for the Globe, Tropics and Southern Hemisphere and 0.01 g kg ^-1 decade ^-1 larger for the Northern Hemisphere (Figs. 1f to i). Across the globe (Figure 1a) almost all gridboxes show trends in the same direction in both versions (98%  of gridboxes - Figure 1b), with 91.5 % showing moistening trends and 6.5% showing drying trends.

The the long-term trends in RH for v1.5.0.2022f compared to v1.4.1.2022f are identical for the Tropics. Over the Globe and Northern Hemisphere the decreasing trends in RH are slightly (0.01-0.02 %rh decade ^-1) weaker (less negative) for v1.5.0.2022f, but 0.01 %rh decade ^-1 stronger (more negative) for the Southern Hemisphere (Figs. 2f to i). Across the globe (Figure 2a) most gridboxes show trends in the same direction in both versions (93%  of gridboxes - Figure 2b), with 61 % showing decreasing saturation trends and 32 % showing increasing saturation trends.

As noted for v1.4.1.2022f, the decrease in RH over oceans remains an uncertain conclusion as it does not reconcile with models or theory. Conceivably, there may be regions where relatively warmer and dry air may be advected from the land which could locally lower RH, or regions where wind speed changes or differing trends in SST vs MAT might have the effect of lowering RH. Ultimately, spatial coverage of HadISDH.marine remains limited, with very little representation over the Southern Hemisphere and so the trends in q and RH may not be truly representative of the global trend. However, the increasing specific humidity is consistent with both climate models and theoretical expectation.

Figure 1. Difference in regional timeseries and decadal trends between HadISDH.marineq v1.5.0.2022f and v1.4.1.2022f. a) Ratio of v1.5.0.2022f to v1.4.1.2022f decadal trends (1973-2022) with change in number of gridboxes (with at least 70% temporal completeness) annotated and identified by yellow (gained) and pink (lost). b) Scatter plot of gridbox trends with percentage in each quadrant of positive/positive, positive/negative, negative/negative and negative/positive annotated. c) Distribution of gridbox decadal trends for each version with mean and standard deviation annotated. d) Total gridbox coverage by 5 degree latitude band for each year for v1.5.0.2022f. e) Difference in total gridbox coverage by 5 degree latitude band for each year between v1.5.0.2022f and v1.4.1.2022f. f to i) regional mean monthly time series and decadal trends with 90th percentile confidence range annotated.

Figure 2. Difference in regional timeseries and decadal trends between HadISDH.marineRH v1.5.0.2022f and v1.4.1.2022f. a) Ratio of v1.5.0.2022f to v1.4.1.2022f decadal trends (1973-2022) with change in number of gridboxes (with at least 70% temporal completeness) annotated and identified by yellow (gained) and pink (lost). b) Scatter plot of gridbox trends with percentage in each quadrant of positive/positive, positive/negative, negative/negative and negative/positive annotated. c) Distribution of gridbox decadal trends for each version with mean and standard deviation annotated. d) Total gridbox coverage by 5 degree latitude band for each year for v1.5.0.2022f. e) Difference in total gridbox coverage by 5 degree latitude band for each year between v1.5.0.2022f and v1.4.1.2022f. f to i) regional mean monthly time series and decadal trends with 90th percentile confidence range annotated.

2) What's New?

A bug within the gridding code for HadISDH.marine has been detected and fixed.

The assigning of observations to gridboxes had an indexing error and an error related to the use of int() instead of np.floor(). This was putting Southern and Western Hemisphere observations with a latitude/longitude with a precision of .0, and all Northern and Eastern Hemisphere observations in the 1x1 degree gridbox to the south/west. 

The final gridded HadISDH product is at 5x5 degree resolution so the impact of this is dampened to a large degree by averaging over the 5x5 degree box. Differences between the older v.1.4.1.2022f and the now fixed v1.5.0.2022f are negligible in terms of large scale means and long-term trends. 

However, at smaller scales and over shorter time periods, differences are detectable. Changes to the observations contributing to each 5x5 gridbox changes will directly change both the spatial coverage and gridbox mean values. This will also affect the climatological gridbox values and spatial coverage of the climatology fields which may then change the results of the quality control tests based on climatology. Some observations will be gained - where they now either pass the climatology outlier test and if there is now a climatological value for their respective gridbox where there wasn't previously. The opposite is also true, leading to further changes to the observations contributing the final 5x5 degree gridbox mean values.

Although this change is a bug fix I have classed this as a minor change (Y) in the version numbering because of the nature of the bug fix. Although the overall impact is negligible in terms of what the dataset tells us about our changing climate, it is clear that the fixed methodology has lead to small changes over time and space, including spatial coverage.

3) Summary of changes by level of technicality.

MAJOR CHANGES (X): 

• none

MINOR CHANGES (Y): 

• A bug fix to change int() to np.floor() and correct the indexing within the gridding code.

BUG FIXES AND HISTORICAL DATA UPDATES (Z): 

• None