1) How does the new version compare?
v1.1.0.2023f is very similar to v1.0.0.2022f over the 1973-2022 period for maximum daily wet-bulb temperature magnitude (TwX) and the number of days exceeding a local, seasonal 90th precentile of daily maximum wet-bulb temperature (TwX90p). Each year there is usually a small change to the source station dataset as a few stations or subperiods of stations are added or lost from processing within the source dataset (NOAA NCEI's ISD). A new wet-bulb temperature algorithm means that there are small changes to the Tw fields only. A bug fix in the HadISD parent dataset (initial post, HadISD update post) has also resulted in a greater removal of flagged errors. Both of these will have affected the PHA homogenisation algorithm resulting in differences to the location and magnitude of detected and adjusted inhomogeneities. A further minor bug fix was applied to use the np.isclose method when comparing equality in floats. This mostly related to assignment of the homogenisation quality score (HQscore) which is used to filter the HadISDH.extremes gridboxes to remove poor quality data. Although resulting in no differences when tested, it is plausible that this may have lead to a few changes in HQscore and therefore the spatial and temporal coverage of gridboxes. The combination of these things has led to an overall increase in spatial coverage. In terms of gridboxes with sufficient presence over the climatology period, the new version v1.1.0.2023f results in 59 (for TwX) and 66 (for TwX90p) additional gridboxes, mostly over the extratropics and tropics. There are 10 (for TwX) and 7 (for TwX90p) lost gridboxes (Figs. 1 and 2 panel a). These have made very little difference to the regional mean timeseries.
The overall story of increasing magnitude of humid heat extremes, as measured by TwX, is concurrent in both versions with no change to long-term trends in v1.1.0.2023f that have an extra year of data other than the trend over the tropics being 0.01 oC decade -1 larger and the negligible decreasing trend in the Southern Hemisphere becoming 0.00 oC decade -1, compared to v1.0.0.2022f. (Figure 1 panels d to g). Across the globe (Figure 1 panel a) the vast majority of gridboxes show trends in the same direction (96.6% of gridboxes - Figure 1 panel b), with 83.6% of these both showing increasing TwX (Figure 1 panel b). Trends in opposite directions occur in 3.4 % of gridboxes.
The overall story of increasing frequency of humid heat extremes is strengthened in all regions by v1.1.0.2023f compared to v1.0.0.2022f. Trends are now 0.05 to 0.2 % days decade -1. Changes to the dataset have made the largest differences in the tropics where differences in the monthly means are noticeable, yet the overall pattern of variability is the same (Figure 2 panels d to g). Across the globe (Figure 2 panel a), 97.7% of gridboxes have trends in the same direction in both versions. Overall, 90.1% of gridboxes agree on positive trends and 7.6% agree on positive RH trends.
Figure 1. Difference in regional timeseries and decadal trends between HadISDH.extremesTwX v1.1.0.2023f and v1.0.0.2022f. a) Ratio of v1.1.0.2023f to v1.0.0.2022f decadal trends (1973-2023 and 2022 respectively) with change in number of gridboxes annotated and identified by red (gained) and pink (lost). b) Scatter plot of gridbox trends with percentage in each quadrant of positive/positive, positive/negative, negative/negative and negative/positive annotated. c) Distribution of gridbox decadal trends for each version with mean and standard deviation annotated. d) Total gridbox coverage by 5 degree latitude band for each year for v1.1.0.2023f. e) Difference in total gridbox coverage by 5 degree latitude band for each year between v1.1.0.2023f and v1.0.0.2022f. f to i) regional mean monthly time series and decadal trends with 90th percentile confidence range annotated.
Figure 2. Difference in regional timeseries and decadal trends between HadISDH.extremesTwX90p v1.1.0.2023f and v1.0.0.2022f. a) Ratio of v1.1.0.2023f to v1.0.0.2022f decadal trends (1973-2023 and 2022 respectively) with change in number of gridboxes annotated and identified by red (gained) and pink (lost). b) Scatter plot of gridbox trends with percentage in each quadrant of positive/positive, positive/negative, negative/negative and negative/positive annotated. c) Distribution of gridbox decadal trends for each version with mean and standard deviation annotated. d) Total gridbox coverage by 5 degree latitude band for each year for v1.0.0.2023f. e) Difference in total gridbox coverage by 5 degree latitude band for each year between v1.0.0.2023f and v1.1.0.2022f. f to i) regional mean monthly time series and decadal trends with 90th percentile confidence range annotated.
2) What's New?
We use HadISD.3.4.0.2023f as the basis for HadISDH.extremes.1.0.0.2023f which includes an additional 12 months of data and any ISD level processing changes during the previous 12 months. There has been a bug fix applied to this dataset (initial post, HadISD update post) after detecting that the buddy check test hadn't been implemented correctly since 2017f versions. The buddy check allows flagged errors to be reinstated if they are supported by neighbouring stations but the bug meant that too many flagged errors were being reinstated. The correction has increased the number of flagged errors removed and therefore changed the observational coverage slightly. Although only last year we implemented a new algorithm (Stull, 2011) to remove the large errors at high temperature/low humidity in the previous Jensen et al., (1990) method, a new method is now available which reduces moist bias errors at high temperature/low humidity (reaching +1.3 °C) and dry bias errors at low temperature/high humidity (approaching -1 °C). The new Non-iterative Evaluation of Wet-bulb Temperature (NEWT) method has been developed by Rob Warren at the Bureau of Meteorology and coded in python. Its errors are far smaller at ± 0.01 °C. Rogers and Warren (2024) introduce the new method and compare it against the Stull (2011) and other methods (https://essopenarchive.org/users/714325/articles/698601-fast-and-accurate-calculation-of-wet-bulb-temperature-for-humid-heat-extremes). Here we use their python code to calculate adiabatic wet-bulb temperatures using polynomial fits from surface pressure, air temperature and specific humidity. This change only affects the wet-bulb temperature fields and differences are negligible in large-scale means and far less than 1 °C for the most part. Differences are largest over hot/dry and cold/humid air conditions. We still use the Stull (2011) method to decide whether to calculate vapour pressure with respect to ice or water because it is faster to implement so there is no change to other variables.
The minor bug fix implemented the use of
numpy.isclose when comparing equality (between a and b) with floats, using the
default tolerance of rtol = 1e-05 and atoll = 1e-08 such that:
absolute(a - b)
<= (atol + rtol *
absolute(b)).
Testing on a sample of 100000 random numbers between
-1 and 2 for equality with 0.0, 0.5 and 1.0 made no difference compared to the
original code.
All other processing steps for
HadISDH.extremes remain identical. The new version of HadISD (3.4.0.2023f) has
pulled through some historical changes to stations which are passed on to
HadISDH.land resulting in 9667 compared to 9555 initial stations. The end
station count is further reduced after completeness checks and homogeneity
assessment. The homogeneity scores differ slightly due to sensitivity to the
addition and loss of stations, historical changes to stations previously
included and the additional 12 months of data.
The combined effect of the new wet-bulb algorithm
and improved removal of random error, in addition to any historical updates
made to the ISD source database, has collectively resulted in an overall
increase in gridbox coverage compared to last year’s version. This is because
these differences result in differences in the detection of inhomogeneity changepoints
and application of homogeneity adjustments by the PHA algorithm, which is very
sensitive. Although overall there were more stations passing through the
various checks in the previous version (1.0.0.2022f), the final number of
extremes stations used were very similar at 4453 for 1.1.0.2023f compared to
4460 for 1.0.0.2022f. However, looking at overall gridbox coverage where each
gridbox month has to have a homogenization quality score (HQscore) less than 7
and at least 70% completeness over the 1973-2023 time period, there has been an
addition of ~60 new gridboxes (depending on variable) and a loss of ~10 other
gridboxes (depending on variable) in this year’s version (1.1.0.2023f). The
additional gridboxes are mainly over the extratropics and tropics.
Rogers, C.D.W. and Warren, R.A. (2024). Fast and Accurate Calculation of Wet-bulb Temperature for Humid-Heat Extremes. ESS Open Archive. January 18, 2024. DOI: 10.22541/essoar.170560423.39769387/v1
Stull, R., 2011: Wet-Bulb Temperature from Relative Humidity and Air Temperature. J. Appl. Meteor. Climatol., 50, 2267–2269, https://doi.org/10.1175/JAMC-D-11-0143.1.3) Summary of changes by level of technicality.
3) Summary of changes
MAJOR CHANGES (X):
BUG FIXES AND MINOR CHANGES (Y):
MINOR BUG FIXES AND HISTORICAL DATA UPDATES (Z):
- Implementation of numpy.isclose to compare equality
of floats within the allocation of homogeneity quality scores (HQscore).
- 9667 compared to 9555 initial selection stations last year.
- Use of HadISD.3.4.0.2023f as the basis which includes retrospective improvements (to correct data, add or remove data sections) to the historical data in NCEI's ISD archive are ongoing. These are not documented.
4) Station Counts
| Version | Initial Stations | Selected Stations | Final Stations TwX | Final Stations TwX90p |
|---|
| v1.0.0.2023f | 9667 | 4821 | 4453 | 4453 |
| v1.1.0.2022f | 9555 | 4775 | 4460 | 4460 |
No comments:
Post a Comment
Note: only a member of this blog may post a comment.