Using PHA at MOHC to homogenise HadISDH

What and Why:

Previously the homogenisation was run in house at NCDC using their PHA software. We need to bring this in house to MOHC so that we can run updates quickly and also build other humidity products as part of HadISDH.

Its important the version of the code used is well documented and that running it can be reproduced.

In theory, for the same input data we should be finding exactly the same number of changepoint locations, of the same magnitude and direction.

In practice, there will be small differences due to the different operating system and fortran set ups and floating point inconsistencies - how much difference is too much?

Main Conclusions:

Running the code on two different computers at MOHC gives no difference - the code is stable in house.

Running the code on the same data at MOHC does give a small number of differences compared to running at NCDC.

We do not believe these differences to be significant.

Remaining questions to be answered:                                                                      

Does the new homogenisation change any large scale features?

Working:

PHA set up using hadisdh7311.conf and inhomog.parm.MTHLY.TEST.incl:

• region=hadisdh
• phaext=TEST
• dataset=hadisdh
• datatag=hadisdh7311
• datatype="P"
• endyr=2011
• icand=0

• begyr = 1970
• endyr = 2012
• maxstns = 4000
• maxnstns = 200
• amiss = -99.99

PHA run on 3688 HadISD stations of monthly mean specific humidity (g/kg*100) from 1973-2011

7367 changepoints were found in total

7330 changepoints for network found that match exactly in location (month)

7160 (98%) of these match exactly in magnitude/direction

Figure 1 shows the differences - these are all between -0.14 and +0.13. There is some tendency for MOHC homogenisation to be larger - more negative (when the adjustment is negative) and more positive (when the adjustment is positive) but this is only for the 2% of changepoints that are not in exact agreement.

Figure 1. Differences between PHA homogenisation for HadISDH run at NCDC (OLD) and at MOHC (NEW) using exactly the same data. Circles show each changepoint for which the locations matched (7330 out of 7367) coloured by latitude band.

The changepoints that do not match in location belong to 14 stations (out of 3688). For 13 stations all except one changepoint matches in location exactly but either the OLD (NCDC) or NEW (MOHC) run finds one extra changepoint. In one case (station 987410) the OLD homogenisation finds two changepoints but the NEW finds none.

Looking at individual stations the changes are small - here are 10 examples of the largest changes. In all cases, red=raw, blue=homogenised, black=raw neighbours, decadal trends and 2sigma uncertainty are shown. None show any change in the long-term trend.

Station 401910, 31.2330N, 34.7830E, 280.0m, BEER-SHEVA, Israel - no difference in trend

MOHC homogenisation 401910

NCDC homogenisation 401910

Station 412460, 24.4670N, 56.6330E, 4.0m, SOHAR MAJIS, Oman - no difference in trend

MOHC homogenisation 412460

NCDC homogenisation 412460

Station 412680, 22.5330N, 59.4670E, 14.0m, SUR, Oman - no difference in trend

MOHC homogenisation 412680

NCDC homogenisation 412680

Station 413140, 17.6670N, 54.0170E, 467.0m, THUMRAIT, Oman - no difference in trend

MOHC homogenisation 413140

NCDC homogenisation 413140

Station 413160, 17.0330N, 54.0830E, 23.0m, SALALAH, Oman - no difference in trend

MOHC homogenisation 413160

NCDC homogenisation 413160

Station 430410, 19.0830N, 82.0330E, 553.0m, JAGDALPUR, India - no difference in trend

MOHC homogenisation 430410

NCDC homogenisation 430410

Station 486650, 2.2670N, 102.2500E, 9.0m, MALACCA, Malaysia - no difference in trend

MOHC homogenisation 486650

NCDC homogenisation 486650

Station 616300, 15.6500N, -13.2500W, 17.0m, MATAM, Sengal - no difference in trend

MOHC homogenisation 616300

NCDC homogenisation 616300

Station 768056, 16.7500N, -99.7500W, 5.0m, GENERAL JUAN N ALVA, Mexico - no difference in trend

MOHC homogenisation 768056

NCDC homogenisation 768056

Station 837680, -23.3330S, -51.1330W, 569.0m, LONDRINA, Brazil - no difference in trend

MOHC homogenisation 837680

NCDC homogenisation 837680

Station 912320, 15.1170N, 145.7000E, 32.0m, SAIPAN MARSHALL ISLAND, Mariana Is. - no difference in trend

MOHC homogenisation 912320

NCDC homogenisation 912320

 But there are some stations that show differences in the long-term trend between the OLD (NCDC) homogenisation and the NEW (MOHC) homogenisation. 55 stations in total show different trends. Out of 3202 stations that have adjustments applied to them, 3145 have identical trends - that's 98%. Of the 57 stations that differ, these differences are within -0.30 and +0.21 g/kg/dec. In only two cases does the NEW (MOHC) homogenisation change the direction of the trend: for stations 969350 and 712430 NEW homogenisation makes the trends negative when they were positive - see Figure 2 below.

Figure 2: Difference in trends between the NEW (MOHC) homogenisation and the OLD (NCDC) homogeinsation. Trends are shown as decadal trends in g/kg. 3202 stations are shown - the others do not have adjustments applied. 57 stations show a difference in trend. Only two stations (identified in red) show a change of direction in the trend.

These stations are also shown in more detail comparing their raw, homogenised and raw neighbour series for both OLD (NCDC) and NEW (MOHC):

Station 712430, 49.7000N, -112.7830W, 921.0m, LETHBRIDGE CDA, Canada

MOHC homogenised 712430

NCDC homogenised 712430

Station 969350, -7.3670S, 112.7670E, 3.0m, SURABAYA/JUANDA, Indonesia

MOHC homogenisation 969350

NCDC homogenisation 969350

I do not know why these differences have occurred. In the latter case the network is varied and patchy and so the convergence upon changepoint magnitude and direction may be down to quite a small margin where very small differences in floats may make a difference.

The differences are very few and mostly negligible - especially in terms of the bigger picture which is what this dataset is to be used for - assessing long-term trends on large spatial scales. However, it would be good to know why these differences have occurred.