Quick update

Just a very quick update on this.  We have now received all the data we were expecting to receive - and so can start adjusting some of the processing criteria. 

However, to give an indication of the number of stations and rough coverage for trend analysis, I thought I'd put the latest plots here.  Once the processing criteria are settled on my intention is to put a draft document together.

Fig 1 - Stations in the Rx1day grids

Fig 2 - Stations in the TXx grids

As you can see, we have stations now covering the majority of the globe.  However, we want to ensure that the hard work from colleagues who have submitted data is recognised, and include as many stations as makes sense.  Currently, we're requiring that stations report relatively continuously during the period when they do - so can only have a maximum of 2 missing years, and I think that this is reducing the final numbers from over Brazil, India and elsewhere. 

Fig 3 - Rx1day trends (only from grid boxes with 66% completeness)

Fig 4 -TXx trends (only from grid boxes with 66% completeness)

And from these trend plots, the coverage is much reduced.  This arises from knock on effects, fewer stations being currently selected because of the completeness criterion.  Those that do may result in a too sparse network to ensure grids are calculated (which is more likely for indices which have a short spatial correlation).  And when they are, they may not be for sufficient years to calculate the trends. 

Fig 5 - Timeseries of Rx1day, compared to GHCNDEX and HadEX2

Fig 6 - Timeseries of TXx, compared to GHCNDEX and HadEX2

What is interesting is that, despite the limitations of the spatial coverage, the long term trends in quasi-global averages match very well with the other ETCCDI datasets available (GHCNDEX and HadEX2).

We're seeing what can be done to improve the spatial coverage without impacting the integrity of the dataset, so watch this space.

Filling in the Globe

Just a quick update this time.  Work has gently been going on behind the scenes to include more data and continue to fill in the globe.  Thank you to the many folks who have spent time to process the data they have kindly sent over for inclusion in this project.  We now have data from China, India, Myanmar, Brunei and Malaysia, with more on the way from some other countries.  For some indices this has really reduced the regions for which we do not have coverage, though of course this depends on the interpolation our gridding routine can perform.

To give an idea of where we are at, below are the same plots from the end of last month, but with the current coverage. These are all for the 2.5 x 3.75 degree gridding options using 1961-90 as the reference period (where appropriate). We have also made trial versions using smaller grid boxes (1.25 x 1.875 deg), a simpler (non-interpolating) gridding method, and also using 1981-2010 as a reference period, which allows the inclusion of some pre-calculated indices from some of our contributors.

Fig 1 - stations for CDD annual

Fig 2 - stations for CDD monthly

Fig 3 - stations for TNn annual

Fig 4 - stations for TNn monthly

And of course, the trends are here (and yes, we've not yet looked at the colour maps for monthly CDD!):

Fig 5 - linear trends for CDD, Annual

Fig 6 - linear trends for CDD, January

Figure 7 - linear trends for CDD, July

Figure 8 - linear trends for TNn, Annual

Figure 9 - linear trends for TNn, January

Figure 10 - linear trends for TNn, July

As you can see, in some cases we have more complete spatial coverage for the annual indices, rather than in individual months, which tends to arise when we have not been given the monthly index values.

We're still working on including more data from new sources and also check that the station selection criteria are sensible - to see if we can improve the underlying station coverage where possible.

Latest data update

In the last few months we've been gently gathering more data, and adapting the routines used to handle it to improve the coverage that we have.  

GHCND - this was using just the HCN over the US, but we've now expanded this to include the GSN and so have world wide coverage.  There is a risk that by using these data we are including stations which have significant inhomogeneties in them.  However, in regions with other data sources, the effect should be minimised as a result of the other stations present.  And in regions with no other data, having some data is better than none!

ACRE - we have adapted how this dataset is processed by Climpact2.  As few stations will have overlaps with the reference period we are using (1961-90 by default) the quality control removes these stations, and so no indices are calculated - even those where the reference period isn't used.  We now use a temporary reference period for each station and remove the indices which rely on it at a later stage.

Arabian, West African & South African data - these have been calculated using a more recent reference period (1981-2010).  Again, we've adapted how these are handled to make sure as many stations are retained and those requiring the base period are removed.

What our intention is that we will release a version for a 1981-2010 reference period - where this situation is reversed. Of course, for raw data where the indices are calculated using Climpact, this isn't a problem. And for ECAD, LACAD and SACAD we can revert to the underlying daily data from EOBS, "LAOBS", and "SAOBS" and recalculate the indices.

We are continuing to source data - over Brazil, China, India, Iran, Japan, and Mexico - and also hope to obtain more data over south-east Asia in due course.  We have set a deadline of June 2019 for data submissions to allow the finalisation of the paper.

The latest station distribution plots are below.  This time we're showing the annual coverage and also for one of the months - as these tend to have more stations pulled through.  The reason for this is that there are data presence requirements on a monthly and annual basis - and it is more likely to have insufficient data to calculate an annual quantity, than to calculate a monthly quantity.

Fig 1 - Consecutive Dry Days - Annual

Fig 2 - Consecutive Dry Days - Monthly

Fig 3 - Minimum Tmin - Annual

Fig 4 - Minimum Tmin - Monthly

In some cases the data that have been kindly provided only contain the annual indices, and so the coverage is different between annual and monthly maps shown below.

And here are the trend plots (simple linear values) for the same indices, also monthly (January and July) and annual.  We note that the colour scales are the same for the annual and monthly maps - which we will reassess and adjust in future runs.

Fig 5 - trends for CDD, annually, in January and in July

Fig 6 - trends for TNn, annually, in January and in July

The next step is to run some different versions of this dataset, using the alternative reference period (1981-1990) and also using smaller grid boxes, to more closely match the resolution of the current generation of GCMs.

Improved spatial and temporal coverage

Since the last post, more data have arrived and been included in the latest run of HadEX3.  These include temperature and rainfall over Canada and Chile, and rainfall over Australia.

With the Angular Distance Weighting gridding, these additions are starting to fill in the globe for some indices - those which have long correlation length scales.  However, our quest for further data is not over, and we are indebted to our colleagues around the world for their efforts in providing data and calculating the indices.

Station Distribution for TX90p (annual)

Station distribution for R10mm (annual)

The location of all the stations which currently contribute to HadEX3 for two example indices are shown above.  As is clear, we still have large gaps, in Africa and Asia, as well as parts of South America, but we are working on filling these in the next few months.  However, as always, if you are able to send daily Tx, Tn and Precipitation data, or know of collections we could use, please get in touch!

Trend in TX90p (annual)

Trend in R10mm (annual)

Using the preliminary data the above two panels show the linear trends in these two example indices.  These plots are automatically produced, so the colour-scales are not necessarily the best. What is also clear is that some stations in central-north Africa seem to have some unexpected values for TX90p which are influencing the region surrounding them.  These may be true measurements, but are in stark contrast to neighbouring regions.  Similarly over south-eastern Asia there are strong trends not occurring in neighbouring grid-boxes.  As these are preliminary plots, we are not commenting on the correctness, just that we should take a look and check.

 

Timeseries of TX90p, comparing HadEX3 with earlier products (and matching to HadEX2 coverage).

Timeseries of R10mm, comparing HadEX3 with earlier products (and matching to HadEX2 coverage)

 

What we can also do at this stage is compare the "global" average (land surface, where there are data) to previous products in the "Climdex Family".  Even at this point, with relatively large gaps in the spatial coverage, there is some agreement with HadEX2 (and more so when HadEX3 is coverage matched). For TX90p, as this is an index using a reference period, then GHCNDEX also matches relatively well.  However, for R10mm, although the short-timescale variation is reasonable, there is an offset to GHCNDEX, most likely due to different regions having data. This is also a likely cause for the "edge-effects" in the most recent years, as there are no stations in Asia and few in Africa post-2014 (see below).

HadEX3 stations for R10mm in 2005

HadEX3 stations for R10mm in 2015