A new estimator of heat periods for decadal climate predictions &ndash; a complex network approach

Weimer, Michael; Mieruch, Sebastian; Schädler, Gerd; Kottmeier, Christoph

doi:https://doi.org/10.5194/npg-23-307-2016

Articles | Volume 23, issue 4

https://doi.org/10.5194/npg-23-307-2016

© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/npg-23-307-2016

© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 23, issue 4

Research article

|

24 Aug 2016

Research article |

| 24 Aug 2016

A new estimator of heat periods for decadal climate predictions – a complex network approach

Michael Weimer, Sebastian Mieruch, Gerd Schädler, and Christoph Kottmeier

Download

Final revised paper (published on 24 Aug 2016)
Supplement to the final revised paper
Preprint (discussion started on 13 Oct 2015)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC C527: 'Referee Comment on npg-2015-52 "Predicting climate extremes - a complex network approach" by M. Weimer, S. Mieruch, G. Schädler, and C. Kottmeier', Anonymous Referee #1, 25 Oct 2015
- AC C592: 'Answers to the referee comments #1', Michael Weimer, 15 Dec 2015
RC C554: 'Review for manuscript "Predicting climate extremes – a complex network approach" by M. Weimer, S. Mieruch, G. Schädler, and C. Kottmeier', Anonymous Referee #2, 25 Nov 2015
- AC C593: 'Answers to the referee comments #2', Michael Weimer, 15 Dec 2015

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Anna Mirena Feist-Polner on behalf of the Authors (03 Feb 2016) Author's response

ED: Referee Nomination & Report Request started (16 Mar 2016) by Jürgen Kurths

RR by Anonymous Referee #2 (25 Mar 2016)

Suggestions for revision or reasons for rejection

Second Review for manuscript "Decadal prediction of heat periods based on regional climate model data – a complex network approach" by M. Weimer, S. Mieruch, G. Schädler, and C. Kottmeier

I am sorry to say that I am still not convinced by the authors analysis. My main issues are the following:

1. It is still not stated clearly that the authors do not propose a method to predict heat periods, but rather propose a different way to define a skill for the prediction of heat periods using a regional climate model: in the modified title, it says “prediction of heat periods”, and in the abstract the authors write “We show that the skill of the network measure to predict the low frequency dynamics of heat periods is superior to the typical approach”. In the main text, it is claimed “that the network method is clearly superior in three regions”, and in the conclusion: “We found that the network approach is superior (significance is ≈5%) to the standard approach in predicting heat periods in Europe”, and that the network approach is is “the more robust estimator of heat periods” These sentences are, at least, misleading.

As explained in my previous review, the authors use P70 as a proxy for heat waves, and quantify the skill of heat wave prediction by CCLM by comparing the development of P70 in the CCLM data with observed heat waves, instead of comparing the observed number of heat waves with the modelled number of heat waves (the latter would be the standard approach). However, P70 is only an indicator of coherent behavior of the time series. More heat waves trivially lead to more coherent behavior of the time series, but so do, as I already mentioned, more cold episodes, or any other periods of joint behavior. This is the reason why I agree that P70 is correlated to the number of heat waves, but still question the suitability of P70 as a predictor for heat waves (which the authors found contradictory). The standard way of assessing the forecast skill is to compare the number of observed and simulated heat waves. Instead, the authors propose to compare the number of observed heat waves with a characteristic of the simulated data (P70) which is not in one-to-one correspondence with the number of heat waves, but may instead react to many other climatic features as well (given that it only measures coherent behavior of the time series, no matter if daily maximum temperature are employed).

2. I don’t think that the significance test proposed by the authors is appropriate. Instead of first placing 20 red and 20 blue entries, and then replacing 8 of them by white entries, the null model should be that each entry has a probability of 0.5 to be either red or blue, with a total of 20% undecidable. Hence, fill an 8x5 matrix with standard normally distributed values, color the entries blue if the value is above 0.25, red if it is below -0.25, and white otherwise. Here, the value 0.25 is chosen such that on average 20% of the entries are colored white (the ~20th percentile of the standard normal). With this model, I get a probability of ~0.2 that I have 5 or more regions with more blue than red entries, indicating that the author’s result would only be significant at a confidence level of 0.2 (see top panel in significance.pdf), which is a quite weak statement.

Alternatively, one could simply ask for the probability of having at least 18 blue entries (the observed number) in total (regardless of how they are distributed in the single rows) in the same setting as above (note that the expectation value for this is 16!), for which I get ~0.2 again (see bottom panel in significance.pdf). Note that N = 10’000 matrices were used to quantify this.

I agree that both tests I proposed suggest (although not significantly) that there are more blue boxes in Fig.7 than expected. However, this does not indicate that the proposed skill definition is better suited for heat wave forecasts. Instead, these results only indicate that the dependence between P70 and observed heat periods is a little bit (and non-significantly) stronger than the dependence between simulated and observed heat periods themselves. For example, there might be a period of intermediate (i.e., neither very high or very low) temperature anomalies that are coherent over a region of interest in the CCLM data, which is detected as a high P70 and according to the authors “translated” to the occurrence of a heat period when compared to the number of actually observed heat periods in that decade, ultimately resulting in a high skill value although something completely different was actually going on.

Since there is no physical reason to believe that P70 only reacts to heat periods, and the significance tests also fail, I cannot suggest this study for publication.

Referee Report: PDF

Hide

RR by Anonymous Referee #1 (27 Mar 2016)

Suggestions for revision or reasons for rejection

Dear authors,

I did not fully understand the draft version of your paper. Now I understand more of what you have done but unfortunately I find your ideas ever less convincing. Have you actually ever checked how well your network method is at identifying heatwaves in the observational data? I do not find anything on this matter in the manuscript. You should make a classic prediction skill analysis and state the rates of false positives, false negatives and so on. This is the fundamental problem I have with your work. Related and other issues are listed in the following.

Page 6, paragraph around word "true":
(i) So, what do you consider a "true" heatwave, then?? Is your whole approach based on a new idea of how heatwaves should be defined? If yes, then please explain this new idea.
(ii) The way you write it here ("static threshold" and so on), I get the impression that you do not intend to apply any bias correction method while, in fact, later you implicitly apply one in your percentile-based "standard" heatwave detection method.

Page 6, "[...] but also to consider complex long-term climate evolution in contrast to short-term weather": I do not understand what you mean by that.

Page 13, equations (6) and (7): This quantification of prediction skill is most probably useless. As far as I understand you want to predict the number of heatwaves a given summer will be struck by. You should look at how well your network metric actually reproduces the interannual variability of heatwave counts.

Page 14, first paragraph: The figures in the supplement tell a pretty different story than Fig. 4. You should not choose your best example for the paper and hide your bad examples in the supplement. If you want to objectively assess the prediction skill of a method, you will have to talk about failures as openly as about successes.

Page 15-16: Sorry to say this, but I think that this significance test is a good example for why quite many statisticians these days are worried about flawed research being hidden behind statistical significance. (See, e.g., http://amstat.tandfonline.com/doi/abs/10.1080/00031305.2016.1154108 or read Nate Silver's The signal and the noise.) I wrote above that I do not consider the metrics defined in equations (6) and (7) appropriate for any quantification of the skill of you heatwave identification method. Consequently, I consider any significance test based on any application of these methods useless in terms of telling anything about whether your method is any good.

Hide

RR by Anonymous Referee #3 (01 Jun 2016)

RR by Anonymous Referee #4 (30 Jun 2016)

ED: Publish subject to minor revisions (further review by Editor) (11 Jul 2016) by Ana M. Mancho

AR by Michael Weimer on behalf of the Authors (20 Jul 2016) Author's response

ED: Publish as is (21 Jul 2016) by Ana M. Mancho

Short summary

This paper is the first time that a complex network approach has been used for analysis of decadal climate predictions. We have developed an alternative estimator of heat periods based on network statistics, which turns out to be superior for parts of Europe. This paper opens the perspective that network measures have the potential to improve decadal predictions.