The seasonal prediction of the WNAO for the period of 1982 to 2017 is shown in Fig. . Every dot represents one WNAO value of one ensemble member, which has also available the full meteorological and oceanographical fields during the associated winter period. These hindcast predictions for the WNAO have a large spread, covering the range of the observations given by the reanalysis, but do not give indication of a specific NAO value 2 to 4 months ahead. As a general skill measure the community applies correlation skills. Those measures have indicated in recent years significant hindcast skill for several different prediction systems .

Our approach will be applied to every single year independently. As an example we choose the year 2010, which shows an extreme negative WNAO value. The first step is to generate one probability density function (pdf) for each ensemble member prediction (ei) of the WNAO value, which is generated by a 2000-member bootstrap of the EOF fields . In the bootstrap the first EOF field is recalculated by resampling the mean sea-level pressure fields from each year. To create from these predictions a pdf for all ensemble members (E), mixture modelling at discrete NAO index values is applied: 1E(v)=∑i∈Iei(v). Here, v corresponds to each value of the discretised NAO values and I to the indices of the ensemble members. The chosen resolution for the discretised NAO values is 0.01 and, after creating the sum of all single-member pdfs, the overall pdf E is normalised. For 2010 the results are shown in Fig. . As expected from Fig. , the dynamical model prediction has a very broad pdf equating to a low signal.

To sharpen the prediction we introduce literature-backed physical statistical predictors. As predictors (pi) we use those defined by : sea-surface temperature in the Northern Hemisphere, Arctic sea-ice volume, Siberian snow cover and stratospheric temperature at 100 hPa. All predictors and their influence on the WNAO have been discussed in the paper. For the physical validity of a prediction the selection of the correct predictors is essential and has to be adapted to any newly analysed phenomena individually. Each predictor makes a prediction from the climatic state taken from the ERA-Interim reanalysis before the initialisation of the dynamical model for a WNAO value in the following winter. For the predictors a normalised index over the hindcast period is calculated by forming the mean over the significantly correlated areas between the physical field and the WNAO index. It has been shown by a real forecast test in that this approach is usable also in cases where the predictor is only formed with past information instead of the whole hindcast period.

We treat the predictors pi like the ensemble members before and apply an empirical mixture modelling. For the year 2010 the results are shown in Fig. . Due to the limited number of predictors compared to the ensemble members, and in the shown case also due to their alignment, the resulting statistical prediction pdf (P) is much sharper than the dynamical model prediction.

To create a combined prediction the two pdfs (E and P) are after normalisation multiplied at each of the discretised NAO values: 2M(v)=E(v)⋅P(v). After another normalisation the final combined prediction M creates the statistical–dynamical prediction for the seasonal NAO prediction in the specific year. The pdfs of the observations (O) are determined by the same bootstrapping mechanism as the one applied for the hindcasts. The result for the year 2010 is shown in Fig. . The pdf of the combined prediction is close to the one of the statistical predictions, but shows differences where there is additional information from the dynamical model prediction. Therefore, the combined prediction shows a clearer signal than the dynamical model prediction, which does not give any indication of a specific NAO value at all.

To evaluate the performance of the three different predictions (E, P and M) and compare the predictions with the observation, we use two different scores based on the same formulation. The first is based on the Earth mover's distance . The one-dimensional EMD can be derived by 3DEMD(f,g)=1nb∑i=1nbF(vi)-G(vi), where f and g are two pdfs and F and G the associated cumulative distribution functions (cdfs). nb describe in this case the number of discretised values vi of the cdfs.

The second is the IQD, which is defined in its discrete formulation as 4DIQD(f,g)=1nb∑i=1nb(F(vi)-G(vi))2. It must be mentioned that the IQD is similar to the continuous ranked probability score (CRPS) but is defined for non-deterministic observations. As a consequence, while CRPS needs to have a point observation, the IQD can take into account the full uncertainty distribution of an observation.

We define the scores for both metrics by comparing the pdfs of the model prediction (M), the observations (O) and the climatology (C). It is calculated for any prediction A by 5q(A,O)=1-D(A,O)D(C,O). When D is DEMD we call the score the 1D-continuous-EMD score, and when we apply DIQD it is the 1D-continuous-IQD score.

In the case of a perfect prediction the score becomes 1, a model prediction equal to a climatology 0 and negative for a worse prediction than the climatology. Since the NAO index is normalised for mean and standard deviation, we use as climatology a standard normal distribution N(0,1). It is important to note here that compared the two metrics (EMD as area validation metric). While the EMD is a metric measuring the distance between the pdfs, it is in contrast to the IQD not a proper divergence measure. As a consequence, the EMD prefers, unlike the IQD, underdispersed model simulations. In the following we will demonstrate the effect that the choice of the two different metrics has on the evaluation.

To estimate uncertainties, we use 500 randomly selected uniformly distributed weightings of the ensemble members between 1 and 0 and create with those a pdf for the scores.

To estimate the post-processed variable field, we calculate a weighted mean of the meteorological variable fields, where the field of each individual member is weighted by a coefficient ci. The weighting coefficients ci are estimated by weighting the predictions A (each of E, M and P) with each of the pdfs of the ensemble members (ei): 6cA,i=∑vei(v)⋅A(v). Weighting each ensemble member with its associated coefficient cA,i and calculating the weighted mean of the atmospheric fields of the individual ensembles then generate the model prediction for the specified field and prediction.

For evaluation of the meteorological variable fields, we apply three different strategies. The first is the anomaly correlation coefficient (ACC), a common measure of skill in seasonal predictions. The second and third approaches are to use the 1D-continuous-EMD and 1D-continuous-IQD scores at every grid point. As a climatology all observational values for the investigated time frame are chosen. The observation in each year is a single value with 100 % as a weight. In the case of the weights for the ensemble member, each value of the variable at the grid point gets weighted with the relative weight cA,i given by the three different predictions. With this approach it is possible to calculate the 1D-continuous-EMD and 1D-continuous-IQD scores for each of the three different predictions. In Sect.  the relative positioning between two predictions is shown. Significances are here determined by , which determines the skill significances by comparisons to random walks.

In a next step we evaluate the yearly performance of the WNAO prediction of the three different predictions (E, P and M) with the 1D-continuous-EMD and 1D-continuous-IQD scores. Figure  shows that the results of the combined (M) and statistical (P) predictions are clearly better performing than the dynamical model results (E). In most years, the combined and statistical predictions demonstrate skill for the 1D-continuous-EMD score compared to a climatological prediction over the whole uncertainty range. The dynamical model prediction has less variability over the years in skill than the other two predictions and in only a few years is able to reach the average skill of the combined prediction. The median and interquartile range of the summed-up prediction skill for all evaluated years for the combined prediction (0.39[0.21;0.60]) is higher compared to the dynamical (0.12[0.01;0.22]) and statistical (0.37[0.17;0.56]) predictions. There is only one year (2003) with a strong discrepancy of the combined and statistical predictions and a clearly negative score. For 1D-continuous-IQD the results are less clear. In this case the median of the combined prediction is much closer to the dynamical prediction than the statistical prediction. The uncertainty range for the combined and statistical predictions also increases relative to the dynamical prediction, which can be explained by their sharpness. To better evaluate the performance of each prediction with respect to the other predictions, we determine the relative ranking of the median of each prediction in each year for both scores. The rankings are counted for the whole hindcast period and the results displayed in Table . For the 1D-continuous-EMD score the dynamical prediction has in only a few years a better prediction skill than the other two predictions. In the majority of the years its prediction skill is lower than both other predictions. Looking at the best prediction for each year, the statistical and combined predictions are on equal terms. Nevertheless, the combined prediction is much more unlikely to be the worst of the three predictions in a year, while the statistical prediction takes much more often the third rank. These results show that the combined prediction is closer to the statistical rather than dynamical prediction. In case the combined prediction is not the best one, it is in almost all cases better than one of the two. As such it offers a smoothing of the prediction skill, preventing many worse predictions. In the case of the 1D-continuous-IQD score, the result differs clearly. Here the dynamical prediction is much more competitive. It shares almost equally with the statistical prediction first place, while the statistical prediction hardly changes its statistics of positions. As a consequence the combined prediction is much more often in last place. Still, it is the prediction with the most middle places of the three predictions, stressing the argument that the combined prediction is a mixture of the other two.