The goal of MOS is to improve upon the quality of physical NWP models by identifying their weather-related statistics using regression models trained on historical observations and corresponding predictions . Since MOS was first introduced 50 years ago, there have been substantial changes in both (i) what is meant by weather-related statistics in the context of MOS and (ii) the flexibility of the regression methods used to identify these.

In the simplest case – with a single (deterministic) forecast for an atmospheric quantity and forecast errors that may be assumed to be Gaussian – systematic biases in the NWPs can be identified using a classical linear regression. A classical example is to regress observed temperatures y on the corresponding temperature predictions x: 1E(y∣x)=β0+β1⋅x. MOS coefficients β0 and β1 then describe how the temperature forecast from the physical model should be corrected to better match real-world observations. For the ideal case of an NWP with no systematic biases, these values would be β0=0 and β1=1. In the classical linear model, coefficients are estimated by minimizing the sum of the squared errors (OLS) on some set of training data, which is equivalent to minimizing the root mean square error (RMSE) of the residuals.

This simple post-processing model not only allows biases in the NWP to be corrected but also implicitly estimates the uncertainty of the post-processed forecast. Namely, if y can be assumed to follow a Gaussian distribution conditionally on x, the minimum RMSE obtained during model estimation is an estimate of the standard deviation σ of the forecast distribution, and Eq. () may be rewritten as 2y∼N(μ,σ2),whereμ=β0+β1⋅x,andlog⁡σ=γ0.

Generally though, weather forecasts do not have constant uncertainty, and many atmospheric variables do not follow Gaussian distributions, even conditionally. To allow for more flexibility in post-processing, modern implementations of MOS therefore often employ distributional regressions , also known as generalized additive models for location, scale, and shape GAMLSS,. In distributional regression, the observation y can follow some other parametric distribution, and all parameters (not just the mean) of this distribution are modeled on appropriate predictors derived from the NWP (ensemble).

Typically, coefficients of distributional regression models are estimated by maximizing the log-likelihood ℓ of the distributional parameters given the observations or by minimizing the continuous ranked probability score (CRPS). One prominent example in the post-processing literature is the nonhomogeneous Gaussian regression (NGR) of , also known as EMOS, where the parameters μ and σ in Eq. () are modeled on the mean and spread of an NWP ensemble, respectively. Other examples include truncated Gaussian and generalized extreme value response distributions for forecasting wind speed and censored and shifted gamma distributions for forecasting precipitation .

In the subsequent sections, we therefore assume that the base MOS for y explained by x uses some parametric model with likelihood ℓ((y,x),θ) and r-dimensional parameter vector θ that is estimated through likelihood maximization: 3θ^=argmaxθ∑i=1Nℓ((yi,xi),θ). In the example from Eq. (), the likelihood is Gaussian with parameter vector θ=(β0,β1,σ), but other distributions, like the ones from the previous paragraph, could be used in the same way.

In order to adapt the coefficients of the base MOS chosen in Sect.  to some additional weather-related predictors z1,z2,…zk, a single MOS tree partitions the predictor space Z1×Z2×…×Zk into disjointed subsets that can each be considered to be homogeneous weather situations for the purpose of NWP post-processing – i.e., where constant MOS coefficients work well. It is grown using model-based recursive partitioning algorithms according to the following steps.

MOS coefficients θ are estimated through likelihood maximization on the i=1,…N, observations yi and corresponding predictions xi in the dataset. This is done by solving the first-order condition 4∑i=1Ns((yi,xi),θ)=0, where 5s((yi,xi),θ)=∂ℓ((yi,xi),θ)∂θ1,…,∂ℓ((yi,xi),θ)∂θr⊤ contains the partial derivatives of the log-likelihood with respect to each coefficient – i.e., the model scores – evaluated at the ith observation pair (yi,xi).

Scores with respect to each coefficient are again computed at all observations (Eq. ) and evaluated at the estimated coefficients θ^=(θ1^,…,θr^) from step 1. Since the estimated coefficients were obtained using Eq. (), each score vector has a mean of zero. If the single MOS with constant coefficients fits well, the scores for each observation should randomly fluctuate around zero. On the other hand, systematic departures of the scores from zero along some of the variables in z suggest that predictions can be improved by splitting the data and estimating separate post-processing models based on the two resulting subsamples. Whether or not the scores fluctuate randomly or depend on one of the weather-related predictors can be assessed using an independence test between the scores and each of the variables in z see permutation tests of. If there is a significant dependence with respect to at least one of the variables, then the most significant variable (i.e., with the smallest p value) is selected for splitting. The underlying test statistic captures the overall dependence in all score components (i.e., all MOS coefficients) simultaneously using a quadratic form. To account for assessing multiple variables from z, a Bonferroni correction for multiple testing is employed.

Once the splitting variable zj has been selected, an exhaustive search is performed over all possible split points to identify the partition that improves the log-likelihood the most. For numerical splitting variables, up to 2⋅(N-1) different MOSs are estimated in this step – separate models in both subsamples for each of the N-1 possible split points. The number of possible split points (and thus estimated models) decreases for each tie among the realizations of zj. For unordered categorical splitting variables, the number of possible split points is equal to the number of ways in which the different categories can be divided into two subgroups and thus increases exponentially with the number of distinct categories.

The three steps described above split a dataset of size N into two disjoint subsamples that are then each post-processed using a separate MOS. In order to grow a MOS tree, these steps are repeated for each subsample until a stopping criterion has been reached. The terminal nodes of a MOS tree (i.e., those nodes that are not split any further) contain disjointed subsamples of the full data that correspond to different homogeneous weather situations for post-processing with MOS (Figs.  and ).

Coefficients θ1,…,θk in each terminal node are obtained through likelihood maximization on the corresponding subsample. Note that this can also be understood as a weighted estimated using the full data, where weights are either 0 or 1, indicating whether or not the respective observation is in the subsample of interest. In the following Sect. , this idea is extended to use weights that may change smoothly (rather than abruptly) between 0 and 1. This can express the degree of similarity (with respect to MOS coefficients) between some new weather situation and those historical weather situations in the training data.

Individual MOS trees grown according to Sect.  are easy to understand and interpret (see Sect. ) but can be sensitive to small changes in the data and may have a suboptimal fit if the model parameters change smoothly with the weather situation variables. To solve this problem and improve out-of-sample predictive skill, a MOS forest combines partitions from many different trees grown on bootstrap-aggregated (bagged) data and using only a randomly chosen subset of the atmospheric variables in z for splitting at each node.

Given a MOS forest with T trees and Pt partitions in each tree t, MOS coefficients are adapted to a new weather situation z⋆∈Z1×Z2×…×Zr by maximizing the likelihood of the base MOS in relation to the full training data, as in Eq. (): 6θ^(z⋆)=argmaxθ∑i=1Nw(z⋆,zi)⋅ℓ((yi,xi),θ), but with observations (yi,xi) weighted according to 7w(z⋆,zi)=1T∑t=1T∑p=1Pt1((z⋆∈Ppt)∧(zi∈Ppt))∣Ppt∣. These weights thus capture how similar the new weather situation z⋆ is to any of the historical weather situations zi from the training data by computing how often they end up in the same homogenous weather partition from the different trees in the forest. Thus, they characterize their similarity with respect to the MOS coefficients.

By using partitions from many different trees to estimate the weather-adapted MOS, model coefficients are not restricted to a discrete number of unique values at most equal to the number of terminal nodes (as can be seen with estimates for σ from the MOS tree of Fig. ). Instead, coefficients are allowed to have smooth dependencies on the additional predictors, and, as a result, predictions are more stable (see estimates for σ from the MOS forest of Fig. ).

The MOS coefficients θ^(z⋆) that have been adapted to the new weather situation z⋆ can be used to post-process the corresponding forecast x⋆ in the same way as coefficients obtained from a MOS tree or from the base MOS itself. That is, the (log-transformed) probability density function for the unknown observation y⋆ is given by ℓ(y⋆∣x⋆,θ^(z⋆)), and the parameters of the response distribution are those values predicted by the MOS.

Using neighborhood weights as described above is commonplace in forests that contain more complex models rather than just a single scalar value in the terminal nodes e.g.,. An alternative approach would be to obtain the weather-adapted MOS model by averaging over MOS coefficients predicted by the individual trees. In the application considered here, the two performed equally well, except in the case of smaller sample sizes, where using the weights was slightly better.

The RainTyrol dataset is composed of observed daily precipitation sums from the Austrian National Hydrographical Service and NWPs from the 11-member global ensemble forecast system GEFS, of the US National Oceanic and Atmospheric Administration (NOAA). Data are available at 95 different stations in Tyrol, Austria, (and surrounding border regions) for all July days between 1985 and 2012, except the missing day of 19 July 2011. July is a month with some of the largest precipitation amounts and variability within the year, and both large-scale precipitation events and local convective events occur. To reduce skewness, daily precipitation sums observed at 06:00 UTC (robs) are power transformed using a parameter value of 1/1.6, which corresponds to the median of the power coefficients estimated at all stations .

There are 80 different predictor variables derived from the GEFS that can be used for post-processing. These include the direct predictor of the observation: the mean of the ensemble forecast of total (24 h) precipitation between +6 and +30 h but also the ensemble spread and its minimum and maximum. To account for the fact that summertime rainfall in Tirol is often caused by convection during the late afternoon and evening hours, ensemble statistics for the four sub-daily 6 h precipitation forecasts (+6 to +12, +12 to +18, +18 to +24, and +24 to +30 h) are also used as predictors. The same variations are also included for forecasts of the convective available potential energy (CAPE), a key ingredient in thunderstorms. Forecasts of temperature and temperature differences at and between different heights, as well as incoming solar radiation (i.e., sunshine), pressure, precipitable water, and total column-integrated condensate, are also added. Predictors derived from these atmospheric variables are not included for every sub-forecast, but the ensemble means and spreads are temporally aggregated using the minimum, maximum, or mean. For example, pwat_mean_max refers to the maximum ensemble mean of precipitable water forecasted by the GEFS for a lead time between +6 and +30 h. A thorough description of all available predictor variables and their naming conventions can be found in Table 1 of .

The ensemble forecasts described in Sect.  are post-processed using MOS forests, two other forest-based weather-adaptive reference methods, and a non-adaptive EMOS. An overview of the methods is given in Table , and more details are supplied below.

A MOS tree for Axams is grown from the first 24 years of data and is visualized in Fig. . The first split of the tree separates rare (n=23) weather situations with very high ensemble-averaged total column liquid condensate (tcolc_mean_mean) from the remainder of the data. The rest of the data are then split based on the maximum temperature predicted by the ensemble (tmax_mean_mean). The lower temperature branch has two subsequent splits: first based on precipitable water (pwat_mean_max) and then on the ensemble spread of precipitation (tppow_sprd). This results in three terminal nodes (nodes 5, 6, and 7). The higher temperature branch has three splits, the first based again on (tcolc_mean_mean) and the other two based on the ensemble spreads of 500 hPa temperature (t500_sprd_min) and precipitation (tppow_sprd1824). This results in four terminal nodes (nodes 11, 12, 13, and 14).

MOS models for each terminal node (i.e., distinct weather situation) are visualized in Fig. . The majority of observations are found in either node 5, 13, or 11. For nodes 5 and 13, the MOS are quite similar, the largest difference being that forecasts in node 13 are less certain (i.e., γ0 is greater). In contrast, the MOS used to post-process NWPs in node 11 is very different, with a strongly negative intercept for the mean model (β0=-8.16) and a high forecast uncertainty (γ0=1.09). This is because node 11 contains many days where the ensemble mean is greater than zero – i.e., some ensemble members predict precipitation for Axams – although no precipitation is actually observed. To understand when the tree makes such a prediction, it is only necessary to consider the splits in Fig.  that lead to node 11: high maximum temperature, low column liquid condensate, and narrow ensemble spreads for minimum temperature at 500 hPa and accumulated precipitation between 18:00 and 24:00 UTC.

MOS forests are compared to the reference methods described in Sect.  by evaluating the skill of post-processed forecasts using the widely used continuous ranked probability score CRPS,. To replicate a true operational scenario, all evaluations are performed out of sample on data that were not used to train the models. First, forecasts at Axams are evaluated using multiple replications of a randomized 7-fold cross-validation (Sect. ). At all other stations, forecasts are issued for a single hold-out fold (containing the last 4 years), and the remaining six folds (containing the first 24 years) are used for model training (Sect. ). Finally, models are also trained using different amounts of data (12, 6, and 3 years) to investigate their robustness in this respect (Sect. ).