For most statistical postprocessing schemes used to correct weather forecasts, changes to the forecast model induce a considerable reforecasting effort. We present a new approach based on response theory to cope with slight model changes. In this framework, the model change is seen as a perturbation of the original forecast model. The response theory allows us then to evaluate the variation induced on the parameters involved in the statistical postprocessing, provided that the magnitude of this perturbation is not too large. This approach is studied in the context of a simple Ornstein–Uhlenbeck model and then on a more realistic, yet simple, quasi-geostrophic model. The analytical results for the former case help to pose the problem, while the application to the latter provides a proof of concept and assesses the potential performance of response theory in a chaotic system. In both cases, the parameters of the statistical postprocessing used – the Error-in-Variables Model Output Statistics (EVMOS) method – are appropriately corrected when facing a model change. The potential application in an operational environment is also discussed.
Introduction
A generic property of the atmospheric dynamics is its sensitivity to initial conditions. This implies that probabilistic forecasts is necessary to adequately describe this behaviour . Indeed, these methods represent a way to go beyond the natural predictability barrier that the chaotic atmospheric models exhibit . These forecasts are at the same time subject to the impact of the presence of structural uncertainties, also known as “model errors”. Such errors degrade the forecasts as well, and their impact needs to be mitigated.
Statistical postprocessing methods are used to correct the operational predictions of the atmospheric models. An important family of statistical techniques used to postprocess the forecasts are linear-regression techniques, possibly with multiple predictors , also known as model output statistics (MOS). This rather simple but very efficient technique can be adapted to ensemble forecasts (e.g. ). One of the first approaches that was proposed is called the Error-in-Variable MOS (EVMOS) method because it takes into account the presence of errors in both the observations and model observables .
Despite their simplicity, most postprocessing schemes depend on the availability of a database of past forecasts, which allows one to train the regression algorithm by comparison with the observations database. Operational models are however subject to frequent evolution cycles, which are needed to improve their representation of the atmospheric processes. Therefore, there is a continuous need to recompute forecasts starting from past initial conditions with the latest model version to avoid a degradation of the postprocessing schemes due to model change. Such a recomputation of the past forecasts are called “reforecasts” and typically requires a huge data storage and management framework, as well as many computational resources .
For instance, the European Centre for Medium-range Weather Forecast (ECMWF) and the National Weather Service (NWS) both produce hundreds of reforecasts every week .
Recent research has investigated non-homogeneous regression with a time-adaptative training scheme, for which a trade-off between large training data sets for stable estimates and the benefit of a shorter training period for faster adaptation to data changes is considered . These results can help mitigate the impact of model change on postprocessing and may call into question the need for reforecast systems. These systems do however help to better represent rare events; they increase the size of the training data sets and greatly improve sub-seasonal forecasts , which can justify their very high cost.
The present work investigates another research direction and considers a new technique to reduce the cost of adapting a postprocessing scheme to a model change. This method relies on the response theory for dynamical systems and assumes that the model change can be written as analytical perturbations of the model tendencies. In this context, parameter modifications as well as new terms in the tendencies are potential model changes.
In Sect. , we start by introducing the Ruelle response theory that is used to adapt past postprocessing parameters to new model versions. A didactical example of such an adaptation is considered with a simple Ornstein–Uhlenbeck model in Sect. . It is used to describe the methodology and the concept involved. We show that obtaining a new postprocessing scheme after a model change requires the computation of the response of the average of the involved predictors, seen as observables of the system. In the simple case considered, exact analytical results for the response can be obtained up to any order. The correction of the model observables and the postprocessing parameters due to the model change only requires the response-theory corrections up to the second order.
In Sect. , a more complex case is considered with a toy model of atmospheric variability in the form of a two-layer quasi-geostrophic model with an orography. We compute the linear response of the parameters of the postprocessing scheme for two model change experiments involving a modification of the friction and the horizontal temperature gradient of the model. The response-theory approach provides an efficient correction of the postprocessing scheme up to a lead time of 4 d, which matches the lead-time window where the scheme's correction is efficient.
In the last section, we discuss the implications that this new method could have on operational forecast postprocessing systems, as well as new research avenues.
Response theory
The systems used to produce the weather forecasts are typically non-linear dynamical systems whose time evolution is governed by multi-dimensional ordinary differential equations:
y˙=F(t,y).
The generic chaotic nature of these systems for some parameter values implies that they are sensitive to the initial data used to produce the forecasts. For such chaotic dynamical systems, one can assume that a well-defined time-invariant measure exists with which the averages are performed. However, the existence of such measures has been proved for systems that are uniformly hyperbolic, and they are called Sinai–Ruelle–Bowen (SRB) measures , but rigorous proofs for other systems are rather difficult to obtain. A way to proceed is then to continue as if physical systems were uniformly hyperbolic. This assumption is called the Gallavotti–Cohen hypothesis . With this assumption, response theory has been successfully used in various weather and climate-related problems . Indeed, the systems used to produce weather forecasts are typically not uniformly hyperbolic, but thanks to the aforementioned hypothesis, one can still use what will follow and compare with the results obtained with experiments.
We point the reader to recent articles dealing with the validity of the response theory for weakly hyperbolic systems and time series .
It is the rationale behind the formal presentation of the linear response theory for general systems like Eq. () in . The main concepts that will be used in this article are now introduced.
Perturbations of dynamical systems
We shall assume for simplicity that the system defined by Eq. () is autonomous and given by
y˙=F(y).
In the general setting considered, let us assume that any given probability measure converges to a unique invariant measure ρ under the time evolution given by the Liouville equation of Eq. (). This measure is used to compute the average of an arbitrary observable A (a smooth function of the state y) of the system, which is given by
〈A〉y=∫ρ(dy)A(y),
and assuming the ergodicity of the system, a time average of the observable A along a trajectory of the system on its attractor can be equivalently performed as
〈A〉y=limT→∞1T∫0TdτAy(τ),
where y(τ) is a solution of Eq. (). If a perturbation Ψ of the dynamical system is introduced in the original system at time τ=0 as
y˙=F(y)+Ψ(y),
it induces a perturbation of the observable's average, which at the first order is given by
When taking the gradient of a function A, the notation ∇yA means taking the gradient at the point y, i.e. evaluating ∇yA(y).
δ〈A(τ)〉y=∫ρ(dy0)δAfτ(y0)=∫ρ(dy0)δy(τ)T⋅∇fτ(y0)A,
where fτ is the flow of the system defined by Eq. () mapping an initial condition y0 to the system's state at time τ as y(τ)=fτ(y0), and the capital T represents the transposition. δy is the perturbation of the trajectory of the system induced by the perturbation Ψ. This formula gives the transient response to the perturbation, and the long-time average of the integrand of Eq. () gives the stationary response to the perturbation, i.e. the sensitivity δ〈A〉y of the system observables to the perturbation . The higher-order corrections δk〈A(τ)〉y can in principle be computed as well but are quite complicated to obtain for chaotic dynamical systems; see for instance . We will show an analytically tractable case in Sect. .
The tangent linear model
The linear perturbation δy of the trajectories of Eq. () can be computed by introducing y+δy in Eq. () to get at the first order
δy˙=∇yF⋅δy+Ψ(y),
where y is the solution of Eq. () and ∇yF is the Jacobian matrix evaluated along this solution. Therefore, both Eqs. () and () have to be integrated simultaneously.
In the weather forecasting context, this latter linearised equation without the perturbation term Ψ is called the tangent linear model of Eq. () . Here, Eq. () is initialised with δy(0)=0 and provides the linear response of the trajectory y(τ) to the perturbation Ψ. It is thus assumed that there is no interference due to initial-condition errors in the perturbation problem. Note however that the effects on the trajectories of both the initial-condition perturbation and the Ψ perturbation can be investigated through this equation by setting δy(0)≠0, although we are not aware of any study of the response to both types of perturbations together.
The tangent model provides thus the tool through which we will evaluate the impact of the model change on the average used by statistical postprocessing schemes. In other words, the tangent model will allow us to take into account the information on the model change (viewed as a perturbation of the initial model) to modify the previous postprocessing scheme and adapt it to the new model version. The solution to Eq. () with δy(0)=0 is by
δy(τ)=∫0τdτ′Mτ-τ′,fτ′(y0)⋅Ψfτ′(y0),
where M is the fundamental matrix of Eq. () defined as
M(τ,y)=∇yfτ,
which is the solution of the homogeneous equation M˙=∇yF⋅M.
Using the chain rule, the response defined by Eq. () is rewritten in terms of the perturbation alone:
δ〈A(τ)〉y=∫0τdτ′∫ρdy0Ψfτ′(y0)T⋅∇fτ′(y0)Afτ(y0),
where the causality of the perturbation acting on the system and perturbing the averaged observable appears , since τ′<τ. We will also use this alternative expression throughout the article. Note that when the initial perturbation δy(0) is not equal to 0, additional terms to Eqs. () and () will appear. These will not be addressed here, but some references to this can be found in and .
Non-stationary response theory
Equation () gives the transient, non-stationary response to the perturbation, evaluated for averages computed with the invariant measure. However, in this work, we need to evaluate the response to perturbations for averages computed with non-stationary measures evolving in time. In that sense, it is a “non-stationary response theory”, which is performed with an arbitrary initial probability density. As such, all the formulas presented are valid if the measure being used is the measure at the time when the perturbation is introduced (τ=0), as shown in Appendix . In this case, other usual formulas obtained through substitution, for instance to obtain an adjoint representation of Eq. (), should be used with care, since the measure is no longer invariant and an extra Jacobian term appears in the integrand.
We will thus also assume that the measures ρτ being used are absolutely continuous with respect to the Lebesgue measure. In this case, we can write ρτ(dy)=ρτ(y)dy. We now present the problem of model change in the framework of postprocessing and show on a simple stochastic model
Response theory is also valid for stochastic models with a well-defined stationary measure, as shown for instance in .
how response theory allows us to tackle the issue.
A simple analytical example
In order to get a first impression of the impact of a model change on a postprocessing scheme, we consider two Ornstein–Uhlenbeck processes representing reality x(τ) and a model y(τ) of reality. These processes obey the following equations:
11x˙(τ)=-λxx(τ)+Kx+Qxξx(τ),12y˙(τ)=-λyy(τ)+Ky+Qyξy(τ),
where ξx and ξy are Gaussian white-noise processes such that
〈ξx(τ)〉=〈ξy(τ)〉=0〈ξx(τ)ξx(τ′)〉=δ(τ-τ′)〈ξy(τ)ξy(τ′)〉=δ(τ-τ′)〈ξx(τ)ξy(τ′)〉=0.
These are therefore uncorrelated Ornstein–Uhlenbeck processes with noise amplitudes Qx and Qy.
We then consider a change Ψy of model y(τ), possibly improving or degrading the forecast performances as
y^˙(τ)=-λyy^(τ)+Ky+Qyξy(τ)+Ψy(τ),
where
Ψy(τ)=-κ(δK+δQξy(τ))
with δK=Ky-Kx and δQ=Qy-Qx. It can represent, for example, a better parameterisation of subgrid-scale processes or an increase of the model resolution. Note that the best correction is obtained if κ=1.
We have thus reality x(τ) and two different models of it: y(τ) and y^(τ). We now want to evaluate the difference between a postprocessing scheme constructed before the model change (with the past forecasts of model y(τ)) and one constructed after (with the past forecasts of model y^(τ)).
The postprocessing method
We now consider a forecast situation where model y is initialised at time τ=0 with a perfect observation of reality: y(0)=x(0)=x0. We use the Error-in-Variables Model Output Statistics (EVMOS) postprocessing scheme to correct the forecasts of model y based on these initial conditions. In this context, given N past forecasts yn and observations xn, the correction of the univariate EVMOS postprocessing of variable x from a new forecast y(τ) is provided by the linear regression
yC(τ)=α(τ)+β(τ)y(τ).
The coefficients α and β are obtained by minimising the functional
J(τ)=∑n=1N{α(τ)+β(τ)yn(τ)}-xn(τ)2σx2(τ)+β2(τ)σy2(τ)
and are thus given by the following equations:
17α(τ)=〈x(τ)〉-β(τ)〈y(τ)〉,18β(τ)=σx2(τ)σy2(τ),
where
19σx2(τ)=(x(τ)-〈x(τ)〉)2,20σy2(τ)=(y(τ)-〈y(τ)〉)2,
The averages 〈⋅〉 are taken over an ensemble of past forecasts and observations. This approach has been developed to obtain a correct climatological forecast calibration. It constitutes a simple setting in which the impact of model changes can be evaluated and corrected. More sophisticated approaches can be evaluated in the future (other linear MOS schemes, ensemble MOS, etc.).
Since we are dealing with simple analytical models here, we can compute the theoretical values of the coefficient α and β with an infinite ensemble of past forecasts, and the averaged quantities involved in this computation are then given by the averages of an infinite number of realisations of the Ornstein–Uhlenbeck processes, as if we had an infinite ensemble of past forecasts.
Averaging the Ornstein–Uhlenbeck processes
For reality x and model y, we directly get the averages 21〈x(τ)〉=〈x0〉e-λxτ+Kxλx1-e-λxτ,22σx2(τ)=σx02e-2λxτ+Qx22λx1-e-2λxτ
and
23〈y(τ)〉=〈x0〉e-λyτ+Kyλy1-e-λyτ,24σy2(τ)=σx02e-2λyτ+Qy22λy1-e-2λyτ,
where we note that the model is initialised with the same initial conditions as reality:
〈y(0)〉=〈x(0)〉=〈x0〉,σy2(0)=σx2(0)=σx02.
We get the postprocessing coefficients before the model change α(τ) and β(τ) by inserting these expressions in Eqs. () and ().
Similarly, we get the same kind of results for model y^, after model change Ψy:
26〈y^(τ)〉=〈x0〉e-λyτ+Ky-κδKλy1-e-λyτ,27σy^2(τ)=σx02e-2λyτ+(Qy-κδQ)22λy1-e-2λyτ.
We also obtain the postprocessing coefficients after the model change α^(τ) and β^(τ) (see also the analysis in ). We can also compute the variation of the bias α:
α^(τ)-α(τ)=δα(τ)=β(τ)〈y(τ)〉-β^(τ)〈y^(τ)〉.
The ratio between the parameters β is given by
β^(τ)β(τ)=σy2(τ)σy^2(τ).
For τ≫max(1/λx,1/λy), we note that this ratio tends to
β^(τ)β(τ)≈11-κδQ/Qy,
and the difference between the biases α of the two models is approximatively given by
δα(τ)≈-β(τ)Kyλy1-κδK/Ky1-κδQ/Qy-1.
Let us now assume that model change Ψy can be considered as a perturbation of the initial model y. Using response theory, the averages 〈y^〉 and σy^2 can be estimated using the initial model y instead of the perturbed model y^. In turn, these new estimated averages give us the new postprocessing scheme coefficients α^ and β^. We now detail the results obtained by using this method.
Model change and response theory
After the model change, the forecasts are provided by model y^, and their time evolution is given by Eq. (). This model can be seen as a perturbation of model y by the term Ψy given by Eq. (). In such a case, given an observable A, its average after the model change can then be related to its original average by
〈A(τ)〉y^=〈A(τ)〉y+δ〈A(τ)〉y+δ2〈A(τ)〉y+…,
where the averages on the right-hand side are taken over the forecasts of model y. Response theory allows us to obtain the average over the model y^ forecasts (the left-hand side) based solely on the average over the model y forecasts. The y^ model forecasts are therefore not required to estimate the new postprocessing scheme.
The observables depend on the lead time τ of the forecast, as do the parameters α and β which determine the postprocessing correction for every lead time. This reflects the fact that the postprocessing problem is typically a non-stationary initial-value problem, since the initial conditions of the model Eqs. () and () are typically not chosen on their respective model attractor but rather as observations
Here we consider that the observation are perfectly assimilated in the models and that there is no observation errors. However in operational setups, such errors are of course to be taken into account.
of reality defined by Eq. (). As a consequence, the model averages of Eq. () relax toward the stationary response in the long-time limit, and the stationary response theory cannot provide us their short-time relaxation behaviour. Instead, the Ruelle time-dependent response theory should be used . It follows that, if the perturbation () is small, then the first order is given by (see Sect. )
δ〈A(τ)〉y=∫0τdτ′∫dx0ρ0(x0)Ψy(τ′)∇fτ′(x0)Afτ(x0),
where ρ0 is the distribution of the initial conditions (observations) used to initialise the models. ∇x is the gradient evaluated at the point x, and here it is the simple derivative. As indicated by Eq. (), in the postprocessing framework, ρ0 is taken as the stationary distribution of reality. As shown in Appendix , Eq. () can be obtained through a Kubo-type perturbative expansion . We remark that this example deals with stochastic models, due to which we have to perform an additional averaging over the realisations of the stochastic processes, denoted here as 〈⋅〉. Finally the mapping fτ which appears in Eq. () is the stochastic flow
fτ(x0)=x0e-λyτ+∫0τdτ′e-λy(τ-τ′)[Qyξy(τ′)+Ky].
This maps an initial condition x0 of model y to the state fτ(x0) of a realisation of this model at the later lead time τ. The principle of causality is thus implicit in Eq. (), which estimates the impact of the perturbation Ψy on the subsequent perturbed-model time evolution by developing around the unperturbed-model y trajectories.
Evaluating Eq. () and its stochastic integrals gives us the variation of the averages 〈y(τ)〉 and 〈y(τ)2〉 to the perturbation Ψy:
δ〈y(τ)〉y=-κ∫0τdτ′δKe-λy(τ-τ′)=-κλyδK1-e-λyτ,
δ〈y(τ)2〉y=-2κδK∫0τdτ′〈x0〉e-λy(2τ-τ′)+Kyλye-λy(τ-τ′)1-e-λyτ-2κδQQy∫0τdτ′e-2λy(τ-τ′)=-2κδKλyy(τ)1-e-λyτ-κλyδQQy1-e-2λyτ.
Rearranging these two terms, we also get the following expression for the variation of the variance given by Eq. ():
δσy2(τ)=-κλyδQQy1-e-2λyτ-κ2λy2δK21-e-λyτ2.
Note that the variation given by Eq. () corresponds exactly to the difference between the average of the two models 〈y^(τ)〉-〈y(τ)〉. On the other hand the variation given by Eq. () lacks the term of order κ2 involving δQ that appears in the difference between σy^2(τ) and σy2(τ) given respectively by Eqs. () and (). Instead, another term of order κ2 and involving δK is present, indicating that higher-order terms of response theory need to be considered to correct it . The second-order term is given by the expression
This expression is equivalent to the second term of Eq. (1) in upon a time transformation. It can also be obtained by explicitly computing the second-order perturbation of the average in Eq. () in Appendix .
:
δ2〈A(τ)〉y=∫0τdτ′∫τ′τdτ′′∫dyρ0(x0)Ψy(τ′)∇fτ′(x0)Ψy(τ′′)∇fτ′′(x0)Afτ(x0).
Applying this to the first moment of the y models directly yields
δ2〈y(τ)〉y=0.
On the other hand, integrating the stochastic integrals present in this expression for the moment 〈y(τ)2〉 gives
δ2〈y(τ)2〉y=κ2δK2λy21-e-λyτ2+κ2δQ22λy1-e-2λyτ,
which corrects the κ2δK2 term in Eq. () and makes the response theory up to order 2 exactly match the difference between σy^2(τ) and σy2(τ), for every lead time τ. In fact, the subsequent orders of the response vanish due to the linearity of the simple Ornstein–Uhlenbeck models, which enables us to truncate the response Kubo-like expansion to the second order. Finally, this shows that the (non-stationary) response theory can be used to estimate the postprocessing parameters after the model change based on the forecasts of the initial model. Indeed, instead of the averages 〈y^(τ)〉 and σy^2(τ), the approximate averages 〈y(τ)〉+δ〈y(τ)〉y and σy2(τ)+δσy2(τ)+δ2σy2(τ) can be used to compute α^ and β^. We emphasise that the second-order contribution had to be considered in order to obtain the exact result. Nevertheless, the difference between the first- and the second-order response is of the order κ2, which implies that for a small perturbation (model change), the first order will generally be a sufficiently good approximation. A more detailed derivation of the results obtained in this section can be found in the Supplement.
In order to investigate this research avenue on a case closer to those encountered in reality, we will now consider the application of postprocessing and response theory to a low-order atmospheric model displaying chaos.
Application to a low-order atmospheric model
A two-layer quasi-geostrophic atmospheric system on a β plane with an orography is considered . This spectral model possesses well-identified large-scale flow regimes, such as “zonal” and “blocked” regimes. The horizontal nondimensionalised coordinates are denoted as x and y, with the model's domain being defined by (0≤x≤2πn,0≤y≤π), with n=2Ly/Lx as the aspect ratio between its meridional and zonal extents Ly and Lx. The two main fields of this model are the 500 hPa pressure anomaly and temperature, which are proportional to the barotropic streamfunction ψ(x,y) and the baroclinic streamfunction θ(x,y), respectively. Both fields are defined in a zonally periodic channel with no-flux boundary conditions in the meridional direction (∂⋅/∂x≡0 at y=0,π). The fields are expanded in Fourier modes respecting these boundary conditions:
F1(x,y)=2cos(y),F2(x,y)=2cos(nx)sin(y),F3(x,y)=2sin(nx)sin(y),F4(x,y)=2cos(2y),⋮
such that
∇2Fi(x,y)=-ai2Fi(x,y)
with eigenvalues a12=1,a22=a32=1+n2,a42=4,…. We have thus the following decomposition
42ψ(x,y)=∑i=1naψiFi(x,y),43θ(x,y)=∑i=1naθiFi(x,y),
where na is the number of modes of the spectral expansion.
The partial differential equations controlling the time evolution of the fields ψ(x,y) and θ(x,y) can then be projected on the Fourier modes to finally give a set of ordinary differential equations for the coefficients ψi and θix˙=F(x),x=(ψ1,…,ψna,θ1,…,θna)
that can be solved with usual numerical integrators. All variables are nondimensionalised. The ordinary differential equations of the model are detailed in Appendix .
In the version proposed by using the 10 first modes beyond a certain value of the zonal temperature gradient, the system displays chaos and makes transitions between the blocked and zonal flow regimes embedded in its global attractor. Here, we use their main nondimensionalised parameters values: the friction at the interface between the two layers kd=0.1, the friction at the bottom surface kd′=0.01 and the aspect ratio of the domain n=1.3. The β plane lies at midlatitudes (50∘) and the Coriolis parameter f0 is set accordingly.
In the present work, the parameter hd, the Newtonian cooling coefficient is fixed to 0.3 instead of the value found in (which is hd=0.045). Two additional fields have to be specified on the domain: θ*(x,y), the radiative equilibrium temperature field, and h(x,y), the topographic height field. These fields can be decomposed by projecting them onto the eigenfunctions of the Laplacian as before. The corresponding coefficients θi* and hi then allow for writing these fields as sums of weighted eigenfunctions:
45θ*(x,y)=∑i=1naθi*Fi(x,y),46h(x,y)=∑i=1nahiFi(x,y).
In the present case, we consider that the only non-zero coefficients are θ1*=0.2 and h2=0.4, meaning that the radiative equilibrium profile is given by the zonally varying function 2cos(y) and the orography is made of a mountain and a valley shaped by the function 2cos(nx)sin(y). Again, the value of the temperature gradient θ1* is larger than the one chosen in (which is θ1*=0.1) to increase the chaotic variability in the system. Trajectories of variables θ1 and ψ2 are depicted in Fig. , for the reference system (reality) and a model version (model 0) for which the friction coefficient has been slightly modified.
Dynamics of the reference system and model 0 of the postprocessing experiment with a modification of the friction coefficient (see Table ) for (a) time evolution of the variable θ1 and (b) time evolution of the variable ψ2.
The main parameters used and modified in the experiments. Model 0 and model 1 are respectively the forecast model of reality before and after the model change.
These parameter changes induce slight modifications of the dynamics. In particular the system possesses two distinct weather regimes, depicted in Fig. a and b: one characterised by a zonal circulation (see Fig. c) and another characterised by a blocking situation (see Fig. d). In the former case, the variables ψ2 and ψ3 characterising the strength of the meridional anomalies are small, while in the latter case they are large, indicating indeed a blocking situation. This is different from the situation considered in , where two different blocking regimes coexist with the zonal regime.
Attractors for the experiment with a modification of the friction coefficient: (a) two-dimensional isodensity of the attractors estimated with a Gaussian kernel density estimator for the variables ψ2 and ψ3 and (b) two-dimensional scatterplot of the attractors for the variables ψ2 and ψ3. The attractors of reality and model 0 are qualitatively similar, with two different parts which are indicated by ellipses. The blue and red crosses correspond respectively to equilibrium points of the reference model (reality) and of model 0, respectively. The dashed ellipse corresponds on average to a zonal circulation depicted in panel (c). The dash-dotted ellipse corresponds on average to a blocking situation depicted in panel (d). In both panels (c) and (d), the underlying colour map denotes the orography in the domain, and the contours denote the geopotential height anomaly at 500 hPa.
Postprocessing experiments
The model described above with 10 modes (na=10) is used, and two different postprocessing experiments are performed, one involving the Newtonian cooling parameter hd and another involving the friction parameter kd between the two atmospheric layers. The parameter values detailed above correspond to the long-term reference (i.e. reality). A first model is defined (model 0) which is a copy of the two-layer quasi-geostrophic model defining reality, but the parameters hd or kd are slightly changed; i.e. the model error of the forecasting system lies in either the Newtonian cooling or the friction parameter. Then, as in Sect. , a model change is imposed, leading to another forecasting model (model 1) that can either improve or degrade the model error by a factor κ. The parameter variations involved in these experiments are detailed in Table . Without a loss of generality, we consider model changes that improve the representation of reality in the sense that the amplitude of the model error in model 1 is smaller than in model 0. The effect of the model change is depicted in Figs. and for the friction parameter experiment. These figures display the mean and the standard deviation of the model forecasts and observations coming from the reference forecasts, as a function of the lead time τ. We have used a set of 1 million trajectories of each system to compute these averages.
Behaviour of the averages as a function of the lead time τ in reality and the forecast models before (a) and after (b) the model change, for the experiment with a modification of the friction coefficient (see Table ). The variable considered is the temperature meridional gradient θ1. The solid lines denote the mean, while the shaded areas denote the interval of 1 standard deviation.
Same as Fig. but for the variable ψ3 of the streamfunction ψ.
In the framework of the EVMOS postprocessing scheme, the predictors and the predictands are the same nominal variable, and no other predictors are used. In both experiments considered, the postprocessing parameters α and β of the EVMOS for model 0, as well as α^ and β^ for model 1, are computed. The main objective here is then to estimate the difference between the former and the latter using Ruelle response theory. The approach in a multivariate setting is presented below.
Model change, response theory and the tangent linear model
Let us consider again the response theory described in Sect. but in the general multivariate deterministic case described in Sect. . In the postprocessing framework, models 0 and 1 evolve in time from a set of initial conditions taken outside of their respective attractors. Response formulas found in Ruelle's work have to be adapted to take this into account. One therefore has to consider the density of initial conditions as the measure. For a system with a time-independent perturbation Ψ(y^),
y^˙=F(y^)+Ψ(y^)=F^(y^),
an observable A with average 〈A(τ)〉y at the lead time τ for the system
y˙=F(y)
has a first-order response of
δ〈A(τ)〉y=∫dy0ρ0(y0)δy(τ)T⋅∇fτ(y0)A,
where fτ is the flow of the unperturbed system given by Eq. (), ρ0 is the distribution of initial conditions and δy(τ) is the solution of the equation y˙+δy˙=F^(y+δy), which can be approximated at the first order by the following linear inhomogeneous differential equation
δy˙=∇yF⋅δy+Ψ(y),
where y(τ) is the solution of the unperturbed Eq. () with the initial condition y(0)=y0, and we see that the systems of Eqs. () and () have to be integrated simultaneously . The homogeneous part of Eq. () is the well-known tangent linear model of the system, and here it has to be solved with an additional boundary term which is the perturbation itself.
Equation () is derived in Appendix and can be computed in the same way as the averages depicted in Figs. and , by averaging over multiple initial conditions of the reference system. Since we initialise the unperturbed (model 0) and perturbed systems (model 1) with the same initial conditions, the initial state of the tangent model defined by Eq. () is δy(0)=0. Therefore we do not estimate the impact of the observation or assimilation errors but rather the direct impact of the model errors viewed as time-independent perturbations. The formulation of the problem and Eq. () can be adapted to take these errors into account, as described for instance by .
In what follows, we will numerically integrate Eq. () to evaluate the response on the average due to the perturbation induced by the model change. This will in turn, as in Sect. , enable us to compute the postprocessing parameters for the new model.
Main results
For each of the two experiments detailed in Table , we start by obtaining 1 million observations of reality that will be used to initialise the forecast models. For each observation, this is done by starting model x (the reference) with a random initial condition and running it for a very long time (100 000 nondimensionalised time units) to achieve convergence to its global attractor. Once the observations have been obtained, we run the reference model, model 0 and model 1 over 200 time units (corresponding to roughly 22 d) to obtain reality and the forecasts. The systems have been integrated using the fourth-order Runge–Kutta integration scheme with a time step of 0.1 time units corresponding to 16.15 min. The averaging over the 1 million trajectories of reality and of the forecasts at each lead time is used to compute the postprocessing coefficients α and β of the EVMOS by using Eqs. () and (). For each predictand, the corresponding model variable is used as the unique predictor.
Corrections of the moments of θ1 from model 0 to model 1 using the response-theory formula of Eq. (), for the experiment with a modification of the friction coefficient.
Corrections of the moments of θ1 from model 0 to model 1 using the response-theory formula of Eq. (), for the experiment with a modification of the Newtonian cooling coefficient.
The response-theory approximations of the averages of model y^ (model 1) averages are obtained by integrating the linearised equations of model 0 along its trajectories with the perturbation Ψ as an inhomogeneous term. This is done by integrating Eq. () over a lead time of 200 time units with a zero initial condition, using the same integration scheme as before. It gives us the integrand of Eq. () for each trajectory, and the integral is then approximated as the average of this integrand over the whole set of trajectories. The result of this integration and averaging is shown in Figs. and for the first and second moment of the variable θ1. The results for other variables are available in the Supplement. The black curve shows the moments of model 0 with the addition of their linear response δ〈θ1〉 and δ〈θ12〉 to the perturbation Ψ. This curve agrees well with the green curves of the model 1 moments up to a lead time of 4–5 d, showing the efficiency of response theory. Note that in contrast to the calculation of the averages shown in Figs. and and computed with 1 million trajectories, we have here considered a limited subset of 10 000 trajectories of model 0 and its tangent to compute the corrections to these averages. The correction of the moments of model 1 are accurate until 4 d for both experiments. After this critical lead time, obtaining a good accuracy requires a huge increase in the number of forecasts and tangent model integrations to perform the averaging. This problem is well-known and is due to the appearance of fat tails in the distribution of the perturbations δy in the integrand of Eq. (). As it can be seen in Fig. for the perturbations on θ1, the problem worsens with the increase of the lead time: initially the distributions are near-Gaussian, and fat tails appear progressively. Therefore, the number of samples of δy needed to converge to the correct mean up to a certain precision increases exponentially as the lead time increases. This problem has consequences on the method used to perform the average. Indeed, to avoid rare and unrealistic extreme responses of the system located far in the tails of the distributions, outliers above a certain threshold (set to three nondimensional units) have been removed from the averaging.
Histograms of the solutions δθ1(τ) of Eq. () for the perturbation δy(τ) (with θ1 being the 11th component of y) along the trajectories of model 0, for different values of the lead time τ. The solid orange curves are fits of a Gaussian distribution function to the different histograms. The fat-tail phenomenon described in is apparent and becomes more prominent as the lead time increases.
Coefficients α and β of the postprocessing schemes of variable θ1 and their correction using the response theory, for the experiment with a modification of the Newtonian cooling coefficient.
Performance of the corrections on the variable θ1 for the experiment with the modification of the friction coefficient. (a) Mean square error (MSE) evolution between the different forecasts and their correction and reality. (b) Mean of the different trajectories (reality, model 0 and model 1) and corrected forecasts. (c) Variance of the different trajectories and corrected forecasts.
Performance of the corrections on the variable θ1 for the experiment with the modification of the Newtonian cooling coefficient. (a) Mean square error (MSE) evolution between the different forecasts and their correction and reality. (b) Mean of the different trajectories (reality, model 0 and model 1) and corrected forecasts. (c) Variance of the different trajectories and corrected forecasts.
The moments obtained by the response-theory approach are used to compute new EVMOS postprocessing α and β coefficients, thanks to Eqs. () and (). These corrected coefficients for variable θ1 are shown in Fig. for the experiment with a modification of the Newtonian cooling coefficient and in panels (c) and (d) of Fig. for the experiment with a modification of the friction coefficient. In Figs. and , we compare the performances of the four postprocessing schemes hence obtained: the postprocessing of model 0 (red curves) and model 1 (green curves) obtained by averaging over their trajectories (forecasts) and the postprocessing of model 1 obtained with the past model 0 forecasts (green + crosses) and with the response-theory approach (black × crosses). In panel (a) of Figs. 9 and 10, the mean square error (MSE) between the trajectories of the models and the reference trajectories is displayed by solid curves, while the MSE between both models correction and the reference is depicted by dash-dotted curves. The EVMOS postprocessing is able to partly correct the forecasts, reducing the MSE until a lead time of the order of a few times the model's Lyapunov time (the inverse of the leading Lyapunov exponent). After that, the MSE curves of the postprocessed and uncorrected forecasts converge toward a plateau corresponding to twice the variance of the reference solution . Here, the statistical postprocessing corrections are indeed efficient until lead times of 4–5 d, with a skill of the corrections decreasing with the lead time. Thus the EVMOS schemes do not become better than the original models after roughly 4 d. Note also that even if the model change is small, the postprocessing using the past forecasts of model 0 (green + crosses) completely fails to correct model 1 forecasts, highlighting the need for an adaptation of the postprocessing to the model change. In contrast, the adaptation with the response-theory method (black × crosses) produces valid corrections up to 4 d later. In panels (b) and (c) of Figs. and , the mean and variance of the corrected forecasts are compared with those of the original models. Again, the corrections obtained with response theory are efficient until 4 d for the postprocessing schemes.
In conclusion, the correction of model 1 using the response-theory EVMOS matches almost perfectly the score of the “exact” EVMOS obtained with the forecasts of model 1 (dash-dotted green curve), up to a 4 d lead time. After that lead time, the errors due to the fat tails in the response of the first moments of the statistics induce errors in the variance needed to compute the α and β coefficients (see Eqs. and ). These coefficients therefore degrade sharply after 4 d, as shown by the solid black curve in Fig. and in Fig. c and d. This in turn induces a degradation of the response-theory postprocessing scheme. Nevertheless, this limitation of response theory is not a concern here, since after a lead time of 4 d, the EVMOS skill improvement vanishes anyway.
Comparison of the efficiency of the response-theory correction for different numbers m of trajectories used to average Eq. (), for the experiment of varying the friction coefficient: (a) mean square error with reality, (b) absolute difference between the response-theory correction and the correction based on the forecast of model 1, and (c, d) postprocessing coefficients α and β. In panels (b), (c) and (d), the higher (100 000) and the lower (20) numbers are depicted respectively by a solid black line and a dashed red line. The other cases in between are depicted by dotted lines.
Discussion and conclusions
Statistical postprocessing techniques used to correct numerical weather predictions (NWP) require substantial past forecast and observation databases. In the case of a model change, which frequently occurs during the normal life cycle of an operational forecast model, one has to reforecast the entire database of past forecasts to update the postprocessing coefficients and parameters. In the present work, we proposed a new methodology based on response theory to produce these new coefficients without having to reforecast. Instead, the database of past forecasts is reused to perform integrations in the tangent space of the model. It allows us to obtain the new postprocessing coefficients as modifications of the old ones. These new coefficients were shown to be accurate enough within the lead-time range for which the postprocessing corrections improve the forecast.
Figure summarises the main results of this work, with the quasi-geostrophic system described in Sect. , using a different number m of trajectories of model 0 and its tangent model to compute the response-theory corrections. It shows that up to a lead time of 2 d, good postprocessing scheme coefficients are obtained even with a mere 20 integrations in the tangent space.
Note however that in the context of this conceptual model, good estimates of the postprocessing coefficients α and β can be obtained by simply using a small set of reforecasts. It is indeed enough to directly integrate the updated model 1, given by the non-linear Eq. (), with only 20 trajectories. So the response-theory approach in the present case cannot really compete with the simple reforecasting method. How this can be improved in an operational context is an important question that should be addressed in the future. For instance, we can use a simplified tangent linear model to reduce the computational burden, as is often used in data assimilation . This approach could also be implemented for short-range forecasts, say from 1 to 3 d.
The response theory is efficient because the model changes are assumed to be small in comparison with the original parameterisation of the models. The method cannot improve a postprocessing scheme, but it can efficiently adapt it to a new model version. As such, the success of this method also depends on the quality of the past postprocessing scheme. There are situations where linear response theory is known to fail, but statistical tests which allow for the identification of its breakdown have been derived in and in . In addition, the approach presented here applies only for models for which a tangent model is available. The model change itself has to be provided as an analytic function, which can in some circumstances limit the applicability of the approach.
To test this approach, we have focused on the EVMOS statistical postprocessing method, but other methods could be considered as well. The only requirement is that the outcome of the minimisation of the cost function uses averages of the systems being considered. For instance, member-by-member methods that correct both the mean square errors and the spread of the ensemble while preserving the spatial correlation could be considered. These methods generally use the covariance between the model forecasts and the observations as an important piece. Response theory can also be applied here, since this covariance can be written as an average. This will be investigated in a future work, together with the applicability of the approach to parameters of probability distributions, as is often used in meteorology .
The impact of initial-condition errors has not been addressed here, since the purpose was to demonstrate the applicability of the approach in a perfectly controlled environment. The main limiting issue of response theory in the present context is the presence of fat tails in the distribution of the perturbations δy in the tangent model. This implies that beyond a certain lead time, typically 2–3 d for the synoptic scale, the number of trajectories of the tangent model needed for the averages to converge increases exponentially. This renders the approach impractical at lead times beyond 2–3 d. This is a well-known problem, which is typically due to the trajectories passing close to the stable manifolds structuring the dynamics of chaotic systems , generating an extreme response of the system to the perturbations Ψ. This is possibly due to the exacerbated sensitivity of these manifolds to the perturbation of the system. We see two possibilities to overcome this issue in the case where a long lead-time correction is needed.
First, as suggested by , the problem should be studied in other systems. It might be resolved by itself in other systems. Indeed, in very large atmospheric systems, the encounter of such manifolds might become rare. This could be related to the chaotic hypothesis which states that large systems can be considered to behave like Axiom-A hyperbolic systems for the physical quantities of interest, and thus Ruelle response theory might get better as the dimensionality of a system increases. This hypothesis would be interesting to test in current state-of-the-art NWP systems.
Secondly, another avenue would be to adapt the techniques based on the covariant Lyapunov vectors (CLVs) or on unstable periodic orbits (UPOs) to non-stationary dynamics. These techniques were recently introduced to deal with stationary responses of chaotic systems, i.e. the response of a system that lies on its attractor.
The CLVs methods mentioned focus on finding an adjoint representation of the response, while in the present work the approach is based on forward integrations (direct method). The adjoint representation allows for easily changing the perturbation function Ψ for a fixed observable A, while the direct method enables the consideration of different observables while keeping the perturbation function fixed. The adjoint representation, however, requires one to integrate the tangent model backward in time. Therefore, its accuracy depends on the absolute value of the smallest Lyapunov exponent of the system, which might render its results less well than the direct forward representation.
In conclusion, the response-theory approach developed here is an effective method to deal with the problem of the impact of model change on the postprocessing scheme. Its main advantage is to be computed on the past model version and does not require reforecasts of the full model. Its operational implementation, however, is still an open question that should be addressed in the future.
Code availability
The quasi-geostrophic model used is called QGS and was obtained by adapting the Python code of the MAOOAM ocean–atmosphere model , following the model description in . It was recently released on Zenodo and is also available at https://github.com/Climdyn/qgs (last access: 18 May 2020). The additional notebooks computing the response to model changes and generating the figures are also provided in the Supplement. They have been released on Zenodo as well and are available at https://github.com/jodemaey/Postprocessing_and_response_theory_notebooks (last access: 18 May 2020).
Non-stationary response theory
We consider a perturbed autonomous dynamical system
y^˙=F(y^)+Ψ(y^)=F^(y^)
with a prescribed distribution of initial conditions ρ0.
For the unperturbed system
y˙=F(y),
an observable A has the average at time τ〈A(τ)〉y=∫dy0ρ0(y0)A(fτ(y0))=∫dyρτ(y)A(y),
where fτ is the flow of the unperturbed system given by Eq. () and where ρτ is the distribution obtained by propagating the initial distribution ρ0 with the Liouville equation . In this section, the variation of this average due to the presence of the perturbation is evaluated as
〈A(τ)〉y^=〈A(τ)〉y+δ〈A(τ)〉y+δ2〈A(τ)〉y+….
In other words, we compute the average of A in the system defined by Eq. ()
〈A(τ)〉y^=∫dy0ρ0(y0)A(f^τ(y0))
as a perturbation of the average given by Eq. () for the unperturbed system defined by Eq. (). Here, f^τ is the flow of the perturbed system defined by Eq. ().
In the following, we will derive these corrections thanks to a Kubo-type perturbative expansion that amounts to constructing a Dyson series in the interaction picture framework, where the perturbation is seen as an interaction Hamiltonian . We start by considering the time evolution of the observable A in Eq. ():
ddτAf^τ(y0)=L0+L1Af^τ(y0)
with the operators
L0A(y)=F(y)T⋅∇yAL1A(y)=Ψ(y)T⋅∇yA
and define an interaction observable as
AI(τ,y0)=Π0(-τ)Af^τ(y0)
with Π0(τ)=expL0τ. It is easy to show that the interaction observable satisfies the differential equation:
ddτAI(τ,y0)=LI(τ)AI(τ,y0)
with the interaction operator LI(τ)=Π0(-τ)L1Π0(τ). The solution to this equation is
AI(τ,y0)=AI(0,y0)+∫0τds1LI(s1)AI(s1,y0)=A(y0)+∫0τds1LI(s1)AI(s1,y0),
which can be rewritten as
Af^τ(y0)=Π0(τ)A(y0)+∫0τds1Π0(τ-s1)L1Π0(s1)AI(s1,y0).
Iteratively replacing the interaction observable by Eq. () finally leads to the Dyson series:
Af^τ(y0)=Π0(τ)A(y0)+∫0τds1Π0(τ-s1)L1Π0(s1)A(y0)+∫0τds1∫0s1ds2Π0(τ-s1)L1Π0(s1-s2)L1Π0(s2)A(y0)+….
Using the definitions in Eqs. () and (), as well as the fact that
gfτ(y0)=Π0(τ)g(y0)
for any smooth function g, we get finally a formula for the perturbations in Eq. ():
〈A(τ)〉y^=〈A(τ)〉y+∫0τds1∫dy0ρ0(y0)Π0(τ-s1)L1Π0(s1)A(y0)+….
We will now focus on the first term of this expansion, but the subsequent orders of the response can be treated in the same way. We thus have
δ〈A(τ)〉y=∫0τds1∫dy0ρ0(y0)Ψfτ-s1(y0)T⋅∇fτ-s1(y0)Afτ(y0),
which with the change of variable s1→t-τ′ can be rewritten as
δ〈A(τ)〉y=∫0τdτ′∫dy0ρ0(y0)Ψfτ′(y0)T⋅∇fτ′(y0)Afτ(y0)
and then
A17δ〈A(τ)〉y=∫0τdτ′∫dy0ρ0(y0)Ψfτ′(y0)T⋅∂fτ(y0)∂fτ′(y0)T⋅∇fτ(y0)AA18=∫0τdτ′∫dy0ρ0(y0)Ψfτ′(y0)T⋅Mτ-τ′,fτ′(y0)T⋅∇fτ(y0)A.M is the fundamental matrix of the homogeneous part of the linear differential equation
δy˙=∇yF⋅δy+Ψ(y),
where y is solution of Eq. () with initial condition y0, and we have the definition
M(t,y)=∂ft(y)∂y.
Equation () is the linearised approximation of Eq. ():
y˙+δy˙=F(y+δy)+Ψ(y+δy),
which provides a tool to estimate Eq. (). Indeed, since the solution of Eq. () can be written as
δy(τ)=∫0τdτ′Mτ-τ′,fτ′(y0)⋅Ψfτ′(y0),
we can write the first-order variation of the average of the observable A in terms of these solutions:
δ〈A(τ)〉y=∫dy0ρ0(y0)δy(τ)T⋅∇fτ(y0)A.
The interpretation of this equation is that a specific averaging of an observable over the trajectories of the linear approximation given by Eq. () of the perturbed Eq. () provides the first-order response of the observable. It is the main result used to compute the new postprocessing scheme in the present work. It is explained in detail in Sects. and .
The quasi-geostrophic model equations
The ordinary differential equations of the model are given by
B1ψ˙i=-ai,i-1∑j,m=1nabi,j,mψjψm+θjθm-ai,i-12∑j,m=1nagi,j,mhmψj-θj-βai,i-1∑j=1naci,jψj-kd2ψi-θi,B2θ˙i=-ai,i-1∑j,m=1nabi,j,m(ψjθm+θjψm)+ai,i-12∑j,m=1nagi,j,mhmψj-θj-βai,i-1∑j=1naci,jθj+kd2ψi-θi-2kd′θi+ai,i-1ωi,B3θ˙i=-∑j,m=1nagi,j,mψjθm+σ2ωi+hdθi*-θi,
where nondimensional parameters values and description can be found in Table and Sect. . β is the meridional gradient of Coriolis parameter which has the nondimensional value of 0.21 at 50∘ of latitude . The vertical velocity ωi can be eliminated, leading to Eqs. () and () being reduced to a single equation for θi. The parameter σ is the nondimensional static stability of the atmosphere set typically to 0.2. The coefficients ai,j, gi,j,m, bi,j,m and ci,j are the inner products of the Fourier modes Fi defined in Sect. :
B4ai,j=n2π2∫0π∫02π/nFi(x,y)∇2Fj(x,y)dxdy=-δijai2,B5gi,j,m=n2π2∫0π∫02π/nFi(x,y)JFj(x,y),Fm(x,y)dxdy,B6bi,j,m=n2π2∫0π∫02π/nFi(x,y)JFj(x,y),∇2Fm(x,y)dxdy,
ci,j=n2π2∫0π∫02π/nFi(x,y)∂∂xFj(x,y)dxdy,
where the coefficients ai are given by Eq. () and where J is the Jacobian present in the advection terms defined as
J(S,G)=∂S∂x∂G∂y-∂S∂y∂G∂x.
The supplement related to this article is available online at: https://doi.org/10.5194/npg-27-307-2020-supplement.
Author contributions
JD and SV developed the idea of using the response theory in the context of postprocessing, together with the overall experimental setup. JD made the analytical and numerical computations. Both authors contributed to the writing of the paper.
Competing interests
Stéphane Vannitsem is a member of the editorial board of the journal. Jonathan Demaeyer declares that he has no conflict of interest.
Special issue statement
This article is part of the special issue “Advances in post-processing and blending of deterministic and ensemble forecasts”. It is not associated with a conference.
Acknowledgements
The authors warmly thank Lesley De Cruz for her suggestions throughout the paper. They also thank Michaël Zamo and the anonymous reviewer for their comments and suggested improvements.
Financial support
This research has been supported by EUMETNET (Postprocessing module of the NWP Cooperation Programme).
Review statement
This paper was edited by Maxime Taillardat and reviewed by Michaël Zamo and one anonymous referee.
ReferencesBódai, T., Lucarini, V., and Lunkeit, F.: Can we use linear response theory
to assess geoengineering strategies?, Chaos: An Interdisciplinary Journal of
Nonlinear Science, 30, 023124, 10.1063/1.5122255, 2020.Bonavita, M., Trémolet, Y., Holm, E., Lang, S. T. K., Chrust, M.,
Janisková, M., Lopez, P., Laloyaux, P., De Rosnay, P., Fisher, M., Hamrud, M., and English, S.: A strategy for data assimilation, European Centre for Medium Range
Weather Forecasts, available at: https://www.ecmwf.int/sites/default/files/elibrary/2017/17179-strategy-data-assimilation.pdf (last access: 20 May 2020), 2017.
Cehelsky, P. and Tung, K. K.: Theories of multiple equilibria and weather
regimes – A critical reexamination. Part II: Baroclinic two-layer models,
J. Atmos. Sci., 44, 3282–3303, 1987.
Charney, J. G. and Straus, D. M.: Form-drag instability, multiple equilibria
and propagating planetary waves in baroclinic, orographically forced,
planetary wave systems, J. Atmos. Sci., 37, 1157–1176,
1980.De Cruz, L., Demaeyer, J., and Vannitsem, S.: The Modular Arbitrary-Order Ocean-Atmosphere Model: MAOOAM v1.0, Geosci. Model Dev., 9, 2793–2808, 10.5194/gmd-9-2793-2016, 2016.Demaeyer, J.: Postprocessing and response theory notebooks: version 0.1.1
release, Zenodo, 10.5281/zenodo.3755313, 2020.Demaeyer, J. and De Cruz, L.: qgs: version 0.1.0 release, Zenodo, 10.5281/zenodo.3716322, 2020.Demaeyer, J. and Vannitsem, S.: Comparison of stochastic parameterizations in the framework of a coupled ocean–atmosphere model, Nonlin. Processes Geophys., 25, 605–631, 10.5194/npg-25-605-2018, 2018.Eyink, G., Haine, T., and Lea, D.: Ruelle's linear response formula, ensemble
adjoint schemes and Lévy flights, Nonlinearity, 17, 1867, 10.1088/0951-7715/17/5/016, 2004.Gallavotti, G. and Cohen, E.: Dynamical ensembles in nonequilibrium statistical
mechanics, Phys. Rev. Lett., 74, 2694, 10.1103/PhysRevLett.74.2694, 1995a.
Gallavotti, G. and Cohen, E. G. D.: Dynamical ensembles in stationary states,
J. Stat. Phys., 80, 931–970, 1995b.
Gardiner, C. W.: Handbook of stochastic methods, fourth edn., Springer, Berlin, 2009.
Gaspard, P.: Chaos, scattering and statistical mechanics, vol. 9, Cambridge
University Press, Cambridge, UK, 2005.
Glahn, B., Peroutka, M., Wiedenfeld, J., Wagner, J., Zylstra, G., Schuknecht,
B., and Jackson, B.: MOS uncertainty estimates in an ensemble framework,
Mon. Weather Rev., 137, 246–268, 2009.
Glahn, H. R. and Lowry, D. A.: The use of model output statistics (MOS) in
objective weather forecasting, J. Appl. Meteorol., 11,
1203–1211, 1972.
Gottwald, G. A., Wormell, J., and Wouters, J.: On spurious detection of linear
response and misuse of the fluctuation–dissipation theorem in finite time
series, Physica D, 331, 89–101, 2016.
Hagedorn, R., Hamill, T. M., and Whitaker, J. S.: Probabilistic forecast
calibration using ECMWF and GFS ensemble reforecasts. Part I: Two-meter
temperatures, Mon. Weather Rev., 136, 2608–2619, 2008.
Hamill, T. M.: Practical aspects of statistical postprocessing, in: Statistical
Postprocessing of Ensemble Forecasts, edited by: Vannitsem, S., Wilks, D. S.,
and Messner, J. W., 187–217, Elsevier, Amsterdam, the Netherlands, 2018.
Hamill, T. M., Hagedorn, R., and Whitaker, J. S.: Probabilistic forecast
calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation,
Mon. Weather Rev., 136, 2620–2632, 2008.
Hamill, T. M., Bates, G. T., Whitaker, J. S., Murray, D. R., Fiorino, M.,
Galarneau Jr, T. J., Zhu, Y., and Lapenta, W.: NOAA's second-generation
global medium-range ensemble reforecast dataset, B. Am. Meteorol. Soc., 94, 1553–1565, 2013.
Johnson, C. and Bowler, N.: On the reliability and calibration of ensemble
forecasts, Mon. Weather Rev., 137, 1717–1720, 2009.
Kalnay, E.: Atmospheric modeling, data assimilation and predictability,
Cambridge University Press, Cambridge, UK, 2003.Lang, M. N., Lerch, S., Mayr, G. J., Simon, T., Stauffer, R., and Zeileis, A.: Remember the past: a comparison of time-adaptive training schemes for non-homogeneous regression, Nonlin. Processes Geophys., 27, 23–34, 10.5194/npg-27-23-2020, 2020.
Lasagna, D.: Sensitivity and stability of long periodic orbits of chaotic
systems, ArXiv [Preprint], arXiv:1910.06706, 2019.
Lasagna, D., Sharma, A., and Meyers, J.: Periodic shadowing sensitivity
analysis of chaotic systems, J. Comput. Phys., 391, 119–141,
2019.
Lembo, V., Lucarini, V., and Ragone, F.: Beyond Forcing Scenarios: Predicting
Climate Change through Response Operators in a Coupled General Circulation
Model, ArXiv [Preprint], arXiv:1912.03996, 2019.
Lucarini, V.: Response theory for equilibrium and non-equilibrium statistical
mechanics: Causality and generalized Kramers-Kronig relations, J. Stat. Phys., 131, 543–558, 2008.
Lucarini, V.: Evidence of dispersion relations for the nonlinear response of
the Lorenz 63 system, J. Stat. Phys., 134, 381–400, 2009.
Lucarini, V.: Stochastic perturbations to dynamical systems: a response theory
approach, J. Stat. Phys., 146, 774–786, 2012.
Ni, A.: Hyperbolicity, shadowing directions and sensitivity analysis of a
turbulent three-dimensional flow, J. Fluid Mech., 863, 644–669,
2019.
Ni, A. and Wang, Q.: Sensitivity analysis on chaotic dynamical systems by
Non-Intrusive Least Squares Shadowing (NILSS), J. Comput. Phys., 347, 56–77, 2017.
Nicolis, C.: Dynamics of model error: Some generic features, J. Atmos. Sci., 60, 2208–2218, 2003.
Nicolis, C.: Error dynamics in extended-range forecasts, Q. J. Roy. Meteor. Soc., 142, 1222–1231, 2016.
Nicolis, C., Perdigao, R. A., and Vannitsem, S.: Dynamics of prediction errors
under the combined effect of initial condition and model errors, J. Atmos. Sci., 66, 766–778, 2009.
Reinhold, B. B. and Pierrehumbert, R. T.: Dynamics of weather regimes:
Quasi-stationary waves and blocking, Mon. Weather Rev., 110, 1105–1145,
1982.
Ruelle, D.: General linear response formula in statistical mechanics, and the
fluctuation-dissipation theorem far from equilibrium, Pys. Lett. A, 245,
220–224, 1998a.Ruelle, D.: Nonequilibrium statistical mechanics near equilibrium: computing
higher-order terms, Nonlinearity, 11, 5, 10.1088/0951-7715/11/1/002, 1998b.Ruelle, D.: A review of linear response theory for general differentiable
dynamical systems, Nonlinearity, 22, 855, 10.1088/0951-7715/22/4/009, 2009.
Scheuerer, M. and Hamill, T. M.: Statistical postprocessing of ensemble
precipitation forecasts by fitting censored, shifted gamma distributions,
Mon. Weather Rev., 143, 4578–4596, 2015.
Van Schaeybroeck, B. and Vannitsem, S.: Ensemble post-processing using
member-by-member approaches: theoretical aspects, Q. J. Roy. Meteor. Soc., 141, 807–818, 2015.
Vannitsem, S.: A unified linear model output statistics scheme for both
deterministic and ensemble forecasts, Q. J. Roy. Meteor. Soc., 135, 1801–1815, 2009.Vannitsem, S.: Bias correction and post-processing under climate change, Nonlin. Processes Geophys., 18, 911–924, 10.5194/npg-18-911-2011, 2011.Vannitsem, S.: Predictability of large-scale atmospheric motions: Lyapunov
exponents and error dynamics, Chaos: An Interdisciplinary Journal of
Nonlinear Science, 27, 032101, 10.1063/1.4979042, 2017.
Vannitsem, S. and Nicolis, C.: Dynamical properties of model output statistics
forecasts, Mon. Weather Rev., 136, 405–419, 2008.
Vannitsem, S., Wilks, D. S., and Messner, J.: Statistical postprocessing of
ensemble forecasts, Elsevier, Amsterdam, the Netherlands, 2018.
Vissio, G. and Lucarini, V.: A proof of concept for scale-adaptive
parametrizations: the case of the Lorenz'96 model, Q. J. Roy. Meteor. Soc., 144, 63–75, 2018.
Wang, Q.: Forward and adjoint sensitivity computation of chaotic dynamical
systems, J. Comput. Phys., 235, 1–13, 2013.
Wilks, D. S.: Statistical methods in the atmospheric sciences, vol. 100,
Academic press, Oxford, UK, 2011.
Wormell, C. L. and Gottwald, G. A.: On the validity of linear response theory
in high-dimensional deterministic dynamical systems, J. Stat. Phys., 172, 1479–1498, 2018.Wouters, J. and Lucarini, V.: Disentangling multi-level systems: averaging,
correlations and memory, J. Stat. Mech.-Theory E., 2012, P03003, 10.1088/1742-5468/2012/03/P03003, 2012.
Young, L.-S.: What are SRB measures, and which dynamical systems have them?,
J. Stat. Phys., 108, 733–754, 2002.