The dynamics of several natural systems, including the atmosphere and the ocean, are characterized by chaotic conditions which, roughly speaking, describe the property that a system has sensitivity to initial states. This means that, even in the presence of a perfect model, small errors in the initial conditions will grow in size with time, until the forecast becomes de facto useless

In the words of Ed Lorenz, “Chaos: When the present determines the future, but the approximate present does not approximately determine the future”; see https://tinyurl.com/faf3pnda (last access: 17 December 2021).

The properties of the dynamical models have large implications on data assimilation DA;. Data assimilation refers to the family of theoretical and numerical methods that optimally combine data with a dynamical model with the goal of improving the understanding of the phenomenon under study, enhancing the prediction skill, and quantifying the associated uncertainty. Data assimilation has long been studied and developed in the geosciences. It is an unavoidable piece of the operational numerical weather prediction workflow, but it is nowadays used in a growing range of scientific areas .

Numerical and analytic evidence has emerged recently showing that under certain observational conditions (data types, spatio-temporal distribution, and accuracy), the performance of DA with chaotic dynamics relates directly to the instability properties of the dynamical model where data are assimilated. One can thus in principle use the knowledge of the dynamical features to inform not only the design of the DA that better suits the specific application – e.g. how many model realizations for the Monte Carlo based DA methods, or the length of the assimilation window in variational DA – but also the best possible observational deployment.

A stream of research has shed light on the mechanisms driving the response of the ensemble-based DA , i.e. its functioning and suitability, when applied to chaotic systems. A recent comprehensive review can be found in , while we succinctly recall the main findings in the following. In the deterministic linear and Gaussian case with a Kalman filter (KF) and smoother (KS), it has been analytically proved that the error covariance matrices converge in time onto the model's unstable–neutral subspace, i.e. the span of the backward Lyapunov vectors (BLVs) or of the covariant Lyapunov vectors (CLVs), associated with the non-negative LEs . These results have been shown numerically to hold for the ensemble Kalman filter/smoother in weakly non-linear regimes EnKF/EnKS; by . In practice, for sufficiently well observed scenarios, the error of the state estimate is fully confined within the unstable–neutral subspace. Because this subspace is usually much smaller than the full system's phase space, the above convergence results imply that an ensemble size as large as the unstable–neutral subspace dimension, n0, suffices to achieve satisfactorily performance, i.e. to track the “truth” and effectively reduce the estimation error along with a substantial computational saving. The impact of instabilities on non-linear DA, in particular particle filters PFs, see e.g., has been recently elucidated in : the number of particles needed to reach convergence depends on the size of the unstable–neutral subspace rather than the observation vector size.

The picture above slightly changes in the presence of a degenerate spectrum of LEs, which often arises in systems with multiple scales, associated with the presence of coupling between subsystems with different characteristic dynamical timescales . The degeneracy is usually concentrated on the unstable–neutral portion of the LE spectrum. In these cases it is necessary to increase the ensemble size to account for all of the degenerate modes . The necessity for going beyond the number of asymptotic unstable–neutral modes is also connected to the local (in phase space) variability of the degree of instability of the system . The large heterogeneity of the atmospherics's predictability is due to the presence of substantial variability in the number of unstable dimensions of the unstable periodic orbits (UPOs) populating the attractor and defining the skeletal dynamics of the system . As a result of the fact that the orbit of a chaotic system shadows the UPOs supported on the attractor in some of its regions, certain directions of the stable space experience finite-time error growth due to locally important instabilities, causing the need for a larger ensemble size than the number of the unstable-neutral modes.

In the stochastic scenario, noise is usually injected irrespective of the flow-dependent modes of instabilities. Consequently, with a non-zero probability, error is also injected onto stable directions that would not have been otherwise influential in the long term. The trade-off between the frequency of the noise injection and its amplitude on the one hand, and the dissipation rate of stable modes on the other, determines the amplitude of the long-term error along stable modes . This mechanism implies the need to include additional members in the ensemble to encompass weakly stable modes that experience instantaneous growth .

The knowledge of the LEs and its associated Lyapunov vectors (LVs) can be used to operate key choices in the implementation of ensemble-based DA schemes aimed at enhancing accuracy with the smallest possible computational cost. This point of view is at the core of DA algorithms that operates a reduction in the dimension of the model e.g. the assimilation in the unstable subspace, AUS;, of the data , or both .

While extremely theoretically appealing and practically useful in low-to-moderate dimensional problems, the use of the dynamically informed DA approaches is difficult in high dimensions, where even just computing the asymptotic spectrum of LEs, let alone the very relevant state-dependent local LEs (LLEs), is very difficult or just impossible. A major but not exclusive issue is that LE estimation algorithms require computation of the tangent space of the dynamical system, a task usually unfeasible for high-dimensional systems, or impossible when the model equations of are not explicitly accessible. On the other hand, the existence of a relationship between the DA and the unstable–neutral subspace suggests reversal of the viewpoint: use DA as a tool for estimating the properties of a given system that would be otherwise very difficult to compute. As a model-agnostic technique, DA, and in particular ensemble-based methods such as the EnKF, can be applied to any model without the need of computing the tangent space. This makes the EnKF a potentially powerful instrument to reveal the stability properties of a dynamical system. This is the goal of this work. Specifically, we shall investigate whether we can use DA to infer the spectrum of the LEs and the Kolmogorov–Sinai entropy (σKS) of the system whereby data are assimilated.

The paper is structured as follows. In Sect. , an upper bound of the root mean squared error of the Kalman filter for the linear dynamics in the asymptotic limit is derived. In Sect.  we present the (VL20) model and its DA setup. The VL20 model is a recently proposed generalization of the Lorenz '96 model that is able to describe in a simple yet meaningful way the interplay between dynamical and thermodynamical variables. Additionally, the presence of a qualitatively distinct set of spatially extended variables allows one to consider non-trivial cases of partial observations for DA exercises. Section  presents the main results of the paper by comparing the skill of the performed DA exercises with some fundamental measures of instability of the VL20 model. Finally, in Sect.  we discuss our results and present perspectives for future investigations.

Our test bed for numerical experiments is the low-order model recently developed by , hereafter referred to as the VL20. The VL20 model is an extension of the classical Lorenz 96 model that includes additional thermodynamic variables. The model is given by the following set of n ordinary differential equations (ODEs) (with n being an even integer): 27dXidt=Xi-1(Xi+1-Xi-2)-αθi-γXi+F28dθidt=Xi+1θi+2-Xi-1θi-2+αXi-γθi+G, where X represents the momentum, θ is the thermodynamic variable, and the subscript 1≤i≤n/2 is the grid point index. The model is spatially periodic, and the boundary condition is expressed as 29Xi-n/2=Xi+n/2=Xi30θi-n/2=θi+n/2=θi.

In the VL20 model it is possible to introduce a notion of kinetic energy K=∑=1n/2Xj2/2 and potential energy P=∑=n2nθj2/2. Additionally, the model features an energy cycle that allows for the conversion between the kinetic and potential forms and for introducing a notion of efficiency. The parameter α modulates the energy transfer between the two forms, while γ controls the energy dissipation rate, and F and G are external forcing defining the energy injection into the system. The model's evolution can be written as the sum of a quasi-symplectic term, which conserves the total energy, and of a gradient term, which describes the impact of forcing and dissipation. In the turbulent regime, the VL20 allows for propagation of signals in the form of wave-like disturbances associated with unstable waves exchanging energy in both potential and kinetic form with the background. In terms of energetics, the difference between the L96 and the VL20 model mirror the one between a one-layer and a two-layer quasi-geostrophic model because the former features only barotropic processes, while the latter features the coupling between dynamical and thermodynamic processes via baroclinic conversion, which makes its dynamics much more complex . The VL20 model is thus a very good test bed for research in DA, a further step toward realism from the very successful L96. Further details on the model as well as an extensive analysis of its dynamical and statistical properties can be found in .

In all the following experiments, we set n=36, implying both model variables X and θ have 18 components, and consider three model configurations differing in the values of the external forcings: F=G=10, F=10,G=0, and F=0,G=10. Unless otherwise stated, the model runs with the default parameters α=γ=1, and it is numerically integrated using the standard fourth-order Runge–Kutta time stepping method with a time step Δt=0.05 time units. A summary of the model instability properties with the chosen configurations is given in Table .

Synthetic observations are generated according to Eq. () by sampling a “true” solution of the VL20 model, Eq. (), and then adding simulated observational error from the Gaussian distribution N(0,R). Observational error is assumed to be spatially uncorrelated so that the error covariance, R, is a diagonal matrix, and we observe the model components directly, implying that the observation operator is linear and under the form of a matrix, H∈Rp×n. The observation error variance is set to be 5% of the variance (i.e. the squared temporal variability) of the climatology of the corresponding state vector component such that 31diag(R)i=5%Var(X),i=1,…,n2;32diag(R)i=5%Var(θ),i=n2+1,…,n. Linking the observation error to the model variance makes the setup more realistic, but it ties the error amplitude to the choice of the model parameters. For example, the model's state vector variance gets very small when the dissipation is strong, potentially making the R matrix degenerate. Under such circumstances, the corresponding entries in R are set to 5×10-6.

In line with previous studies e.g., we work with deterministic EnKFs, whereby it is possible to study the filter performance in relation to the model instabilities without the inclusion of additional noise that is inherent to stochastic versions of the EnKFs . In particular we choose to use the finite-size ensemble Kalman filter EnKF-N; because it automatically computes the required covariance inflation, thus saving us from running many inflation tuning experiments. The initial conditions for the ensemble are sampled from the Gaussian distribution N(x0t,R), with the x0t being the “truth” at t0: this choice signifies that the initial condition error is taken to be equal to the observational error.

The performance of DA experiments will be assessed primarily using the root mean square error of the analysis, normalized by the observation variance: 33nRMSEa=1n∑i=1n/2(Xi-Xi,truth)2diag(R)i+∑i=1n/2(θi-θi,truth)2diag(R)i+n/2. The nRMSEa measures the analysis error independent from observation error, allowing for a multivariate assessment of the performance. Unless otherwise stated, observations are taken at every time step, and the experiments last 2000 model time units. With this setting, an experiment comprises 40000 DA cycles, and when computing time averages of the nRMSEa, we only consider the last 500 model time units. Finally, and again unless otherwise stated, we shall adopt N=40 ensemble members in the EnKF-N.

Figure  shows the time series of the nRMSEa over the first 100 time units, for the three main model configurations under consideration. In all cases, the error drops to below 20% of the observational error after approximately 10 time units (corresponding to 200 DA cycles) and then fluctuates with oscillations that only sporadically lead the error to exceed 0.3. The configuration, (F,G)=(10,0) (red line), attains the smaller error, while the other two configurations (blue and purple lines respectively) show comparable error levels slightly larger than configuration (F,G)=(10,0). Recall that in the configuration of (F,G)=(10,0), the model is not thermodynamically forced (G=0), and is also slightly stabler than in the other two configurations (cf. Table ).

The first connection between the filter performance and the model instabilities is drawn from Fig.  that shows the nRMSEa as a function of the number of the ensemble members. In line with previous findings for uncoupled univariate and with coupled models , Fig.  shows that, even with a multivariate model, the error converges to very low values as soon as the ensemble size exceeds the number of unstable–neutral modes, n0, and that it does not further decrease by adding more members. This behaviour is possible because error evolution is bounded to be linear or weakly non-linear. This means that one can in principle induce linearity intentionally in the error evolution to meet the aforementioned relation between filter accuracy and ensemble size and use it to infer the number of unstable–neutral modes. In a DA experiment, a “practical” way to achieve this is by strengthening the observational constraint (i.e. by increasing the measurements spatial and temporal density); here we observe the full system's state at every time step.

As mentioned above, the VL20 model represents four main physical mechanisms: (i) conversion between kinetic and potential energy, (ii) the energy injection from external forcing, (iii) the advection, and (iv) the dissipation. Although these processes all participate in the evolution of the model, the non-linear interplay cannot be straightforwardly disentangled. Nevertheless, we shall try to refer to them when interpreting the outcome of the DA experiments. In particular, in each experiment we will attempt to identify the prevailing mechanism over the aforementioned four. We perform three experiments, where we observe the full system state (i.e. H=I36), or alternatively X or θ alone (implying in both cases H∈R18×36). Results are given in Fig. , that displays the time-averaged nRMSEa (global or for the dynamics or thermodynamics only) over a range of the coefficient α that modulates the energy transfer rate.

Overall, and as expected, the analysis error is smaller in the observed variables (cf. the left and middle columns and corresponding colour lines) and attains the smallest level when X and θ are simultaneously observed (right column). Nevertheless, a few remarkable points can be raised. First, when the system is fully observed, for large values of α (i.e. for large conversion between available potential energy and kinetic energy) the skills in X and θ become very similar (right column): we conjecture this to be a consequence of the system getting more evenly turbulent with all variables sharing a similar internal variability as energy is exchanged efficiently between the kinetic and potential form. Second, for small values of α (i.e. small energy conversion), the effect of external forcing becomes dominant and determines the analysis error of X and θ (last column in Fig. ). For instance, whenever the momentum is externally forced (F=10), the error in X is systematically smaller than in θ (first and second rows of the last column): DA is more effective in controlling the dynamics than the thermodynamics, even when they are subject to the same observational constraint. The situation is somehow reversed when only the thermodynamical variables are forced (F=0,G=10): the analysis error of the momentum and the thermodynamic variable is undifferentiated. With small values of α and no forcing for the momentum, the non-linear momentum advection is limited by the small magnitude of the momentum that is not able to activate much the dynamical variables, so that we observe similar analysis error between the thermodynamics variable and the momentum.

Finally, the effect of the energy transfer and advection can be revealed by looking at the partially observed experiments (left and middle columns). Both mechanisms involve the momentum, making it more efficacious to observe X than θ, especially in the energy-transfer-dominated regime (large α). However, in an advection-dominated regime (small α), if θ is unobserved, X has limited capability to constrain the error in θ due to the weak feedback from θ to X. On the other hand, observing θ reduces error in X via the accurate estimate of the advection process of θ (see middle column).

Further insight into the role of the driving (unstable) variable and on the interplay between the prevailing physical mechanisms and the analysis error is given by looking at the CLVs . In Fig.  we show at (normalized time-averaged) absolute amplitude of CLVs components along the state vector: it tells us which variable type/component has the larger influence on each CLV, thus indicating what processes participate more in a specific direction of error growth/decay.

As discussed above, changes in the value of α lead to shifting between the advection-dominated regime and an energy mixing one: these two regimes are portrayed in Fig. , by selecting α=0.4 and α=2.2. Moreover, these two values of α correspond roughly to those giving the largest differences in nRMSEa between the momentum and the thermodynamic variables (cf. left and middle panels of Fig. ). For small energy exchange (α=0.4 – left column in Fig. ), the model instabilities are driven by the external forcing, with the driving variable being the one where energy is injected. This is clearly visible when comparing the amplitudes of CLVs between X and θ in the left panel of Fig. : larger amplitudes of the unstable–neutral CLVs are found in the forced variables. When the momentum and the thermodynamics are equally forced (blue lines), the amplitude of the unstable–neutral CLVs for X and θ is close to each other. The non-linear advection process intensifies the error growth, especially for X. The non-linear advection and the momentum are of lesser importance if the momentum is not forced (F=0,G=10 – purple lines), while the thermodynamic processes control both the stable and unstable subspace dominantly. The thermodynamic variable on the stable subspace acts as an energy sink to stabilize the dynamical system. The effect of the thermodynamics is shown noticeably by the large relative amplitude of the CLVs of the thermodynamic variable in the stable subspace when the momentum is directly forced (F=10 – blue and red line).

The behavior changes substantially when the energy exchange is the dominant physical mechanism (α=2.2 – right column). This causes a stronger mixing across the model variables so that both X and θ play a comparable role in the unstable–neutral components of the CLVs, leading to similar amplitude of the CLVs for all types of forcing. Remarkably, the effect of the energy conversion also applies to the stable components of the CLVs, leading to similar amplitude of the CLVs between X and θ.

The results in Fig.  reveal the effect of the prevailing physical mechanisms on determining the driving unstable variables. The figure suggests what variables should in principle be controlled by targeting measurements on the portion of the system's state vector with larger amplitude on the unstable–neutral CLVs.

Along with α, the energy in the system is modulated by the dissipation parameter, γ: larger values of γ imply an efficient removal of energy from the system, thus reducing the system's variability of both the potential and kinetic energy. At dynamical level, the parameter γ controls the contraction of the phase space as the sum of all Lyapunov exponents (equal to the average flow divergence) is -nγ. Hence, one expects that larger values of γ correspond to weaker instability for the model, as in the case of the classical L96 model . Figure  is the same as Fig.  but for the dissipation, γ. Overall, we see that, with large γ, the system's internal variability reduces, and we find similar small errors in both X and θ. For weaker dissipation, the momentum is better controlled than the thermodynamics. With partial observations (left and middle columns), the error is much larger than in the corresponding fully observed cases. Similar to Fig. , the momentum is generally better reconstructed by the DA than the thermodynamics, although observing the latter appears more efficacious (i.e. it leads to smaller analysis error) than observing the momentum. We think that this is due to the prevailing mechanism being the advection of the thermodynamics given that α=1 in these experiments (cf. also Fig. ). The amplitudes of the CLVs along the state vector are studied in Fig. . We consider the cases γ=0.4 and γ=1.0 for which the difference in the nRMSEa between X and θ is roughly the largest (cf. Fig. ).

With the leading CLVs strongly affected by the external forcing, the amplitude of the CLVs along the system's components is similar to the pattern of low energy exchange rate in Fig.  where α=0.4, even though here, α=1. This confirms that the dynamical regime of our experiments lies in the regime dominated by advection, and dissipation does not mix the kinetic and potential energy diffusely as the energy exchange, but rather it uniformly removes both types of energy without changing the prevailing physical mechanism. This is also reflected in the consistently low nRMSEa for the observed variable when varying dissipation rates in the partially observed experiments (see Fig. ). The decreasing analysis error in Fig.  corresponds to the increases of γ, which reduces the dimension of the unstable–neutral subspace with increased relative importance of forced variables in the unstable–neutral subspace as the fast energy removal reduce the amount of energy mixing.

The results of Sect.  confirm the relation between the performance of DA (in terms of analysis error) and the dimension and characteristics of the unstable–neutral subspace. In particular, we conclude that successful DA relies on controlling the error in the unstable–neutral subspace by observing the variable that drives the error growth. The VL20 model enabled the investigation of the relation between the DA and the specific physical mechanisms such as the advection, the energy transfer among dynamics and thermodynamics, and the dissipation. The effect of DA (i.e. its efficacy) is strongly influenced by the form of the coupling between the unobserved and the observed variables that is in turn shaped by the prevailing physical mechanisms.

The derivation in Sect.  shows that, in the linear setting, the assimilation error is asymptotically bounded from above by a factor dependent on the observation error, the first LE and the number of unstable–neutral modes of the underlying forecast model. In this section we explore the extent to which this result holds in a non-linear scenario whereby the observational constraint is strong enough such that the error evolution is maintained approximately linear or weakly non-linear. We shall make use of numerical experiments with the VL20 model.

A first insight on the existence of a direct relation between the model instabilities and the skill of the EnKF-N is already provided in Figs.  and . They display the Kolmogorov–Sinai entropy, σKS (black line), and the first LE, λ1 (amplified by a factor of 3 – grey line), along with the nRMSEa (discussed in Sect. ). Even just by visual inspection, the figures clearly show the high correlation between the analysis error and both the σKS and λ1.

The nature of this relation is further studied in Fig. , which shows scatter plots between the nRMSEa (with black markers) and σKS/λ1 in a log–log scale. Data points relative to experiments with forcing values are given in the panels' legends and with varying energy exchange and dissipation rates in the range (α×γ)∈[0.1,3)×[0.3,1.8). Here, the EnKF-N assimilates the full state vector at each time step. The analysis error appears in a linear relationship with either σKS or λ1, as long as ln⁡(nRMSEa)≥-4. The existence of such an approximate relationship provides the possibility to infer σKS and/or λ1 based on the outcome of DA.

The scatter plots also demonstrate the validity of the upper bound (red markers) of Eq. () in Sect. . To compute the bound we set the coefficient related to observation, β=1, as it is compared to analysis errors normalized by observational error. The nRMSEa is bounded by the theoretical upper bounds for most of the model configurations considered. The linear relationship of the upper bound can be explained by its formulation in Eq. (), where the exponent e2λ1Δt-1 can be approximated as 2λ1Δt if 2λ1Δt is sufficiently small. The spread of upper bounds points for given λ1 values (left panel) reflects the various values of n0 under similar λ1 values. The better correspondence (narrower spread of the scattered points) in the plane nRMSEa with σKS (right panel) shows the importance of including both the dominant error growth rate, λ1, and the unstable-subspace dimension, n0, – both present in σKS – to better characterize the system's instabilities. The correspondence between σKS and the theoretical upper bound could also be a result of the relation between λ1 and σKS as in a highly turbulent case, there is a linear relation between λ1 and σKS .

The linear relation does not hold for numerical experiments when ln⁡(nRMSEa)<-4 (see the black markers' distribution in the panels' inset). We explain this behaviour in the following way. The wide clouds of points correspond to all model configurations with very small σKS and λ1. In these quasi-stable dynamics, the error growth in between successive analysis is very little, with occasional error decay. The observational error, which is random and white in time, will be often larger than the forecast error and will dominate the analysis error, thus breaking its direct dependence on the instability-driven forecast error. In addition, in the weakly unstable model configurations, the most unstable direction behaves almost like a neutral mode, which also breaks the assumptions of the theoretical upper bound for unstable models. In fact, we do not show the weakly unstable results with σKS<1 for both σKS and λ1.

The error bounds in Sect.  rely on the assumption of linear error evolution, a condition that we met in our experiments thanks to a strong observational constraint, with (synthetic) measurements covering the full state vector at each time step. These conditions are rarely achievable in practice, so it is relevant to explore how results will change with a lighter observational constraint. There are three direct ways to achieve this by acting on (i) the number/type of measurements, (ii) the measurement error, and/or (iii) the temporal frequency.

The effect of the first is studied in Fig.  that is similar to Fig.  but for DA experiments whereby only one of each variable in the VL20 is observed.

The impact of partially observing the system causes the emergence of a weakly quadratic relationship between the analysis error and either σKS or λ1. However, the analysis error is still uniquely and monotonically related to them, especially for σKS. A quadratic law requires one additional coefficient to be determined compared to a linear law; yet the mere existence of such a law suggests again that one could in principle infer σKS and/or λ1 based on the analysis error. With the relaxed observation constraint, the analysis error can (and indeed do so in several instances) exceed the theoretical upper bound. However, the general trend of the numerical experiments still follows the theoretical upper bound.

We study the effect of changing the amplitude of the observational error in Fig. . Results reveal that varying the observation error in the range of 5 %–10 % does not break the quasi-linear relationship between the analysis error and σKS or λ1. The nRMSEa is quite insensitive to the observation variance due to the normalization. Nevertheless, the upper bound limit is not violated as in Fig. , and the slope of the nRMSEa from the numerical experiment is remarkably similar to the slope of the theoretical bound.

Finally, the impact of varying the observation frequency is explored in Fig. . It is patent that decreasing the frequency leads to blurring the linear relation between the analysis error and the σKS or λ1. There is a clear deviation from the trend of the theoretical upper bound and from the uniqueness of the relation between analysis error and σKS, as soon as the observational time interval exceeds the error doubling time (that is inverse related to λ1), and DA error evolves beyond the linear regime. However, for frequent enough observations, a linear relation similar to the upper bound appears, and, again, one could in principle deduce σKS and/or λ1 based on DA.

The larger sensitivity to the observation frequency than to observation noise (cf. Figs.  and ) is a direct consequence of the different effects these two factors have in determining the degree of non-linearity of the error. This is explained, for the L96 model, in the Appendix of using a dimensional analysis. The key point is that the observation frequency modulates directly the magnitude of the non-linear term in the model, namely the advection. Decreasing the observation noise, while effectively reducing the analysis error, is not sufficient to keep the error dynamics linear if Δt>0 and the model is chaotic.