Let Nx∈N. We consider the dynamics governed by 3ddtx(t)=f(x(t)),t>0 with x(0)=x0∈RNx, where f:RNx→RNx is a smooth vector field. To study the instability of the dynamics, we examine the evolution of a perturbation δx(t)∈RNx, defined as the difference between two trajectories separated by δx0∈RNx at t=0. Assuming that δx(t) is sufficiently small and smooth, its evolution is approximated by the linearization of Eq. (): 4ddtδx(t)=Jf(x(t))δx(t),t>0 with δx(0)=δx0, where Jf(x(t)) denotes the Jacobian matrix of f at x=x(t). Equation () is referred to as the tangent linear model. Let Φ(t,x0)∈RNx×Nx denote the fundamental matrix solution to Eq. () with Φ(0,x0)=INx. The unique solution is then 5δx(t)=Φ(t,x0)δx0,t≥0. The matrix Φ(t,x0) encodes the deformation and amplification of infinitesimally small perturbations. For the singular values σ1(Φ(t,x0))≥σ2(Φ(t,x0))≥…≥σNx(Φ(t,x0))>0 of Φ(t,x0), we define 6λj(t,x0)=1tlog⁡σj(Φ(t,x0))∈R,j=1,…,Nx. For each j, the singular vector vj∈RNx associated with σj exponentially grows or decays at rate λj over [0,t] under the tangent linear model, i.e., 7‖Φ(t,x0)vj‖=eλjt‖vj‖, where ‖⋅‖ is the Euclidean norm. Thus, the deformation of the initial perturbation δx0 is expressed as exponential growth/decay along the directions vj. Taking the limit t→∞, we obtain the asymptotic rates 8λj(x0)=lim⁡t→∞λj(t,x0)∈R,j=1,…,Nx, known as the Lyapunov exponents (LEs). The existence of these limits is guaranteed by Oseledets' Multiplicative Ergodic Theorem . If the dynamics Eq. () is ergodic, the LEs are uniquely determined regardless of the choice of x0 belonging to an invariant subset of RNx. For autonomous continuous-time dynamical systems of the form Eq. (), at least one exponent is zero, λj=0, corresponding to a perturbation parallel to the vector field δx(t)=f(x(t)) . If the dynamics admits a positive exponent λ1>0, there exists at least one unstable direction in which perturbations grow exponentially, i.e., the dynamics is chaotic. According to Sect. , we define the following dimension to quantify the degrees of freedom of unstable perturbations: 9N+=#{j∈{1,…,Nx}∣λj>0}. See for a more comprehensive introduction to LEs and their associated vectors.

We estimate the LEs numerically using the standard algorithm based on QR decomposition, as detailed in Algorithm  . To implement this algorithm, we require a vector field f:RNx→RNx, its Jacobian Jf∈RNx×Nx, an initial state x0∈RNx, an ODE integrator IntegrateODE, a time step size Δt>0 and a number of iterations n∈N. For the ODE integrator, we use the fourth-order Runge-Kutta method. The state x and the perturbations V are integrated together as an extended state S, where each is treated as a component of S; for example, S[2]=V.

For Nx∈N number of variables, external forcing F∈R and the state vector x=(x1,…,xNx)⊤∈RNx, the Lorenz 96 model is given by 10dxidt=(xi+1-xi-2)xi-1-xi+F,i=1,…,Nx with x-1=xNx-1, x0=xNx and xNx+1=x1. This is a spatio-temporal chaotic model on a one-dimensional periodic domain and often used in data assimilation algorithms. We use this model to show examples of chaotic dynamics with various degrees of instability by changing the parameter F.

We consider a discrete-time filtering problem for the dynamics Eq. () with noisy observations. Let tn=nτ, n=0,1,2,…, denote the observation times with a fixed interval τ>0. We define the flow map Ψτ:RNx→RNx such that Ψτ(x0)=x(τ), where x(t) is the solution to Eq. () with x(0)=x0. This yields the discrete-time dynamical system 11xn=Ψ(xn-1),n=1,2,…, where Ψ=Ψτ and xn=x(tn). The observations are obtained at each tn as 12yn=Hxn+ηn,n=1,2,…, where H∈RNy×Nx is the observation matrix, and ηn∼N(0,R) is a Gaussian observation noise with a symmetric positive definite covariance matrix R∈RNy×Ny. To estimate the state xn from the observations {y1,…,yn}, we employ the ensemble Kalman filter EnKF,, which approximates the mean and covariance of the filtering distribution with an ensemble of state vectors. The EnKF consists of the forecast and analysis steps. In the forecast step, each ensemble member evolves according to the model dynamics as 13xnf(k)=Ψxn-1a(k),k=1,…,m, where m∈N is the ensemble size, and the superscripts f and a denote forecast and analysis, respectively. In the analysis step, the ensemble is updated using Bayes' rule restricted to the Gaussian setting. We employ a particular analysis scheme called the ensemble transform Kalman filter ETKF,. The ETKF updates the mean and perturbation part of the ensemble as 14x‾na=x‾nf+Knyn-Hx‾nf,15Vna=VnfTn, where x‾nf=1m∑k=1mxnf(k), Vnf=[xnf(1)-x‾nf,…,xnf(m)-x‾nf]∈RNx×m, Kn=CnfH⊤(HCnfH⊤+R)-1 is the Kalman gain, Cnf=Vnf(Vnf)⊤/(m-1) is the forecast covariance, and Tn∈Rm×m is a transform matrix defined as 16Tn=Im+1m-1(Vnf)⊤H⊤R-1HVnf-1/2, where the matrix square root is chosen to be symmetric positive definite. Finally, the analysis ensemble members are reconstructed as 17xna(k)=x‾na+vna(k),k=1,…,m, where vna(k) denotes the kth column of Vna.

As mentioned in Sect. , the forecast ensemble is corrected more strongly in directions with higher uncertainty, as represented by the forecast covariance Cnf. However, the rank of Cnf is at most m-1 with m ensemble members. As well as this rank deficiency, the ensemble covariance suffers from the underestimation of variance due to the limited ensemble size. To mitigate these issues, we employ multiplicative covariance inflation: 18Vnf←αVnf, where α≥1 is the inflation factor.

To evaluate the long-term performance of the filter along the context of rigorous error analysis , we define filter accuracy as follows. Assume R=r2INy with r>0. The EnKF achieves filter accuracy if there exists a constant c>0, independent of r, such that 19limsupn→∞E[SEn]=limsupn→∞E[‖xn-x‾na‖2]≤cr2 for sufficiently small r, where the expectation is taken with respect to the probability distributions of the observation noise and the initial ensemble. We reformulate the minimum ensemble size m* for the ETKF based on this definition of filter accuracy as the smallest m∈N such that the ETKF achieves filter accuracy Eq. () with an appropriate choice of the inflation factor α>1. As explained in Sect. , this formulation differs from the standard criterion using the RMSE with a fixed r in two aspects. First, the filter accuracy in Eq. () implies that the time-average of the expectation of RMSE is the order of r. But, the converse is not necessarily true due to Jensen's convex inequality. Thus, Eq. () provides a stronger guarantee for filter performance than the standard criterion (see Appendix  for more details). Second, the observation noise level r is treated as an asymptotic parameter, which makes rigorous mathematical analysis easier and qualitatively distinguishes between divergent and accurate filtering behavior using the dependency on r. Owing to this formulation, we can clearly define the minimum ensemble size m* and simplify the analysis of its relationship with the geometric properties of the instability of dynamics.

To generate an ensemble with its mean close to the true state and its perturbations aligned with the unstable subspace, we introduce an ensemble downsizing method. We begin with a sufficiently large ensemble size, m0, and reduce it to a smaller size, m, at a fixed spin-up time, n=Nspinup. We call the period before n=Nspinup the ensemble spin-up period. In the ensemble downsizing method, we apply the singular value decomposition (SVD) to the ensemble perturbation, V∈RNx×m0, and retain only the leading m modes. This procedure is detailed in Algorithm . We suppose that the SVD algorithm returns the singular values in decreasing order and the associated singular vectors accordingly. For a matrix A, we use MATLAB-style indexing notation: A[:,1:m] denotes the submatrix formed by the first m columns of A, and A[1:m,1:m] denotes the leading m×m principal submatrix.

The resulting ETKF with the multiplicative covariance inflation Eq. () and the ensemble downsizing method is summarized in Algorithm .

See for an efficient implementation of the analysis step.

We first set F=8, a typical parameter for which the Lorenz 96 model exhibits chaotic behavior. The LEs are computed using Algorithm  with Δt=0.001 and n=106 (Fig. ). In the computation, we define the index of the zero exponent as the minimizer of i→|λi|. This yields N+=13 and the largest LE λ1≈1.67.

In this section, we set the observation time interval 0.05 (i.e., nobs=5). We reduce the ensemble size after Nspinup=720 assimilation steps. For each pair (r,m), we vary the inflation factor α to find the optimal value that minimizes the SE defined in Eq. (). The results are shown in Fig.  with log-log plots of the SE against r for different m. If m is larger than or equal to N++1=14, the SE is bounded by the order of r2, indicating that the ETKF achieves filter accuracy. Conversely, if m is smaller than 14, the SE stays around O(1) even for small r, indicating that the ETKF does not achieve filter accuracy. Accordingly, our formulation of the filter accuracy Eq. () qualitatively distinguishes between divergent and accurate filtering behavior and evaluates the minimum ensemble size as m*=N++1=14 in this setting. In particular, in the border case m=14, the ensemble spin-up works effectively under the small r assumption because the SE is larger than the order of r2 for relatively large r but becomes comparable to it for small r. To allow for a relaxed condition of using large r with m=14, we examine an alternative condition, a short time interval, for practical settings with fixed r in Appendix .

We then conduct a longer experiment (N=720000) to investigate the dependence of the RMSE on the spin-up period Nspinup with a single random seed to generate the observation noise. We set r=10-4 and m=14, which avoids filter divergence in the experiment for Fig. . For Nspinup=0 and 720, we show the time series of the RMSE with various α in Fig. . In Fig. a, all series of the RMSE quickly decay to values with the order of r. Moreover, the filter remains stable over a long assimilation period. In Fig. b, the RMSE with α=1.5 decays to a small value. The time required for the RMSE to decay is much longer than that in Fig. a. A potential explanation for this phenomenon is the slow decay of the uncertainty in the neutral direction. Since we focus on the time asymptotic accuracy, this phenomenon is not further investigated in this study. From these results, we conclude that the ensemble spin-up and downsizing method effectively saves the computational time to generate a small ensemble sustaining filter accuracy, which is crucial in more high-dimensional applications.

We then investigate the effect of the ensemble spin-up on the ensemble alignment with the unstable subspace. The integration time is N=360000 and a single random seed is used to generate the observation noise. To clarify the effect of the ensemble downsizing method related to the unstable subspace, we perform experiments with initial ensemble which has accurate mean close to the true state and inaccurate perturbations not aligned in the unstable subspace. Specifically, the initial ensemble X0∈RNx×m is generated as 21X0=[x‾0+v0(k)]k=1m,x‾0∼N(x0,0.01R),v0(k)∼N(0,0.01R) for some m∈N. This construction yields that the expected initial RMSE is the order of 0.1r, but the alignment of the ensemble perturbations with the unstable subspace is not guaranteed. Figure  shows the time series of the RMSE with r=10-4, m=14, various α=1.0,…,1.5 for (a) Nspinup=720 and (b) Nspinup=0. While the RMSE with the ensemble spin-up and downsizing method in Fig. a sustains the order of r for almost all α, the RMSE without the ensemble spin-up immediately diverges for all α and then decays very slowly for only some α. These results distinguish the ETKF with the ensemble spin-up and downsizing method from the one without it in terms of sustaining filter accuracy with inaccurate initial perturbations not aligned with the unstable subspace. To capture the unstable subspace of nonlinear dynamical systems, accurate state estimation is necessary but not sufficient since the unstable subspace depends on the state. The result in Fig.  implies that the ensemble spin-up method can assist not only the convergence of the ensemble mean to the true state but also the alignment of the ensemble perturbations with the unstable subspace.

To verify that m*=N++1 also holds with different value of N+, we set F=16 and compute the LEs as in the previous section, shown in Fig. . This yields N+=15 and the largest LE λ1≈3.82.

We set the observation time interval 0.02 (i.e., nobs=2) in this section. Since we control the interval using nobs in the experiments, the value is determined by the largest integer nobs with which the approximated error expansion nobsλ1Δt in the forecast step for F=16 does not exceed that with nobs=5 for F=8. Indeed, if we write these quantities for F as nobs(F) and λ1(F), the error expansion with each F is approximated as nobs(8)λ1(8)Δt≈5⋅1.67⋅0.01=8.35,nobs(16)λ1(16)Δt≈2⋅3.82⋅0.01=7.64. We compute the SE in the same manner as in the previous section with N=72000 integration steps and Nspinup=1800 assimilation steps, which yields the same integration steps before the ensemble downsizing method. The dependence of the SE on r for different m is shown in Fig. . As in the previous section, m≥N++1=16 gives filter accuracy, while m<16 does not. Therefore, the minimum ensemble size for filter accuracy is m*=N++1=16.