The Mapping Particle Filter (MPF), introduced by , is a non-parametric deterministic data assimilation method based on sample points, i.e., particles. It involves the transformation of the sample states from a prior density function to a posterior density by passing through intermediate states. These intermediate states are driven by an interacting particle flow designed to minimize the Kullback-Leibler divergence between a kernelized distribution of the sample states and the target posterior distribution. presents a general formulation to minimize the KL divergence. It formulates the flow field using the Fokker-Planck equation to evolve particles and sample the posterior distribution without using a reproducing kernel Hilbert space.

In the MPF, based on a hidden Markov model, a state vector evolves over time using a dynamical model and is observed using an observational model simultaneously, 1xk=M(xk-1,ηk),2yk=H(xk,νk), where xk∈RNx represents the state at time k, M is the model operator, ηk denotes the random model error, yk ∈ RNy are the observations, H is the observation operator, and νk represents the observational error. Here, a general framework is presented in which both model and observational errors can be non-additive.

The target density function of the particle flow corresponds to the posterior probability density using Bayes' formula during the assimilation stage, 3p(xk|y1:k)=p(yk|xk)p(xk|y1:k-1)p(yk|y1:k-1),

This probability density function delineates the analysis states by capturing the likelihood of the forecast given a particular set of observations and a specified prior density.

Consider a set of Np particles {xk-1(1:Np)} that samples the posterior density at time k-1. To obtain a state that matches the posterior density at time k, the MPF iteratively computes intermediate states from the prior to the target. The particles that sample the prior density at time k are states that undergo dynamical evolution from the particles that sample the posterior density at time k-1, denoted as {xkf(j)=xk,0(j)=M(xk-1a(j),ηk(j))}j=1Np, where the second subscript represents the pseudo-time of the mapping iteration. The superscript f(j) indicates the jth particle of the forecasted states, and a(j) indicates the jth particle of the analysis states (previous estimates). At each iteration, the particles are transformed by 4xk,i(j)=T(xk,i-1(j))=xk,i-1(j)+ϵv(xk,i-1(j)), where T represents the iteration mapping, v represents the velocity of the particle flow in pseudo-time, ϵ represents the step size of the mapping. It may be considered fixed or estimated adaptively by means of stochastic optimization algorithms .

The velocity seeks to minimize the Kullback-Leibler divergence between the target posterior density function, and the density of the intermediate states. Therefore, the sample from the prior density is transformed towards a sample from the posterior density through a set of discrete transformations, which in the infinitesimal limit may be interpreted as a flow in the state space.

MPF is inspired by the Stein Variational Gradient Descent method which is kernel-based. These methods are algorithms that rely on kernel functions to measure similarities between state vectors from different particles. The MPF selects a space of functions known as the unit ball of a reproducing kernel Hilbert space (RKHS), denoted as F. The optimization task is to find v∈F that indicates the steepest descent direction of the Kullback-Leibler Divergence DKL between the target posterior density and the intermediate density.

By choosing an isotropic kernel K and given a set of particles {xk,i-1(1:Np)} representing a sample of the intermediate density at pseudotime i-1, the gradient of the Monte Carlo integration of the KL divergence is computed as: 5v(x)=1Np∑j=1NpK(xk,i-1(j),x)∇xk,i-1(j)log⁡p(xk,i-1(j))+∇xk,i-1(j)log⁡K(xk,i-1(j),x). The first term in the parenthesis of Eq. (), called the kernel-smoothed gradient of the posterior density, acts as a central force guiding the samples from an initial distribution density function towards the modes of the posterior density. The second term acts as the repulsive force and prevents the particles from collapsing into modes of the posterior. Note that the variables in Eqs. () and () are nondimensionalized by proper scaling as in previous works . To consider dimensional variables one may rewrite ϵ=Dδt, where D is a diffusion coefficient and δt is the pseudo-time step. In that case, the diffusion coefficient controls the optimization convergence rate as in gradient flows and must be incorporated into the velocity term. In this work, however, we keep ϵ as the single effective parameter controlling the convergence rate and adapt it during gradient descent using low-order momentum estimates .

Radial basis functions are used as kernels in this work, 6K(x,x′)=e-12x-x′Σ2, where x-x′Σ2=(x-x′)⊤Σ-1(x-x′) denotes the square of the Mahalanobis distance and Σ is referred to as the kernel covariance matrix. This matrix needs to be specified at the beginning of the process. In this work, it is assumed to be proportional to the forecast covariance matrix, though other approaches for defining it are possible.

The gradient of the logarithm of the posterior density requires the analytical forms of the prior density and the likelihood function. In this work, observational errors are assumed to be additive and Gaussian, but the framework is general and other observational error distributions may be considered. The resulting gradient of the log posterior density, evaluated at xk,i-1(j) is: 7∇xk,i-1(j)log⁡pxk,i-1(j)=H⊤Rk-1yk-Hxk,i-1(j)+∇xk,i-1(j)log⁡p(xk,i-1(j)|y1:k-1).

The first term is the observation likelihood function, in which H is the observational operator, (∈RNy×Nx), and H denotes the tangent linear observation operator, H=dHdx(x), while R stands for the observational error covariance matrix (∈RNy×Ny). The second term in Eq. () is the gradient of the logarithm of the prior density.

In the case of a Gaussian prior density, where 8p(xk(j)|y1:k-1)=Z⋅e-12xk,i-1(j)-x‾k,0Bk2, the second term is reduced to 9∇xk,i-1(j)log⁡p(xk(j)|y1:k-1)=-Bk-1xk,i-1(j)-x‾k,0, where Z is the normalizing constant, Bk is the prior or background covariance matrix and x‾k,0 is the prior mean. For sequential Monte Carlo, the prior density in Eq. () is given by the forecast density, such that Bk=P^kf.

Alternatively, if we assume the prior density is a Gaussian mixture based on the forecast particles, the prior is 10p(xk(j)|y1:k-1)=Z⋅exp⁡-12xk,i-1(j)-μk,i,jQk2, where μk,i,j=∑m=1NpψkijmM(xk-1(m))∑m=1Npψkijm are the Gaussian centroids, ψk,i,j,m=exp-12xk,i-1(j)-M(xk-1(m))Qk2 that represents the adaptive weights, and Qk represents the covariance matrix of the Gaussian mixture. The second term in Eq. () results in 11∇xk,i-1(j)log⁡p(xk(j)|y1:k-1)=-Qk-1xk,i-1(j)-∑m=1NpψkijmM(xk-1(m))∑m=1Npψkijm, If the model is stochastic with additive Gaussian errors, Eq. () is exact .

To illustrate the practical implementation of this approach, we evaluate Eq. () under the assumption of a Gaussian prior distribution with a radial basis function kernel, 12v(x)=1Np∑j=1Np{e-12xk,i-1(j)-xΣ2H⊤Rk-1yk-Hxk,i-1(j)-Bk-1xk,i-1(j)-x‾k,0-Σk-1xk,i-1(j)-x}. The computational cost of a single pseudo-time iteration in the Gaussian-mixture prior case is 13ONx2Np2+Ny2Np+NxNyNp, where the first term corresponds to the kernel and its gradient calculation, while the second and third terms correspond to the computational cost of the likelihood. Assuming that the matrix inversion can be performed in Oinv(Nx), the overall computational cost becomes 14OOinv(Nx)+NitNx2Np2+Ny2Np+NxNyNp, where Nit denotes the number of pseudo-time iterations.

The underlying assumption in the two developed localization methods is that on average, error correlations decay with the physical distance so that when the distance between two variables is larger than a given threshold known as the localization radius, the correlation is assumed to be negligible. The correlations of these far points are neglected so that it becomes feasible to produce an inference using only the points of the background state and the observations within the localization radius. This reduction in algorithmic complexity allows to reduce sampling noise and enhance the quality of the analysis for high-dimensional state spaces.

Both and the present work address the challenge of applying MPF in high-dimensional systems, but through different approaches. Hu and van Leeuwen focus on an intrinsic modification of the particle interaction mechanism by transitioning from a scalar kernel to a matrix kernel; within the matrix kernel, they assume the distance between particles is independent for each component of the state vector. In addition to the kernel modification, their work also applies localization to the prior covariance matrix through a Schur product with a distance-decaying correlation matrix. In contrast, this work starts from a localization assumption, and applies it coherently to both the posterior distribution and the kernel. This results in explicit localization schemes that restructure how the optimization process is applied in state space, thereby modifying the sequencing of the optimization process.

The α-localization algorithm assumes that the kernels are localized around each variable, so that distances between particles are measured in a low dimensional space. Furthermore, it uses a localized prior covariance matrix, for instance, by keeping blocks of the global sample covariance via the Schur product. The state updates are determined globally.

On the other hand, the β-localization algorithm assumes the full local variational mapping process is localized around each variable. In terms of the localization assumption, this method is similar to the localization in and has also some resemblance to the methodology implemented in . For each variable, the optimization is conducted separately considering the observations and the prior state variables within the localization radius.

The nature model is defined by the two-scale Lorenz system equations : 26X˙n=-Xn-1(Xn-2-Xn+1)-Xn+F-hcb∑j=J(n-1)+1nJYjn=1,⋯,NLSY˙m=-cbYm+1(Ym+2-Ym-1)-cYm+hcbXint[(m-1)/J]+1m=1,⋯,NSS within a cyclic domain, i.e, XNLS+1=X1,X0=XNLS, and X-1=XNLS-1; YNSS+1=Y1, YNSS+2=Y2 and Y0=YNSS. NLS is the number of large-scale (LS) variables, and NSS the number of the small-scale (SS) variables. The equations are solved using a fourth-order Runge-Kutta scheme. The parameters of the nature model are specified in Table . They correspond to the standard configuration of the two-scale Lorenz system following .

The forecast model employed in the data assimilation system is the corresponding one-scale Lorenz system . This model exclusively replicates the large-scale equations so that the influence of the small-scale variables must be parameterized. As the true model, the equations are solved using a fourth-order Runge-Kutta scheme. The equations for the one-scale case are 27X˙n=-Xn-1(Xn-2-Xn+1)-Xn+f(Xn)n=1,⋯,NSU within a cyclic domain. NSU is the number of variables of the surrogate model. In order to be consistent, NSU must be equal to NLS. The external forcing, f(Xn), is defined as f(Xn)=F+f*(Xn) and consists of a linear parameterization of the effects of small-scale dynamics. The parameterization coefficients, F and f*, are estimated using the methodology proposed by . The parameters of the forecast model are specified in Table .

The first experiment evaluates the performance of the local mapping particle flow filters under a linear observation operator, as in Eq. (). Figure  shows the results for the fully observed (left panels) and partially observed (right panels) scenarios, employing 20 in Fig. a and 50 particles in Fig. b. Black dots and crosses represent Gaussian filters or MPFs that assume Gaussian priors. Red dots and crosses represent particle flow filters with Gaussian mixture priors.

All MPF experiments exhibit better performance than ETKF for the 20-particle experiments with a full observed state, as in Fig. a, except for the global MPF using a pure Gaussian prior PDF. This last case converges to an RMSE of 0.644 ± 0.001 but with an extremely high spread value (6.70 ± 0.04). On the other hand, when a Gaussian mixture prior density is utilized, represented by the red dots, all three MPF experiments demonstrate performances similar to LETKF.

Regarding the spread, MPF and LMPF with pure Gaussian priors tend to have large dispersion. In the case of LMPF-β, the spread is 0.92 and is not shown. However, this pattern changes in Gaussian mixture experiments, where the spread is much closer to the RMSE. The global case provides the spread that is closest to the RMSE. It is important to note that these spread results come from experiments using optimal hyperparameters in terms of RMSE.

Despite the linearity of the observational operator, the model dynamics is non-linear. Consequently, it is expected that Gaussian mixtures capture non-linearities more effectively compared to experiments utilizing pure Gaussian priors. This could explain the better performance of the Gaussian mixture experiments.

The right panel of Fig. a shows results for partially observed experiments. In the Gaussian prior case, the global MPF converged to a very high RMSE (1.85 ± 0.03) and is therefore not shown. The localized filters achieve a lower RMSE than ETKF, and perform similarly than LETKF. As in the fully observed scenario, the Gaussian mixture experiments show a significant improvement across all MPFs. The resulting RMSE is comparable to that of LETKF.

Figure b presents the results for the 50-particle experiments. The performance relationships among the experiments are similar to the previous case, with the notable exception of ETKF, which shows the most significant improvement. In this case, the Gaussian-prior global MPF achieves convergence, although its RMSE remains higher than that of the Kalman filters. Similarly, the Gaussian mixture experiments demonstrate a significant improvement in RMSE.

In the partially observed scenario, the ETKF demonstrates the most significant improvement, and the Gaussian-prior MPF successfully converges. Additionally, the spread of the Gaussian-mixture prior MPF is closer to the RMSE in the global case. The localized particle filters exhibit a similar behavior to that observed in the fully observed case.

We note that these experiments use a localization radius of ℓ=3 which is only optimal for the LETKF with Np=20. The localization radius of ℓ=3 was fixed in all the experiments to ensure fair comparison across all methods and ensemble sizes, recognizing that this choice may not be individually optimal for each configuration. Notably, the LMPFs exhibit better performance for longer radii, so that the ℓ=3 fixed radius choice does not systematically favor the proposed localization methods. This setup ensures that differences in performance can be attributed solely to the algorithms themselves rather than variations in the localization radius.

A square observational operator presents a challenge for data assimilation schemes, as it treats negative and positive true states with the same absolute value as equivalent, so that the error distributions in the hidden state space are likely to be a bimodal distribution.

Figure a presents the square-H results for the 20-particle experiments. Overall, the RMSE values are smaller compared to the linear case. This difference is linked to the choice of model error variance. While σR2=0.5 in both cases, the magnitude of nonlinear observations is typically much greater than that of linear observations, resulting in a relatively smaller error in the nonlinear case.

In the fully observed case, both the ETKF and the global MPF with a Gaussian prior converged to very high RMSE values. The Gaussian-prior MPF achieved an RMSE > NoDA-RMSE. However, the LETKF and Gaussian-prior LMPFs achieve good RMSE performance, though with a significant underestimation of the spread in the LMPF-α. On the other hand, the three experiments employing Gaussian-mixture priors demonstrate very good performance, similar to the LETKF. The impact of localization is pronounced in the ensemble Kalman filters (as seen in the ETKF vs. LETKF performance) but has only a minor effect on the Gaussian-mixture MPFs.

In the partially observed case, the ETKF diverged for all inflation parameters tested and the Gaussian-prior MPF achieved an RMSE > NoDA-RMSE. LETKF shows a similar RMSE than the Gaussian-mixture particle filters. Figure b presents the results for the 50-particle experiments. In the fully observed scenario, the Gaussian-prior MPF successfully converges, unlike in the 20-particle case, but with a high RMSE for this observational operator (0.367 ± 0.007) and a significant low spread. The ETKF demonstrates a notable improvement in accuracy, outperforming its localized version. A similar effect is observed in the Gaussian-mixture experiments, where the MPF achieves similar accuracy than its localized counterparts. In this case, the Gaussian-mixture particle filters provide a higher spread compared to the Gaussian-prior filters, with the exception of the Gaussian-prior LMPF-β.

In the partially observed scenario, the ETKF achieves convergence with an RMSE similar to that of LETKF. Once again, the Gaussian-mixture filters demonstrate the best performance, comparable to the Kalman filters, with the exception of the global MPF, which showed a very high RMSE value. In this case, the effect of localization is very positive. However, as in the 20-particle case, all filters significantly underestimate the spread.

Figure  shows the performance of the filters in the logarithmic observation operator case, assessing a highly non-Gaussian regime.

For the 20-particle experiments in Fig. a, both the ETKF and the Gaussian-prior global MPF achieved very high RMSE values in fully observed cases. In the partially observed case, the ETKF reached an RMSE > NoDA-RMSE. Meanwhile, the LETKF achieves excellent RMSE values with a closely matching spread in the fully observed case. In contrast, the Gaussian-priors filters exhibit the worst performance, while the Gaussian-mixture prior localized filters show good performance, comparable to the LETKF. While the LETKF shows the best mean RMSE, it exhibits large error bars due to its sensitivity to initial conditions. The Gaussian-mixture localized filters show slightly higher mean RMSE values, but these fall within the LETKF's error band and have much smaller error bars, resulting in comparable overall performance.

For 50 particles, Fig. b, the ETKF successfully converges in the fully observed case, showing performance comparable to that of the LETKF. In this scenario, the Gaussian-mixture particle filters also demonstrate competitive results. In the partially observed scenario, ETKF and LETKF again show good results, but with large error bar in the case of ETKF. Meanwhile, the localized Gaussian-mixture filters have performance comparable to LETKF. In the case of LMPF-α, the RMSE falls within the LETKF's error band. The MPF's in all its versions show smaller sensitivity to initial conditions, particularly in the partially observed case.

The performance of localized particle filters is assessed by varying the radius of localization. This study is made on the linear fully observed case, and on the logarithm and partially observed case, the most non-Gaussian scenario. The number of particles used is 20 and only Gaussian-mixture prior densities are used in the MPFs.

Figure  shows the results of the linear experiment. The LETKF achieves a minimum RMSE at a localization radius of ℓ=3. This is the main reason why we selected this localization radius to conduct all localized experiments.

For radii greater than 4, the LETKF degrades more rapidly than LMPFs. The LMPFs tend to converge to the same RMSE performance as the global MPF when using a localization radius of 18. This suggests that for Np=20 the local algorithm benefits from incorporating distant covariances, even with reduced weights, to improve the estimation at each grid point.

LMPF-α exhibits a behavior similar to the β-case but results in slightly higher RMSE values and reaches a minimum around ℓ=12. However, the difference is small considering that the RMSE as a function of the localization radius is almost flat for that range of localization scales.

These results reflect the relationship between localization needs and the system's effective dimensionality relative to the ensemble size. For the 40-dimensional single Lorenz dynamics, the number of positive Lyapunov exponents is smaller than the ensemble size used in these experiments (20 particles). In the case of this surrogate model, using a forcing f=26, the number of positive Lyapunov exponents calculated were around 16–17 with the parameterized forcing, f*, and around 18–19 without it. With 20 particles exceeding the number of unstable directions, the ensemble in principle provides sufficient rank to capture the system's dynamics without requiring strong localization, explaining the optimal performance at larger radii.

To test this hypothesis, we conducted additional experiments with reduced ensemble size (10 particles), where the ensemble rank falls below the number of positive Lyapunov exponents.

The results of this experiment are shown in Fig. . The LETKF exhibits behavior similar to the 20-particle case. For the localized particle filters, a minimum value appears around ℓ=9. In the case of the LMPF-α, the filter does not converge for localization radii greater than 14.

We remind that the experiments are under the presence of model error. This affects the optimal localization radius; in particular, the LETKF has a longer optimal localization radius for twin perfect-model experiments. In realistic applications, the presence of model errors is also expected to affect long-range correlations. The MPF appears to behave more robustly to this effect.

Figure  shows the performance of the filters for the logarithmic and partially observed experiments for 20 particles. In this scenario, all the filters achieve a minimum RMSE around ℓ=2,3. The LETKF shows more sensitivity to initial conditions than the localized filters; however, it achieves slightly lower RMSEs.

The two extreme experimental setups – fully observed with a linear observation operator, and partially observed with a logarithmic observation operator – are used to evaluate the sensitivity of performance to the number of particles. As in the previous subsection, only Gaussian-mixture prior densities are considered. The localization radius is fixed at ℓ=3, and the experiments are conducted with particle numbers of 5, 10, 20, 50, and 100. The results are displayed in Fig. .

Global MPF and LMPF-α demonstrate very good performance for small particle numbers in the linear experiment. For larger particle numbers, both localized particle filters achieve excellent performance, comparable to that of the LETKF.

RMSE and spread for the large model error experiments using linear and fully observed cases and logarithm and partial observations with 20 particles and a localization radius of 3.

In contrast, the results for the partially observed logarithmic case are unexpected. For a small number of particles, only Gaussian-mixture MPFs achieved RMSE less than NoDA-RMSE, although with a high RMSE. At larger particle numbers, LMPF-α achieves convergence with an accuracy greater than that of the Kalman filters. The performance of LMPF-β is similar to Kalman filters.

The performance of the LETKF in this non-Gaussian experiment deteriorates for ensembles of 50 and 100 particles. A plausible explanation is that certain ensemble members diverge and fail to return to the Lorenz attractor, an effect that is found in deterministic filters . The underlying reason for this behavior is that the data assimilation in deterministic Ensemble Kalman filters only scales the prior density without changing its shape. Consequently, if the prior ensemble contains outliers, they persist in the posterior density and can grow towards the next data assimilation cycle. LMPFs do not degrade in performance with increasing numbers of particles and appear to be unaffected by this issue, as the particle communication during pseudo-time iterations modifies the prior's shape, effectively removing outliers. This issue in the LETKF could be mitigated through techniques such as applying random rotations to the analysis perturbations. However, for the purpose of this comparison, we implement a standard LETKF without additional enhancements to provide a consistent baseline against which to evaluate the proposed MPF and LMPFs methods. Interestingly, neither the MPF nor LMPFs exhibit this performance degradation with increasing ensemble size.

In all previous experiments, a linear parameterization of small-scale effects was used. This results in a relatively small model error. To evaluate a large model error scenario, we neglect the linear term of the parameterization f* (Table ) and only use the external forcing f(Xn)=26 in the surrogate model. As said before, NoDA-RMSE for the parametrized surrogate model is 6.78 and the spread is 6.55. For this large model error environment, the RMSE is 6.86 and the spread reaches 9.01.

Both Kalman and particle filters are tested for the linear and logarithmic observation operators, using ℓ=3 for the local filters and Np=20 particles. Again, we use the localization radius that is optimal for the LETKF, and the hyperparameters γ, ξ and inflation are tuned for these cases. The results are displayed in Fig. . In these large model error conditions, the ETKF achieved an RMSE > NoDA-RMSE.

In the linear case, the Gaussian-mixture particle filters have a similar performance in terms of RMSE. Nevertheless, the spread of the global MPF is strongly underestimated. Localized particle filters show higher spreads.

In the logarithmic scenario, the LETKF converged to an RMSE>NoDA-RMSE. In contrast, the MPF and its localized variants handle model error more effectively and perform better in this challenging case. However, all three filters exhibit rather high RMSE values. Among the tested methods, LMPF-α achieves the best performance in this setup.