The method of ensemble variational assimilation (EnsVAR), also known as ensemble of data assimilations (EDA), is implemented in fully non-linear conditions on the Lorenz-96 chaotic 40-parameter model. In the case of strong-constraint assimilation, it requires association with the method of quasi-static variational assimilation (QSVA). It then produces ensembles which possess as much reliability and resolution as in the linear case, and its performance is at least as good as that of ensemble Kalman filter (EnKF) and particle filter (PF). On the other hand, ensembles consisting of solutions that correspond to the absolute minimum of the objective function (as identified from the minimizations without QSVA) are significantly biased. In the case of weak-constraint assimilation, EnsVAR is fully successful without need for QSVA.

In the first part of this work (

EnsVAR is implemented in this second part, still on the Lorenz (1996) model,
over assimilation windows for which a linear approximation is no longer
valid. It is implemented first in the strong-constraint case (Sect. 2), where
it turns out to be necessary to use it together with the method of
quasi-static variational assimilation (QSVA), introduced by

Except when explicitly mentioned (and that will be the case mostly concerning
the length of the assimilation windows and the number

Notations such as Eq. (I-3) or Fig. I-2 will refer to equations or figures of
Part 1

Diagnostics of an experiment performed in the same conditions as for
Figs. I-4 and I-5, with the only difference being that the length of the
assimilation windows is now 10 days.

Figure

Another diagnostic is given in Fig.

Histogram of (half) the values of the minima of the objective
function (I-9), for the same experiment as in Fig.

These results tend to confirm the interpretation that was given of results
obtained in Part 1 (see Fig. I-7 and associated comments). This agrees with
the discussion and conclusions of the paper by

This is what has been done here. Figure

Same as Fig.

The improvement over Fig.

Figure

Same as Fig.

All these results show that ensemble variational assimilation is successful, if implemented with QSVA, over long assimilation windows for which the tangent linear approximation is expected to fail. EnsVAR produces ensembles with a high degree of statistical reliability. In addition, the accuracy of the estimated ensembles, as measured by resolution or by the error in the ensemble mean, is improved when the length of the assimilation window is increased.

It can be noted that, if EnsVAR is successful in non-linear situations, it is not because of a possible intrinsically non-linear character. Minimization of an objective function of form (I-3) is a priori valid for statistical estimation only in a linear situation. The success of EnsVAR probably results from the fact that, through QSVA, it is capable of maintaining the current estimate of the flow within the ever-narrower region of state space in which the tangent linear approximation is valid. If the temporal density of the observations became so small, or alternatively if the dynamics of the observed system became so non-linear that it would not be possible to jump from one set of observations to the next one within a linear approximation, EnsVAR would probably fail. This point will deserve further study.

There is actually no reason to expect any strict link between the validity of
the tangent linear approximation and the possible statistical reliability of
minimizing solutions that lie within that approximation (not to speak of
their Bayesianity). As already said, one can expect the a posteriori
Bayesian probability distribution to be concentrated for long assimilation
windows on a folded non-linear subset in state space. The bimodality of
the histogram in Fig.

Same as the lower four panels of Fig.

This has been done on a set of

Clearly, this procedure is a failure as far as reliability is concerned. But
it can also be noted that the errors (bottom-right panel) are smaller than
those of Fig.

Judging from the above results, restricting the ensembles to minimizations
that lead to the absolute minimum of the objective function degrades
reliability, but improves to some extent the quadratic fit to reality. Now,
the Bayesian expectation

Further diagnostics of 10-day ensemble assimilations performed with
QSVA EnsVAR. All diagnostics are performed at the end of the assimilation
windows.

Diagnostics for assimilations performed with EnKF.

Same as Fig.

As in Part 1, we compare the results produced by EnsVAR with those produced
by ensemble Kalman filter (EnKF) and particle filter (PF).
Figures

Similar results have been obtained, with the same conclusions, for longer assimilation periods (not shown).

EnsVAR shows therefore a slight advantage over EnKF and a more distinct advantage over PF. This conclusion is however to be taken with some caution and will be further discussed in the concluding section of the paper.

We present in this section the results of experiments that have been
performed in the weak-constraint case when the deterministic model (I-6)
is no longer considered as being exact. Following a standard approach, we now
assume that the truth is governed by the equation

A typical experiment is as follows. A reference truth

subject to condition (

In Eq. (

As previously, for given reference solution

The experiments have again been performed with the Lorenz-96 model (Eq. I-12). Experiments performed with a linearized version of the model have produced results (not shown) that are entirely consistent with the theory of the BLUE, within a numerical uncertainty which is similar to what has been observed in Part 1.

The covariance matrix

The experimental procedure is otherwise the same as before. In particular,
the complete state vector is observed every 0.5 days, with the observations
being affected with uncorrelated unbiased Gaussian errors with the same
variance

The first conclusion that has been obtained is that QSVA is no longer
necessary for achieving the minimization, at least up to assimilation windows
of length 18 days (the largest value that has been tried). Clearly the
presence of the additional noise penalty term in Eq. (

Diagnostics of weak-constraint variational assimilations performed
over 18-day assimilation windows.

Figure

The bottom panels of Fig.

RMS estimation errors on the state variable

Values of (half) the minima of the objective function for all realizations of the weak-constraint assimilations over 18-day windows.

Figure

These results show that, although there are clearly imperfections (minimizations occasionally lead to secondary minima), ensemble variational assimilation is on the whole very successful for weak-constraint assimilation.

Compared performance of EnsVAR, EnKF, and PF (columns from left to
right respectively) over the last 13 days of 18-day assimilation windows.

Figure

The principle of ensemble variational assimilation (EnsVAR), which has been discussed in the two parts of this work, is very simple. Namely, perturb the data according to their own error probability distribution and, for each set of perturbed data, perform a standard variational assimilation. In the linear and additive Gaussian case, this produces a sample of independent realizations of the (Gaussian) Bayesian probability distribution for the state of the observed system, conditioned by the data.

The primary purpose of this work was to study EnsVAR as a probabilistic estimator in conditions (non-linearity and/or non-Gaussianity) where it cannot be expected to be an exact Bayesian estimator. Since the degree to which Bayesianity is achieved cannot be objectively evaluated, the weaker property of reliability has been evaluated instead. Standard scores, commonly used for evaluation of probabilistic prediction (rank histograms, reliability diagrams and associated Brier score, and in addition the reduced centred random variable) have been used to that end. The additional property of resolution, i.e. the degree to which the estimation system is capable of a priori distinguishing between different outcomes, has also been evaluated (resolution component of the Brier score, root-mean-square error in the mean of the ensembles). Indeed, one purpose of this work was to stress the importance, in the authors' minds, of evaluating ensemble assimilation systems as probabilistic estimators, particularly through the degree to which they achieve reliability and resolution.

The results presented in both parts of this paper show that EnsVAR is fundamentally successful in that, even in conditions where Bayesianity cannot be expected, it produces ensembles which possess a high degree of statistical reliability. Actually, the numerical scores for reliability that have been used are often as good, if not better, in situations where Bayesianity cannot be expected to hold than in situations where it holds. Better scores can be explained in the present situation only by better numerical conditioning. The resolution, as measured by the RMS error in the mean of the ensembles, or by the resolution component of the standard Brier score, is also high.

In non-linear strong-constraint cases, EnsVAR has been successful here only
through the use of quasi-static variational assimilation, which significantly
increases its numerical cost. However, in the weak-constraint case, QSVA has
not been necessary, providing new evidence as to the favourable effect, on
numerical efficiency of assimilation, of introducing a weak constraint. At
the same time, the comparison of the results shown in the right bottom panels
of Figs.

Comparison with two other standard ensemble assimilation filters, namely ensemble Kalman filter and particle filter, made at constant ensemble size, shows a superior or equal performance for EnsVAR, at least as concerns the dispersion of the ensembles.

Our comparison is of course far from being complete. As already said, there
exist many variants of both the EnKF and the PF, and EnsVAR has been compared
here, for each of those two classes of algorithms, with only one of those
variants. Several of these have been studied by

If a code for variational assimilation is available, EnsVAR is very easy (if costly) to implement. It possesses the advantages and disadvantages of standard variational assimilation. The advantages are the easy propagation of information both forward and backward in time (smoothing) and easy introduction of observations of new types and of temporal correlations between data errors. What is usually considered to be a major disadvantage of variational assimilation is the need for developing and maintaining an adjoint code. Concerning that point, it must however be stressed that algorithms are being developed which might avoid the need for adjoints while keeping most of the advantages of variational assimilation.

EnsVAR, as it has been implemented here, is very costly in that it requires a very large number of iterative minimizations. The comparison with EnKF and PF, which has been made here at constant ensemble size, might have led to different conclusions if it had been made at for example constant computing cost. In addition, the particular versions of EnKF and PF that have been used here may not be, among the many versions that exist for both algorithms, the most efficient ones for the problem considered here. In particular, concerning the EnKF, a deterministic version could be used instead of the stochastic version that has been used here. On the other hand, many possibilities exist for reducing the cost or at least the clock time of EnsVAR, through simple parallelization or through use of the results of the first minimizations to speed up the following ones. The rapid development of algorithmic science makes it difficult to draw definitive conclusions at this stage as to the compared cost of various methods for ensemble assimilation.

EnsVAR, at it has been presented here, is almost uniquely defined on the basis of its principle. It has been necessary to introduce only one arbitrary parameter for the experiments that have been described, namely the temporal increment (1 day) between successive assimilation windows in QSVA. Everything else is unambiguously defined once the principle of EnsVAR has been stated. This may of course not remain true in the future, but is certainly a distinct advantage to start with. On the other hand, EnsVAR, like actually the EnKF and the PF, is largely empirical, with the consequence that, should difficulties arise, conceptual guidelines may be missing to solve these difficulties. The only thing that can be said at this stage is that EnsVAR is successful in non-linear situations probably because it keeps the estimation problem within the basin of attraction of the absolute minimum of the objective function to be minimized.

One can also remark that EnsVAR, in the form in which it has been implemented here, and contrary to EnKF and PF, produces an ensemble of totally independent realizations of a same probability distribution. It is difficult to say if that can be considered as a distinct advantage, but it is certainly not a disadvantage.

The problem of cycling EnsVAR for one assimilation window to the next one has
not been considered here. It has been studied to some extent by

EnsVAR has been implemented here on a small-dimension system. It is
operationally running at both ECMWF and Météo-France to specify initial
conditions for the ensemble forecast and to construct the background error
covariance matrix for the variational assimilation. It has not been
systematically evaluated as a probabilistic estimator on a physically
realistic large-dimensional model. It has also to be compared with other
ensemble assimilation methods, in terms of both intrinsic quality of the
results and of cost efficiency. In addition to the many variants of ensemble Kalman filter and particle filter,
one can mention the Metropolis–Hastings
algorithm, which, as already said in the introduction of Part 1, possesses
itself many variants. It has been used for many applications, most of which,
if not all, are however relative to problems with small dimensions. It would
be extremely interesting to study the performance in problems of assimilation
for geophysical fluids. More recently, and in the continuation of

Data used in this article ia available upon request to the corresponding author.

MJ and OT have defined together the scientific approach to the paper and the numerical experiments to be performed. MJ has written the codes and run the experiments. OT wrote most the paper.

The authors declare that they have no conflict of interest.

This article is part of the special issue “Numerical modeling, predictability and data assimilation in weather, ocean and climate: A special issue honoring the legacy of Anna Trevisan (1946–2016)”. It is a result of a Symposium Honoring the Legacy of Anna Trevisan – Bologna, Italy, 17–20 October 2017.

This work has been supported by Agence Nationale de la Recherche, France, through the Prevassemble and Geo-Fluids projects, as well as by the programme Les enveloppes fluides et l'environnement of Institut national des sciences de l'Univers, Centre national de la recherche scientifique, Paris. The authors acknowledge fruitful discussions with Julien Brajard and Marc Bocquet. The latter also acted as a referee along with Massimo Bonavita. Both of them made further suggestions which significantly improved the paper.Edited by: Alberto Carrassi Reviewed by: Marc Bocquet and Massimo Bonavita