References

NPG

Nonlinear Processes in Geophysics

NPG

Nonlin. Processes Geophys.

1607-7946

Copernicus Publications

Göttingen, Germany

10.5194/npg-20-1047-2013

A potential implicit particle method for high-dimensional systems

Weir

¹ Miller

R. N.

¹ Spitz

Y. H.

College of Earth, Ocean, and Atmospheric Sciences, Oregon State University, Corvallis, OR 97331, USA

28 11 2013

20 6 1047 1060

2013

This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit https://creativecommons.org/licenses/by/3.0/

This article is available from https://npg.copernicus.org/articles/20/1047/2013/npg-20-1047-2013.html

The full text article is available as a PDF file from https://npg.copernicus.org/articles/20/1047/2013/npg-20-1047-2013.pdf

This paper presents a particle method designed for high-dimensional state estimation. Instead of weighing random forecasts by their distance to given observations, the method samples an ensemble of particles around an optimal solution based on the observations (i.e., it is implicit). It differs from other implicit methods because it includes the state at the previous assimilation time as part of the optimal solution (i.e., it is a lag-1 smoother). This is accomplished through the use of a mixture model for the background distribution of the previous state. In a high-dimensional, linear, Gaussian example, the mixture-based implicit particle smoother does not collapse. Furthermore, using only a small number of particles, the implicit approach is able to detect transitions in two nonlinear, multi-dimensional generalizations of a double-well. Adding a step that trains the sampled distribution to the target distribution prevents collapse during the transitions, which are strongly nonlinear events. To produce similar estimates, other approaches require many more particles.

References 1

Ades, M. and van Leeuwen, P. J.: An exploration of the equivalent weights particle filter, Q. J. Roy. Meteor. Soc., 139, 820–840, 2013.

Alspach, D. L. and Sorenson, H. W.: Nonlinear Bayesian estimation using Gaussian sum approximations, IEEE T. Automat. Contr., AC-17, 439–448, 1972.

Anderson, J. L. and Anderson, S. L.: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts, Mon. Weather Rev., 127, 2741–2758, 1999.

Atkins, E., Morzfeld, M., and Chorin, A. J.: Implicit particle methods and their connection with variational data assimilation, Mon. Weather Rev., 141, 1786–1803, 2013.

Bellman, R. E.: Dynamic programming, Princeton University Press, Princeton, 1957.

Bengtsson, T., Snyder, C., and Nychka, D.: Toward a nonlinear ensemble filter for high-dimensional systems, J. Geophys. Res., 108, 8775, <a href="http://dx.doi.org/10.1029/2002JD002900">https://doi.org/10.1029/2002JD002900</a>, 2003.

Bengtsson, T., Bickel, P., and Li, B.: Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems, in: Probability and Statistics: Essays in Honor of David A. Freedman, Institute of Mathematical Statistics, Beachwood, Ohio, 2, 316–334, 2008.

Bertino, L., Evensen, G., and Wackernagel, H.: Sequential data assimilation techniques in oceanography, Int. Stat. Rev., 71, 223–241, 2003.

Bickel, P., Li, B., and Bengtsson, T.: Sharp failure rates for the bootstrap particle filter in high dimensions, in: Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, Institute of Mathematical Statistics, Beachwood, Ohio, 3, 318–329, 2008.

Bocquet, M.: Ensemble Kalman filtering without the intrinsic need for inflation, Nonlin. Processes Geophys., 18, 735–750, <a href="http://dx.doi.org/10.5194/npg-18-735-2011">https://doi.org/10.5194/npg-18-735-2011</a>, 2011.

Bocquet, M. and Sakov, P.: Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems, Nonlin. Processes Geophys., 19, 383–399, <a href="http://dx.doi.org/10.5194/npg-19-383-2012">https://doi.org/10.5194/npg-19-383-2012</a>, 2012.

Bocquet, M., Pires, C. A., and Wu, L.: Beyond Gaussian statistical modeling in geophysical data assimilation, Mon. Weather Rev., 138, 2997–3023, 2010.

Chen, R. and Liu, J. S.: Mixture Kalman filters, J. Roy. Stat. Soc. B, 62, 493–508, 2000.

Chorin, A. J. and Tu, X.: Implicit sampling for particle filters, P. Natl. Acad. Sci. USA, 106, 17249–17254, 2009.

Chorin, A. J., Morzfeld, M., and Tu, X.: Implicit particle filters for data assimilation, Communications in Applied Mathematics and Computational Science, 5, 221–240, 2010.

David, H. A. and Nagaraja, H. N.: Order statistics, John Wiley & Sons, 3rd Edn., Hoboken, 2003.

Dempster, A. P., Laird, N. M., and Rubin, D. B.: Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Stat. Soc. B-Met., 39, 1–38, 1977.

Doucet, A., Godsill, S., and Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering, Stat. Comput., 10, 197–208, 2000.

Dovera, L. and Rossa, E. D.: Multimodal ensemble Kalman filtering using Gaussian mixture models, Comput. Geosci., 15, 307–323, 2011.

Efron, B.: Bootstrap methods: another look at the jackknife, Ann. Stat., 7, 1–26, 1979.

Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res., 99, 10143–10162, 1994.

Evensen, G.: Data assimilation: the ensemble Kalman filter, Springer, Dordrecht, 2nd Edn., 2009.

Evensen, G. and van Leeuwen, P. J.: An ensemble Kalman smoother for nonlinear dynamics, Mon. Weather Rev., 128, 1852–1867, 2000.

Eyink, G. L. and Kim, S.: A maximum entropy method for particle filtering, J. Stat. Phys., 123, 1071–1128, 2006.

Eyink, G. L. and Restrepo, J. M.: Most probable histories for nonlinear dynamics: tracking climate transitions, J. Stat. Phys., 101, 459–472, 2000.

Eyink, G. L., Restrepo, J. M., and Alexander, F. J.: A mean field approximation in data assimilation for nonlinear dynamics, Physica D, 195, 347–368, 2004.

Frei, M. and Künsch, H. R.: Mixture ensemble Kalman filters, Comput. Stat. Data An., 58, 127–138, 2013.

Geweke, J.: Bayesian inference in econometric models using Monte Carlo integration, Econometrica, 57, 1317–1339, 1989.

Gordon, N. J., Salmond, D. J., and Smith, A. F. M.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation, IEE Proc. F, 140, 107–113, 1993.

Hoteit, I., Pham, D.-T., Triantafyllou, G., and Korres, G.: A new approximate solution of the optimal nonlinear filter for data assimilation in meteorology and oceanography, Mon. Weather Rev., 136, 317–334, 2008.

Jazwinski, A. H.: Stochastic processes and filtering theory, Academic Press, New York, 1970.

Kalnay, E. and Yang, S.-C.: Accelerating the spin-up of ensemble Kalman filtering, Q. J. Roy. Meteor. Soc., 136, 1644–1651, 2010.

Kim, S., Eyink, G. L., Restrepo, J. M., Alexander, F. J., and Johnson, G.: Ensemble filtering for nonlinear dynamics, Mon. Weather Rev., 131, 2586–2594, 2003.

Kitagawa, G.: Monte Carlo filter and smoother for non-Gaussian nonlinear state space models, J. Comput. Graph. Stat., 5, 1–25, 1996.

Klaas, M., de Freitas, N., and Doucet, A.: Toward practical $N^2$ Monte Carlo: the marginal particle filter, in: Proceedings of the 21st annual conference on uncertainty in artificial intelligence, AUAI Press, Arlington, 308–315, 2005.

Kloeden, P. E. and Platen, E.: Numerical solution of stochastic differential equations, Springer, Berlin, 1999.

Kong, A., Liu, J. S., and Wong, W. H.: Sequential imputations and Bayesian missing data problems, J. Am. Stat. Assoc., 89, 278–288, 1994.

Konishi, S. and Kitagawa, G.: Information criteria and statistical modeling, Springer, New York, 2008.

Kotecha, J. H. and Djurić, P. M.: Gaussian sum particle filtering, IEEE T. Signal Proces., 51, 2602–2612, 2003.

Kravtsov, S., Kondrashov, D., and Ghil, M.: Multilevel regression modeling of nonlinear processes: derivation and applications to climatic variability, J. Climate, 18, 4404–4424, 2005.

Kushner, H. J. and Yin, G. G.: Stochastic approximation and recursive algorithms and applications, Springer, New York, 2003.

Liu, J. S.: Metropolized independent sampling with comparisons to rejection sampling and importance sampling, Stat. Comput., 6, 113–119, 1996.

Majda, A. J., Timofeyev, I., and Vanden-Eijnden, E.: Systematic strategies for stochastic mode reduction in climate, J. Atmos. Sci., 60, 1705–1722, 2003.

Mardia, K. V.: Applications of some measures of multivariate skewness in testing normality and robustness studies, Sankhya Ser. B, 36, 115–128, 1974.

McLachlan, G. J. and Krishnan, T.: The EM algorithm and extensions, John Wiley & Sons, Hoboken, 2nd Edn., 2008.

McLachlan, G. J. and Peel, D.: Finite mixture models, John Wiley & Sons, New York, 2001.

Miller, R. N., Ghil, M., and Gauthiez, F.: Advanced data assimilation in strongly nonlinear dynamical systems, J. Atmos. Sci., 51, 1037–1056, 1994.

Miller, R. N., Carter Jr., E. F., and Blue, S. T.: Data assimilation into nonlinear stochastic models, Tellus A, 51, 167–194, 1999.

Moore, E. H.: On the reciprocal of the general algebraic matrix, in: The fourteenth western meeting of the American Mathematical Society, edited by: Dresden, A., Am. Math. Soc., 26, 385–396, 1920.

Morzfeld, M. and Chorin, A. J.: Implicit particle filtering for models with partial noise, and an application to geomagnetic data assimilation, Nonlin. Processes Geophys., 19, 365–382, <a href="http://dx.doi.org/10.5194/npg-19-365-2012">https://doi.org/10.5194/npg-19-365-2012</a>, 2012.

Morzfeld, M., Tu, X., Atkins, E., and Chorin, A. J.: A random map implementation of implicit filters, J. Comput. Phys., 231, 2049–2066, 2012.

Øksendal, B. K.: Stochastic differential equations: an introduction with applications, Springer, Berlin, 6th Edn., 2003.

Ott, E., Hunt, B. R., Szunyogh, I., Zimin, A. V., Kostelich, E. J., Corazza, M., Kalnay, E., Patil, D. J., and Yorke, J. A.: A local ensemble Kalman filter for atmospheric data assimilation, Tellus A, 56, 415–428, 2004.

Penrose, R.: A generalized inverse for matrices, Math. Proc. Cambridge, 51, 406–413, 1951.

Rauch, H. E.: Solutions to the linear smoothing problem, IEEE T. Automat. Contr., AC-8, 371–372, 1963.

Reich, S.: A Gaussian-mixture ensemble transform filter, Q. J. Roy. Meteor. Soc., 138, 222–233, 2012.

Robbins, H. and Monro, S.: A stochastic approximation method, Ann. Math. Stat., 22, 400–407, 1951.

Silverman, B. W.: Density estimation for statistics and data analysis, Chapman and Hall/CRC, London, 1986.

Smith, K. W.: Cluster ensemble Kalmn filter, Tellus A, 59, 749–757, <a href="http://dx.doi.org/10.1111/j.1600-0870.2007.00246.x">https://doi.org/10.1111/j.1600-0870.2007.00246.x</a>, 2007.

Snyder, C.: Particle filters, the "optimal" proposal and high-dimensional systems, Seminar on data assimilation for atmosphere and ocean, Reading, England, 6–9 September 2011, ECMWF, 2012.

Snyder, C., Bengtsson, T., Bickel, P., and Anderson, J.: Obstacles to high-dimensional particle filtering, Mon. Weather Rev., 136, 4629–4640, 2008.

Sondergaard, T. and Lermusiaux, P. F. J.: Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. Part I: Theory and scheme, Mon. Weather Rev., 141, 1737–1760, <a href="http://dx.doi.org/10.1175/MWR-D-11-00295.1">https://doi.org/10.1175/MWR-D-11-00295.1</a>, 2013a.

Sondergaard, T. and Lermusiaux, P. F. J.: Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. Part II: Applications, Mon. Weather Rev., 141, 1761–1785, <a href="http://dx.doi.org/10.1175/MWR-D-11-00296.1">https://doi.org/10.1175/MWR-D-11-00296.1</a>, 2013b.

Stordal, A. S., Karlsen, H. A., Nævdal, G., Skaug, H. J., and Vallès, B.: Bridging the ensemble Kalman filter and particle filters: the adaptive Gaussian mixture filter, Comput. Geosci., 15, 293–305, 2011.

Sutera, A.: On stochastic perturbations and long-term climate behavior, Q. J. Roy. Meteor. Soc., 107, 137–151, 1981.

van Leeuwen, P. J.: Particle filtering in geophysical systems, Mon. Weather Rev., 137, 4089–4114, 2009.

van Leeuwen, P. J.: Nonlinear data assimilation in geosciences: an extremely efficient particle filter, Q. J. Roy. Meteor. Soc., 136, 1991–1999, 2010.

van Leeuwen, P. J.: Efficient nonlinear data-assimilation in geophysical fluid dynamics, Comput. Fluids, 46, 52–58, 2011.

van Leeuwen, P. J. and Evensen, G.: Data assimilation and inverse methods in terms of a probabilistic formulation, Mon. Weather Rev., 124, 2898–2913, 1996.

Weare, J.: Particle filtering with path sampling and an application to a biomodal ocean current model, J. Comput. Phys., 228, 4312–4331, 2009.

Weir, B., Miller, R. N., and Spitz, Y. H.: Implicit estimation of ecological model parameters, B. Math. Biol., 75, 223–257, <a href="http://dx.doi.org/10.1007/s11538-012-9801-6">https://doi.org/10.1007/s11538-012-9801-6</a>, 2013.