Articles | Volume 32, issue 3
https://doi.org/10.5194/npg-32-353-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/npg-32-353-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Long-window tandem variational data assimilation methods for chaotic climate models tested with the Lorenz 63 system
Philip David Kennedy
CORRESPONDING AUTHOR
Fakultät für Mathematik, Informatik und Naturwissenschaften, Fernerkundung & Assimilation, Universität Hamburg, Bundesstr. 53, 20146 Hamburg, Germany
Abhirup Banerjee
Fakultät für Mathematik, Informatik und Naturwissenschaften, Fernerkundung & Assimilation, Universität Hamburg, Bundesstr. 53, 20146 Hamburg, Germany
Armin Köhl
Fakultät für Mathematik, Informatik und Naturwissenschaften, Fernerkundung & Assimilation, Universität Hamburg, Bundesstr. 53, 20146 Hamburg, Germany
Detlef Stammer
Fakultät für Mathematik, Informatik und Naturwissenschaften, Fernerkundung & Assimilation, Universität Hamburg, Bundesstr. 53, 20146 Hamburg, Germany
Related authors
No articles found.
Guokun Lyu, Longjiang Mu, Armin Koehl, Ruibo Lei, Xi Liang, and Chuanyu Liu
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2024-189, https://doi.org/10.5194/gmd-2024-189, 2025
Preprint under review for GMD
Short summary
Short summary
In the sea ice-ocean models, errors in the parameters and missing spatiotemporal variations contribute to the deviations between the simulations and the observations. We extended an adjoint method to optimize spatiotemporally varying parameters together with the atmosphere forcing and the initial conditions using satellite and in-situ observations. Seasonally, this scheme demonstrates a more prominent advantage in mid-autumn and show great potential for accurately reproducing the Arctic changes.
Yoshihiro Nakayama, Alena Malyarenko, Hong Zhang, Ou Wang, Matthis Auger, Yafei Nie, Ian Fenty, Matthew Mazloff, Armin Köhl, and Dimitris Menemenlis
Geosci. Model Dev., 17, 8613–8638, https://doi.org/10.5194/gmd-17-8613-2024, https://doi.org/10.5194/gmd-17-8613-2024, 2024
Short summary
Short summary
Global- and basin-scale ocean reanalyses are becoming easily accessible. However, such ocean reanalyses are optimized for their entire model domains and their ability to simulate the Southern Ocean requires evaluation. We conduct intercomparison analyses of Massachusetts Institute of Technology General Circulation Model (MITgcm)-based ocean reanalyses. They generally perform well for the open ocean, but open-ocean temporal variability and Antarctic continental shelves require improvements.
Bjorn Stevens, Stefan Adami, Tariq Ali, Hartwig Anzt, Zafer Aslan, Sabine Attinger, Jaana Bäck, Johanna Baehr, Peter Bauer, Natacha Bernier, Bob Bishop, Hendryk Bockelmann, Sandrine Bony, Guy Brasseur, David N. Bresch, Sean Breyer, Gilbert Brunet, Pier Luigi Buttigieg, Junji Cao, Christelle Castet, Yafang Cheng, Ayantika Dey Choudhury, Deborah Coen, Susanne Crewell, Atish Dabholkar, Qing Dai, Francisco Doblas-Reyes, Dale Durran, Ayoub El Gaidi, Charlie Ewen, Eleftheria Exarchou, Veronika Eyring, Florencia Falkinhoff, David Farrell, Piers M. Forster, Ariane Frassoni, Claudia Frauen, Oliver Fuhrer, Shahzad Gani, Edwin Gerber, Debra Goldfarb, Jens Grieger, Nicolas Gruber, Wilco Hazeleger, Rolf Herken, Chris Hewitt, Torsten Hoefler, Huang-Hsiung Hsu, Daniela Jacob, Alexandra Jahn, Christian Jakob, Thomas Jung, Christopher Kadow, In-Sik Kang, Sarah Kang, Karthik Kashinath, Katharina Kleinen-von Königslöw, Daniel Klocke, Uta Kloenne, Milan Klöwer, Chihiro Kodama, Stefan Kollet, Tobias Kölling, Jenni Kontkanen, Steve Kopp, Michal Koran, Markku Kulmala, Hanna Lappalainen, Fakhria Latifi, Bryan Lawrence, June Yi Lee, Quentin Lejeun, Christian Lessig, Chao Li, Thomas Lippert, Jürg Luterbacher, Pekka Manninen, Jochem Marotzke, Satoshi Matsouoka, Charlotte Merchant, Peter Messmer, Gero Michel, Kristel Michielsen, Tomoki Miyakawa, Jens Müller, Ramsha Munir, Sandeep Narayanasetti, Ousmane Ndiaye, Carlos Nobre, Achim Oberg, Riko Oki, Tuba Özkan-Haller, Tim Palmer, Stan Posey, Andreas Prein, Odessa Primus, Mike Pritchard, Julie Pullen, Dian Putrasahan, Johannes Quaas, Krishnan Raghavan, Venkatachalam Ramaswamy, Markus Rapp, Florian Rauser, Markus Reichstein, Aromar Revi, Sonakshi Saluja, Masaki Satoh, Vera Schemann, Sebastian Schemm, Christina Schnadt Poberaj, Thomas Schulthess, Cath Senior, Jagadish Shukla, Manmeet Singh, Julia Slingo, Adam Sobel, Silvina Solman, Jenna Spitzer, Philip Stier, Thomas Stocker, Sarah Strock, Hang Su, Petteri Taalas, John Taylor, Susann Tegtmeier, Georg Teutsch, Adrian Tompkins, Uwe Ulbrich, Pier-Luigi Vidale, Chien-Ming Wu, Hao Xu, Najibullah Zaki, Laure Zanna, Tianjun Zhou, and Florian Ziemen
Earth Syst. Sci. Data, 16, 2113–2122, https://doi.org/10.5194/essd-16-2113-2024, https://doi.org/10.5194/essd-16-2113-2024, 2024
Short summary
Short summary
To manage Earth in the Anthropocene, new tools, new institutions, and new forms of international cooperation will be required. Earth Virtualization Engines is proposed as an international federation of centers of excellence to empower all people to respond to the immense and urgent challenges posed by climate change.
Guokun Lyu, Armin Koehl, Xinrong Wu, Meng Zhou, and Detlef Stammer
Ocean Sci., 19, 305–319, https://doi.org/10.5194/os-19-305-2023, https://doi.org/10.5194/os-19-305-2023, 2023
Short summary
Short summary
Data assimilation techniques are important for combining observations with numerical models. Here, we approximate the adjoint of viscous-plastic dynamics (adjoint-VP) to replace the adjoint of free-drift dynamics (adjoint-FD) for developing an advanced Arctic Ocean and sea ice modeling and adjoint-based assimilation system. We find that adjoint-VP provides a better ocean and sea ice estimation than adjoint-FD, considering the residual errors and adjustments of the atmospheric states.
Guokun Lyu, Nuno Serra, Meng Zhou, and Detlef Stammer
Ocean Sci., 18, 51–66, https://doi.org/10.5194/os-18-51-2022, https://doi.org/10.5194/os-18-51-2022, 2022
Short summary
Short summary
This study explores the Arctic sea level variability depending on different timescales and the relation to temperature, salinity and mass changes, identifying key parameters and regions that need to be observed coordinately. The decadal sea level variability reflects salinity changes. But it can only reflect salinity change at periods of greater than 1 year, highlighting the requirement for enhancing in situ hydrographic observations and complicated interpolation methods.
Cited articles
Abarbanel, H. D., Creveling, D. R., Farsian, R., and Kostuk, M.: Dynamical state and parameter estimation, SIAM J. Appl. Dynam. Syst., 8, 1341–1381, 2009. a
Allaire, G.: A review of adjoint methods for sensitivity analysis, uncertainty quantification and optimization in numerical codes, Ingénieurs de l'Automobile, 836, 33–36, 2015. a
Bar-Shalom, Y., Li, X. R., and Kirubarajan, T.: Estimation with applications to tracking and navigation: theory algorithms and software, John Wiley & Sons, https://doi.org/10.1002/0471221279, 2004. a
Bertino, L., Evensen, G., and Wackernagel, H.: Sequential data assimilation techniques in oceanography, Int. Stat. Rev., 71, 223–241, 2003. a
Cameron, M. and Yang, S.: Computing the quasipotential for highly dissipative and chaotic SDEs an application to stochastic Lorenz’63, Commun. Appl. Math. Comput. Sci., 14, 207–246, 2019. a
Cummings, J. A. and Smedstad, O. M.: Variational data assimilation for the global ocean, in: Data assimilation for atmospheric, oceanic and hydrologic applications (Vol. II), Springer, 303–343, https://doi.org/10.1007/978-3-642-35088-7_13, 2013. a
Daron, J. and Stainforth, D. A.: On quantifying the climate of the nonautonomous Lorenz-63 model, Chaos, 25, 043103, https://doi.org/10.1063/1.4916789, 2015. a
Dembinski, H., Ongmongkolkul, P., Deil, C., Schreiner, H., Feickert, M., Burr, C., Watson, J., Rost, F., Pearce, A., Geiger, L., Abdelmotteleb, A., Desai, A., Wiedemann, B. M., Gohlke, C., Sanders, J., Drotleff, J., Eschle, J., Neste, L., Gorelli, M. E., and Zapata, O.: scikit-hep/iminuit, Zenodo [code], https://doi.org/10.5281/zenodo.3949207, 2020. a
Desroziers, G., Camino, J.-T., and Berre, L.: 4DEnVar: link with 4D state formulation of variational assimilation and different possible implementations, Q. J. Roy. Meteor. Soc., 140, 2097–2110, 2014. a
Du, Y. J. and Shiue, M.-C.: Analysis and computation of continuous data assimilation algorithms for Lorenz 63 system based on nonlinear nudging techniques, J. Comput. Appl. Math., 386, 113246, https://doi.org/10.1016/j.cam.2020.113246, 2021. a
Errico, R. M.: What is an adjoint model?, B. Am. Meteorol. Soc., 78, 2577–2592, 1997. a
Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res.-Oceans, 99, 10143–10162, 1994. a
Evensen, G., Vossepoel, F. C., and Van Leeuwen, P. J.: Data assimilation fundamentals: A unified formulation of the state and parameter estimation problem, Springer Nature, https://doi.org/10.1007/978-3-030-96709-3, 2022. a
Fisher, M., Tremolet, Y., Auvinen, H., Tan, D., and Poli, P.: Weak-constraint and long-window 4D-Var, ECMWF Technical Memoranda, 655, 47, https://doi.org/10.21957/9ii4d4dsq, 2011. a
Gauthier, P.: Chaos and quadri-dimensional data assimilation: A study based on the Lorenz model, Tellus A, 44, 2–17, 1992. a
Giering, R. and Kaminski, T.: Recipes for adjoint code construction, ACM T. Math. Softw., 24, 437–474, 1998. a
Goodliff, M., Amezcua, J., and Van Leeuwen, P. J.: Comparing hybrid data assimilation methods on the Lorenz 1963 model with increasing non-linearity, Tellus A, https://doi.org/10.3402/tellusa.v67.26928, 67, 1–12, 2015. a
Goodliff, M., Fletcher, S., Kliewer, A., Forsythe, J., and Jones, A.: Detection of Non-Gaussian Behavior Using Machine Learning Techniques: A Case Study on the Lorenz 63 Model, J. Geophys. Res.-Atmos., 125, e2019JD031551, https://doi.org/10.1029/2019JD031551, 2020. a
Gustafsson, N., Källén, E., and Thorsteinsson, S.: Sensitivity of forecast errors to initial and lateral boundary conditions, Tellus A, 50, 167–185, 1998. a
Gustafsson, N., Berre, L., Hörnquist, S., Huang, X.-Y., Lindskog, M., Navascues, B., Mogensen, K., and Thorsteinsson, S.: Three-dimensional variational data assimilation for a limited area model: Part I: General formulation and the background error constraint, Tellus A, 53, 425–446, 2001. a
Hall, M. C.: Application of adjoint sensitivity theory to an atmospheric general circulation model, J. Atmos. Sci., 43, 2644–2652, 1986. a
Hall, M. C. and Cacuci, D. G.: Physical interpretation of the adjoint functions for sensitivity analysis of atmospheric models, J. Atmos. Sci., 40, 2537–2546, 1983. a
Hall, M. C., Cacuci, D. G., and Schlesinger, M. E.: Sensitivity analysis of a radiative-convective model by the adjoint method, J. Atmos. Sci., 39, 2038–2050, 1982. a
Hascoet, L. and Pascual, V.: The Tapenade automatic differentiation tool: Principles, model, and specification, ACM T. Math. Softw., 39, 1–43, 2013. a
Hirsch, M. W., Smale, S., and Devaney, R. L.: 14 – The Lorenz System, in: Differential Equations, Dynamical Systems, and an Introduction to Chaos (Third Edition), edited by: Hirsch, M. W., Smale, S., and Devaney, R. L., 3rd Edn., Academic Press, Boston, 305–328, ISBN 978-0-12-382010-5, https://doi.org/10.1016/B978-0-12-382010-5.00014-2, 2013. a
Houtekamer, P. L. and Mitchell, H. L.: A sequential ensemble Kalman filter for atmospheric data assimilation, Mon. Weather Rev., 129, 123–137, 2001. a
Huai, X.-W., Li, J.-P., Ding, R.-Q., Feng, J., and Liu, D.-Q.: Quantifying local predictability of the Lorenz system using the nonlinear local Lyapunov exponent, Atmos. Ocean. Sci. Lett., 10, 372–378, 2017. a
Kalman, R. E.: A New Approach to Linear Filtering and Prediction Problems, J. Basic Eng., 82, 35–45, https://doi.org/10.1115/1.3662552, 1960. a
Köhl, A.: Evaluating the GECCO3 1948–2018 ocean synthesis – a configuration for initializing the MPI-ESM climate model, Q. J. Roy. Meteor. Soc., 146, 2250–2273, 2020. a
Kravtsov, S. and Tsonis, A. A.: Lorenz-63 Model as a Metaphor for Transient Complexity in Climate, Entropy, 23, 951, https://doi.org/10.3390/e23080951, 2021. a
Naval Research Laboratory.: Global HYCOM+CICE 1/12 degree page, https://www7320.nrlssc.navy.mil/GLBhycomcice1-12/ (last access: 17 April 2025), 2016. a
Le Dimet, F.-X. and Talagrand, O.: Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects, Tellus A, 38, 97–110, 1986. a
Marotzke, J., Giering, R., Zhang, K. Q., Stammer, D., Hill, C., and Lee, T.: Construction of the adjoint MIT ocean general circulation model and application to Atlantic heat transport sensitivity, J. Geophys. Res.-Oceans, 104, 29529–29547, 1999. a
Marzban, C.: Variance-based sensitivity analysis: An illustration on the Lorenz'63 model, Mon. Weather Rev., 141, 4069–4079, 2013. a
Mauritsen, T., Stevens, B., Roeckner, E., Crueger, T., Esch, M., Giorgetta, M., Haak, H., Jungclaus, J., Klocke, D., Matei, D., Mikolajewicz, U., Notz, D., Pincus, R., Schmidt, H., and Tomassini, L.: Tuning the climate of a global model, J. Adv. Model. Earth Sy., 4, M00A01, https://doi.org/10.1029/2012MS000154,2012. a
Miller, R. N., Ghil, M., and Gauthiez, F.: Advanced data assimilation in strongly nonlinear dynamical systems, J. Atmos. Sci., 51, 1037–1056, 1994. a
Navon, I. M.: Data assimilation for numerical weather prediction: a review, Data assimilation for atmospheric, oceanic and hydrologic applications, 21–65, https://doi.org/10.1007/978-3-540-71056-1_2, 2009. a
Nichols, N. K.: Mathematical Concepts of Data Assimilation, Springer Berlin Heidelberg, Berlin, Heidelberg, 13–39, ISBN 978-3-540-74703-1, https://doi.org/10.1007/978-3-540-74703-1_2, 2010. a
Pasini, A. and Pelino, V.: Can we estimate atmospheric predictability by performance of neural network forecasting? The toy case studies of unforced and forced Lorenz models, in: CIMSA, 2005 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications, 2005, IEEE, 69–74, https://doi.org/10.1109/CIMSA.2005.1522829,2005. a
Pelino, V. and Maimone, F.: Energetics, skeletal dynamics, and long-term predictions on Kolmogorov-Lorenz systems, Phys. Rev. E, 76, 046214, https://doi.org/10.1103/PhysRevE.76.046214, 2007. a
Quinn, J. C., Bryant, P. H., Creveling, D. R., Klein, S. R., and Abarbanel, H. D.: Parameter and state estimation of experimental chaotic systems using synchronization, Phys. Rev. E, 80, 016201, https://doi.org/10.1103/PhysRevE.80.016201, 2009. a
Rabier, F. and Liu, Z.: Variational data assimilation: theory and overview, in: Proc. ECMWF Seminar on Recent Developments in Data Assimilation for Atmosphere and Ocean, Reading, UK, 8–12 September, 29–43, https://www.ecmwf.int/en/elibrary/76079-variational-data-assimiltion-theory-and-overview, 2003. a
Ruiz, J. J., Pulido, M., and Miyoshi, T.: Estimating model parameters with ensemble-based data assimilation: A review, J. Meteor. Soc. Jpn. Ser. II, 91, 79–99, 2013. a
Simon, D.: Optimal state estimation: Kalman, H infinity, and nonlinear approaches, John Wiley & Sons, https://doi.org/10.1002/0470045345, 2006. a
Stammer, D., Balmaseda, M., Heimbach, P., Köhl, A., and Weaver, A.: Ocean data assimilation in support of climate applications: status and perspectives, Annu. Rev. Mar. Sci., 8, 491–518, 2016. a
Stensrud, D. J. and Bao, J.-W.: Behaviors of variational and nudging assimilation techniques with a chaotic low-order model, Mon. Weather Rev., 120, 3016–3028, 1992. a
Sugiura, N., Masuda, S., Fujii, Y., Kamachi, M., Ishikawa, Y., and Awaji, T.: A framework for interpreting regularized state estimation, Mon. Weather Rev., 142, 386–400, 2014. a
Talagrand, O.: Variational Assimilation. In: Lahoz, W., Khattatov, B., Menard, R. (eds) Data Assimilation. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74703-1_3, 2010. a
Tandeo, P., Ailliot, P., Ruiz, J., Hannart, A., Chapron, B., Cuzol, A., Monbet, V., Easton, R., and Fablet, R.: Combining analog method and ensemble data assimilation: application to the Lorenz-63 chaotic system, in: Machine Learning and Data Mining Approaches to Climate Science: proceedings of the 4th International Workshop on Climate Informatics, Springer, 3–12, https://doi.org/10.1007/978-3-319-17220-0_1, 2015. a
Tett, S. F. B., Yamazaki, K., Mineter, M. J., Cartis, C., and Eizenberg, N.: Calibrating climate models using inverse methods: case studies with HadAM3, HadAM3P and HadCM3, Geosci. Model Dev., 10, 3567–3589, https://doi.org/10.5194/gmd-10-3567-2017, 2017. a
The pandas development team: pandas-dev/pandas: Pandas, Zenodo [code], https://doi.org/10.5281/zenodo.3509134, 2020. a
Tippett, M. K. and Chang, P.: Some theoretical considerations on predictability of linear stochastic dynamics, Tellus A, 55, 148–157, 2003. a
Tippett, M. K., Anderson, J. L., Bishop, C. H., Hamill, T. M., and Whitaker, J. S.: Ensemble square root filters, Mon. Weather Rev., 131, 1485–1490, 2003. a
Tremolet, Y.: Accounting for an imperfect model in 4D-Var, Q. J. Roy. Meteor. Soc., 132, 2483–2504, 2006. a
Van Der Merwe, R. and Wan, E. A.: The square-root unscented Kalman filter for state and parameter-estimation, in: 2001 IEEE international conference on acoustics, speech, and signal processing, Proceedings (Cat. No. 01CH37221), IEEE, vol. 6, 3461–3464, https://doi.org/10.1109/ICASSP.2001.940586, 2001. a
Waskom, M. L.: seaborn: statistical data visualization, J. Open Source Softw. [code], 6, 3021, https://doi.org/10.21105/joss.03021, 2021. a
Wunsch, C.: The ocean circulation inverse problem, Cambridge University Press, ISBN 9780521480901, 1996. a
Wunsch, C. and Heimbach, P.: Estimated Decadal Changes in the North Atlantic Meridional Overturning Circulation and Heat Flux 1993–2004, J. Phys. Oceanogr., 36, 2012–2024, https://doi.org/10.1175/JPO2957.1, 2006. a
Yin, X., Wang, B., Liu, J., and Tan, X.: Evaluation of conditional non-linear optimal perturbation obtained by an ensemble-based approach using the Lorenz-63 model, Tellus A, 66, 22773, https://doi.org/10.3402/tellusa.v66.22773, 2014. a
Zou, X., Navon, I., and Ledimet, F.: An optimal nudging data assimilation scheme using parameter estimation, Q. J. Roy. Meteor. Soc., 118, 1163–1186, 1992. a
Short summary
This work introduces and evaluates two new tandem data assimilation techniques. The first uses two synchronised forward model runs before a single adjoint model run to consistently increase the precision of the parameter estimation. The second uses a lower-resolution model with adjoint equations to drive a higher-resolution target model through data assimilation with no loss in precision compared to data assimilation without tandem methods.
This work introduces and evaluates two new tandem data assimilation techniques. The first uses...