Articles | Volume 31, issue 3
https://doi.org/10.5194/npg-31-409-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/npg-31-409-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Research article
| Highlight paper
| 
19 Sep 2024
Research article | Highlight paper |
| 19 Sep 2024
| 19 Sep 2024
Representation learning with unconditional denoising diffusion models for dynamical systems
Tobias Sebastian Finn
CORRESPONDING AUTHOR
CEREA, École des Ponts and EDF R&D, Île-de-France, France
Lucas Disson
CEREA, École des Ponts and EDF R&D, Île-de-France, France
Alban Farchi
CEREA, École des Ponts and EDF R&D, Île-de-France, France
Marc Bocquet
CEREA, École des Ponts and EDF R&D, Île-de-France, France
Charlotte Durand
CEREA, École des Ponts and EDF R&D, Île-de-France, France
Related authors
Yumeng Chen, Polly Smith, Alberto Carrassi, Ivo Pasmans, Laurent Bertino, Marc Bocquet, Tobias Sebastian Finn, Pierre Rampal, and Véronique Dansereau
The Cryosphere, 18, 2381–2406, https://doi.org/10.5194/tc-18-2381-2024, https://doi.org/10.5194/tc-18-2381-2024, 2024
Short summary
Short summary
We explore multivariate state and parameter estimation using a data assimilation approach through idealised simulations in a dynamics-only sea-ice model based on novel rheology. We identify various potential issues that can arise in complex operational sea-ice models when model parameters are estimated. Even though further investigation will be needed for such complex sea-ice models, we show possibilities of improving the observed and the unobserved model state forecast and parameter accuracy.
Charlotte Durand, Tobias Sebastian Finn, Alban Farchi, Marc Bocquet, Guillaume Boutin, and Einar Ólason
The Cryosphere, 18, 1791–1815, https://doi.org/10.5194/tc-18-1791-2024, https://doi.org/10.5194/tc-18-1791-2024, 2024
Short summary
Short summary
This paper focuses on predicting Arctic-wide sea-ice thickness using surrogate modeling with deep learning. The model has a predictive power of 12 h up to 6 months. For this forecast horizon, persistence and daily climatology are systematically outperformed, a result of learned thermodynamics and advection. Consequently, surrogate modeling with deep learning proves to be effective at capturing the complex behavior of sea ice.
Tobias Sebastian Finn, Charlotte Durand, Alban Farchi, Marc Bocquet, Yumeng Chen, Alberto Carrassi, and Véronique Dansereau
The Cryosphere, 17, 2965–2991, https://doi.org/10.5194/tc-17-2965-2023, https://doi.org/10.5194/tc-17-2965-2023, 2023
Short summary
Short summary
We combine deep learning with a regional sea-ice model to correct model errors in the sea-ice dynamics of low-resolution forecasts towards high-resolution simulations. The combined model improves the forecast by up to 75 % and thereby surpasses the performance of persistence. As the error connection can additionally be used to analyse the shortcomings of the forecasts, this study highlights the potential of combined modelling for short-term sea-ice forecasting.
Tobias Sebastian Finn, Gernot Geppert, and Felix Ament
Hydrol. Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2020-672, https://doi.org/10.5194/hess-2020-672, 2021
Revised manuscript not accepted
Short summary
Short summary
Through the lens of recent developments in hydrological modelling and data assimilation, we hourly update the soil moisture with ensemble data assimilation and sparse 2-metre-temperature observations in a coupled limited area model system. In idealized experiments, we improve the soil moisture analysis by coupled data assimilation across the atmosphere-land interface. We conclude that we can merge the separated assimilation cycles for the atmosphere and land surface into one single cycle.
Simon Driscoll, Alberto Carrassi, Julien Brajard, Laurent Bertino, Einar Ólason, Marc Bocquet, and Amos Lawless
EGUsphere, https://doi.org/10.5194/egusphere-2024-2476, https://doi.org/10.5194/egusphere-2024-2476, 2024
Short summary
Short summary
The formation and evolution of sea ice melt ponds (ponds of melted water) are complex, insufficiently understood and represented in models with considerable uncertainty. These uncertain representations are not traditionally included in climate models potentially causing the known underestimation of sea ice loss in climate models. Our work creates the first observationally based machine learning model of melt ponds that is also a ready and viable candidate to be included in climate models.
Joffrey Dumont Le Brazidec, Pierre Vanderbecken, Alban Farchi, Grégoire Broquet, Gerrit Kuhlmann, and Marc Bocquet
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2024-156, https://doi.org/10.5194/gmd-2024-156, 2024
Preprint under review for GMD
Short summary
Short summary
We developed a deep learning method to estimate CO2 emissions from power plants using satellite images. Trained and validated on simulated data, our model accurately predicts emissions despite challenges like cloud cover. When applied to real OCO3 satellite image, the results closely match reported emissions. This study shows that neural networks trained on simulations can effectively analyse real satellite data, offering a new way to monitor CO2 emissions from space.
Marc Bocquet, Pierre J. Vanderbecken, Alban Farchi, Joffrey Dumont Le Brazidec, and Yelva Roustan
Nonlin. Processes Geophys., 31, 335–357, https://doi.org/10.5194/npg-31-335-2024, https://doi.org/10.5194/npg-31-335-2024, 2024
Short summary
Short summary
A novel approach, optimal transport data assimilation (OTDA), is introduced to merge DA and OT concepts. By leveraging OT's displacement interpolation in space, it minimises mislocation errors within DA applied to physical fields, such as water vapour, hydrometeors, and chemical species. Its richness and flexibility are showcased through one- and two-dimensional illustrations.
Yumeng Chen, Polly Smith, Alberto Carrassi, Ivo Pasmans, Laurent Bertino, Marc Bocquet, Tobias Sebastian Finn, Pierre Rampal, and Véronique Dansereau
The Cryosphere, 18, 2381–2406, https://doi.org/10.5194/tc-18-2381-2024, https://doi.org/10.5194/tc-18-2381-2024, 2024
Short summary
Short summary
We explore multivariate state and parameter estimation using a data assimilation approach through idealised simulations in a dynamics-only sea-ice model based on novel rheology. We identify various potential issues that can arise in complex operational sea-ice models when model parameters are estimated. Even though further investigation will be needed for such complex sea-ice models, we show possibilities of improving the observed and the unobserved model state forecast and parameter accuracy.
Charlotte Durand, Tobias Sebastian Finn, Alban Farchi, Marc Bocquet, Guillaume Boutin, and Einar Ólason
The Cryosphere, 18, 1791–1815, https://doi.org/10.5194/tc-18-1791-2024, https://doi.org/10.5194/tc-18-1791-2024, 2024
Short summary
Short summary
This paper focuses on predicting Arctic-wide sea-ice thickness using surrogate modeling with deep learning. The model has a predictive power of 12 h up to 6 months. For this forecast horizon, persistence and daily climatology are systematically outperformed, a result of learned thermodynamics and advection. Consequently, surrogate modeling with deep learning proves to be effective at capturing the complex behavior of sea ice.
Joffrey Dumont Le Brazidec, Pierre Vanderbecken, Alban Farchi, Grégoire Broquet, Gerrit Kuhlmann, and Marc Bocquet
Geosci. Model Dev., 17, 1995–2014, https://doi.org/10.5194/gmd-17-1995-2024, https://doi.org/10.5194/gmd-17-1995-2024, 2024
Short summary
Short summary
Our research presents an innovative approach to estimating power plant CO2 emissions from satellite images of the corresponding plumes such as those from the forthcoming CO2M satellite constellation. The exploitation of these images is challenging due to noise and meteorological uncertainties. To overcome these obstacles, we use a deep learning neural network trained on simulated CO2 images. Our method outperforms alternatives, providing a positive perspective for the analysis of CO2M images.
Tobias Sebastian Finn, Charlotte Durand, Alban Farchi, Marc Bocquet, Yumeng Chen, Alberto Carrassi, and Véronique Dansereau
The Cryosphere, 17, 2965–2991, https://doi.org/10.5194/tc-17-2965-2023, https://doi.org/10.5194/tc-17-2965-2023, 2023
Short summary
Short summary
We combine deep learning with a regional sea-ice model to correct model errors in the sea-ice dynamics of low-resolution forecasts towards high-resolution simulations. The combined model improves the forecast by up to 75 % and thereby surpasses the performance of persistence. As the error connection can additionally be used to analyse the shortcomings of the forecasts, this study highlights the potential of combined modelling for short-term sea-ice forecasting.
Joffrey Dumont Le Brazidec, Pierre Vanderbecken, Alban Farchi, Marc Bocquet, Jinghui Lian, Grégoire Broquet, Gerrit Kuhlmann, Alexandre Danjou, and Thomas Lauvaux
Geosci. Model Dev., 16, 3997–4016, https://doi.org/10.5194/gmd-16-3997-2023, https://doi.org/10.5194/gmd-16-3997-2023, 2023
Short summary
Short summary
Monitoring of CO2 emissions is key to the development of reduction policies. Local emissions, from cities or power plants, may be estimated from CO2 plumes detected in satellite images. CO2 plumes generally have a weak signal and are partially concealed by highly variable background concentrations and instrument errors, which hampers their detection. To address this problem, we propose and apply deep learning methods to detect the contour of a plume in simulated CO2 satellite images.
Pierre J. Vanderbecken, Joffrey Dumont Le Brazidec, Alban Farchi, Marc Bocquet, Yelva Roustan, Élise Potier, and Grégoire Broquet
Atmos. Meas. Tech., 16, 1745–1766, https://doi.org/10.5194/amt-16-1745-2023, https://doi.org/10.5194/amt-16-1745-2023, 2023
Short summary
Short summary
Instruments dedicated to monitoring atmospheric gaseous compounds from space will provide images of urban-scale plumes. We discuss here the use of new metrics to compare observed plumes with model predictions that will be less sensitive to meteorology uncertainties. We have evaluated our metrics on diverse plumes and shown that by eliminating some aspects of the discrepancies, they are indeed less sensitive to meteorological variations.
Joffrey Dumont Le Brazidec, Marc Bocquet, Olivier Saunier, and Yelva Roustan
Geosci. Model Dev., 16, 1039–1052, https://doi.org/10.5194/gmd-16-1039-2023, https://doi.org/10.5194/gmd-16-1039-2023, 2023
Short summary
Short summary
When radionuclides are released into the atmosphere, the assessment of the consequences depends on the evaluation of the magnitude and temporal evolution of the release, which can be highly variable as in the case of Fukushima Daiichi.
Here, we propose Bayesian inverse modelling methods and the reversible-jump Markov chain Monte Carlo technique, which allows one to evaluate the temporal variability of the release and to integrate different types of information in the source reconstruction.
Colin Grudzien and Marc Bocquet
Geosci. Model Dev., 15, 7641–7681, https://doi.org/10.5194/gmd-15-7641-2022, https://doi.org/10.5194/gmd-15-7641-2022, 2022
Short summary
Short summary
Iterative optimization techniques, the state of the art in data assimilation, have largely focused on extending forecast accuracy to moderate- to long-range forecast systems. However, current methodology may not be cost-effective in reducing forecast errors in online, short-range forecast systems. We propose a novel optimization of these techniques for online, short-range forecast cycles, simultaneously providing an improvement in forecast accuracy and a reduction in the computational cost.
Joffrey Dumont Le Brazidec, Marc Bocquet, Olivier Saunier, and Yelva Roustan
Atmos. Chem. Phys., 21, 13247–13267, https://doi.org/10.5194/acp-21-13247-2021, https://doi.org/10.5194/acp-21-13247-2021, 2021
Short summary
Short summary
The assessment of the environmental consequences of a radionuclide release depends on the estimation of its source. This paper aims to develop inverse Bayesian methods which combine transport models with measurements, in order to reconstruct the ensemble of possible sources.
Three methods to quantify uncertainties based on the definition of probability distributions and the physical models are proposed and evaluated for the case of 106Ru releases over Europe in 2017.
Tobias Sebastian Finn, Gernot Geppert, and Felix Ament
Hydrol. Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2020-672, https://doi.org/10.5194/hess-2020-672, 2021
Revised manuscript not accepted
Short summary
Short summary
Through the lens of recent developments in hydrological modelling and data assimilation, we hourly update the soil moisture with ensemble data assimilation and sparse 2-metre-temperature observations in a coupled limited area model system. In idealized experiments, we improve the soil moisture analysis by coupled data assimilation across the atmosphere-land interface. We conclude that we can merge the separated assimilation cycles for the atmosphere and land surface into one single cycle.
Colin Grudzien, Marc Bocquet, and Alberto Carrassi
Geosci. Model Dev., 13, 1903–1924, https://doi.org/10.5194/gmd-13-1903-2020, https://doi.org/10.5194/gmd-13-1903-2020, 2020
Short summary
Short summary
All scales of a dynamical physical process cannot be resolved accurately in a multiscale, geophysical model. The behavior of unresolved scales of motion are often parametrized by a random process to emulate their effects on the dynamically resolved variables, and this results in a random–dynamical model. We study how the choice of a numerical discretization of such a system affects the model forecast and estimation statistics, when the random–dynamical model is unbiased in its parametrization.
Thomas Lauvaux, Liza I. Díaz-Isaac, Marc Bocquet, and Nicolas Bousserez
Atmos. Chem. Phys., 19, 12007–12024, https://doi.org/10.5194/acp-19-12007-2019, https://doi.org/10.5194/acp-19-12007-2019, 2019
Short summary
Short summary
A small-size ensemble of mesoscale simulations has been filtered to characterize the spatial structures of transport errors in atmospheric CO2 mixing ratios. The extracted error structures in in situ and column CO2 show similar length scales compared to other meteorological variables, including seasonality, which could be used as proxies in regional inversion systems.
Marc Bocquet, Julien Brajard, Alberto Carrassi, and Laurent Bertino
Nonlin. Processes Geophys., 26, 143–162, https://doi.org/10.5194/npg-26-143-2019, https://doi.org/10.5194/npg-26-143-2019, 2019
Short summary
Short summary
This paper describes an innovative way to use data assimilation to infer the dynamics of a physical system from its observation only. The method can operate with noisy and partial observation of the physical system. It acts as a deep learning technique specialised to dynamical models without the need for machine learning tools. The method is successfully tested on chaotic dynamical systems: the Lorenz-63, Lorenz-96, and Kuramoto–Sivashinski models and a two-scale Lorenz model.
Julien Brajard, Alberto Carrassi, Marc Bocquet, and Laurent Bertino
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2019-136, https://doi.org/10.5194/gmd-2019-136, 2019
Revised manuscript not accepted
Short summary
Short summary
We explore the possibility of combining data assimilation with machine learning. We introduce a new hybrid method for a two-fold scope: (i) emulating hidden, possibly chaotic, dynamics and (ii) predicting its future states. Numerical experiments have been carried out using the chaotic Lorenz 96 model, proving both the convergence of the hybrid method and its statistical skills including short-term forecasting and emulation of the long-term dynamics.
Liza I. Díaz-Isaac, Thomas Lauvaux, Marc Bocquet, and Kenneth J. Davis
Atmos. Chem. Phys., 19, 5695–5718, https://doi.org/10.5194/acp-19-5695-2019, https://doi.org/10.5194/acp-19-5695-2019, 2019
Short summary
Short summary
We demonstrate that transport model errors, one of the main contributors to the uncertainty in regional CO2 inversions, can be represented by a small-size ensemble carefully calibrated with meteorological data. Our results also confirm transport model errors represent a significant fraction of the model–data mismatch in CO2 mole fractions and hence in regional inverse CO2 fluxes.
Alban Farchi and Marc Bocquet
Nonlin. Processes Geophys., 25, 765–807, https://doi.org/10.5194/npg-25-765-2018, https://doi.org/10.5194/npg-25-765-2018, 2018
Short summary
Short summary
Data assimilation looks for an optimal way to learn from observations of a dynamical system to improve the quality of its predictions. The goal is to filter out the noise (both observation and model noise) to retrieve the true signal. Among all possible methods, particle filters are promising; the method is fast and elegant, and it allows for a Bayesian analysis. In this review paper, we discuss implementation techniques for (local) particle filters in high-dimensional systems.
Colin Grudzien, Alberto Carrassi, and Marc Bocquet
Nonlin. Processes Geophys., 25, 633–648, https://doi.org/10.5194/npg-25-633-2018, https://doi.org/10.5194/npg-25-633-2018, 2018
Short summary
Short summary
Using the framework Lyapunov vectors, we analyze the asymptotic properties of ensemble based Kalman filters and how these are influenced by dynamical chaos, especially in the context of random model errors and small ensemble sizes. Particularly, we show a novel derivation of the evolution of forecast uncertainty for ensemble-based Kalman filters with weakly-nonlinear error growth, and discuss its impact for filter design in geophysical models.
Olivier Pannekoucke, Marc Bocquet, and Richard Ménard
Nonlin. Processes Geophys., 25, 481–495, https://doi.org/10.5194/npg-25-481-2018, https://doi.org/10.5194/npg-25-481-2018, 2018
Short summary
Short summary
The forecast of weather prediction uncertainty is a real challenge and is crucial for risk management. However, uncertainty prediction is beyond the capacity of supercomputers, and improvements of the technology may not solve this issue. A new uncertainty prediction method is introduced which takes advantage of fluid equations to predict simple quantities which approximate real uncertainty but at a low numerical cost. A proof of concept is shown by an academic model derived from fluid dynamics.
Anthony Fillion, Marc Bocquet, and Serge Gratton
Nonlin. Processes Geophys., 25, 315–334, https://doi.org/10.5194/npg-25-315-2018, https://doi.org/10.5194/npg-25-315-2018, 2018
Short summary
Short summary
This study generalizes a paper by Pires et al. (1996) to state-of-the-art data assimilation techniques, such as the iterative ensemble Kalman smoother (IEnKS). We show that the longer the time window over which observations are assimilated, the better the accuracy of the IEnKS. Beyond a critical time length that we estimate, we show that this accuracy finally degrades. We show that the use of the quasi-static minimizations but generalized to the IEnKS yields a significantly improved accuracy.
J.-M. Haussaire and M. Bocquet
Geosci. Model Dev., 9, 393–412, https://doi.org/10.5194/gmd-9-393-2016, https://doi.org/10.5194/gmd-9-393-2016, 2016
Short summary
Short summary
The focus is on the development of low-order models of atmospheric transport and chemistry and their use for data assimilation purposes. A new low-order coupled chemistry meteorology model is developed. It consists of the Lorenz40-variable model used as a wind field coupled with a simple ozone photochemistry module. Advanced ensemble variational methods are applied to this model to obtain insights on the use of data assimilation with coupled models, in an offline mode or in an online mode.
M. Bocquet, P. N. Raanes, and A. Hannart
Nonlin. Processes Geophys., 22, 645–662, https://doi.org/10.5194/npg-22-645-2015, https://doi.org/10.5194/npg-22-645-2015, 2015
Short summary
Short summary
The popular data assimilation technique known as the ensemble Kalman filter (EnKF) suffers from sampling errors due to the limited size of the ensemble. This deficiency is usually cured by inflating the sampled error covariances and by using localization. This paper further develops and discusses the finite-size EnKF, or EnKF-N, a variant of the EnKF that does not require inflation. It expands the use of the EnKF-N to a wider range of dynamical regimes.
M. Bocquet, H. Elbern, H. Eskes, M. Hirtl, R. Žabkar, G. R. Carmichael, J. Flemming, A. Inness, M. Pagowski, J. L. Pérez Camaño, P. E. Saide, R. San Jose, M. Sofiev, J. Vira, A. Baklanov, C. Carnevale, G. Grell, and C. Seigneur
Atmos. Chem. Phys., 15, 5325–5358, https://doi.org/10.5194/acp-15-5325-2015, https://doi.org/10.5194/acp-15-5325-2015, 2015
Short summary
Short summary
Data assimilation is used in atmospheric chemistry models to improve air quality forecasts, construct re-analyses of concentrations, and perform inverse modeling. Coupled chemistry meteorology models (CCMM) are atmospheric chemistry models that simulate meteorological processes and chemical transformations jointly. We review here the current status of data assimilation in atmospheric chemistry models, with a particular focus on future prospects for data assimilation in CCMM.
Y. Wang, K. N. Sartelet, M. Bocquet, P. Chazette, M. Sicard, G. D'Amico, J. F. Léon, L. Alados-Arboledas, A. Amodeo, P. Augustin, J. Bach, L. Belegante, I. Binietoglou, X. Bush, A. Comerón, H. Delbarre, D. García-Vízcaino, J. L. Guerrero-Rascado, M. Hervo, M. Iarlori, P. Kokkalis, D. Lange, F. Molero, N. Montoux, A. Muñoz, C. Muñoz, D. Nicolae, A. Papayannis, G. Pappalardo, J. Preissler, V. Rizi, F. Rocadenbosch, K. Sellegri, F. Wagner, and F. Dulac
Atmos. Chem. Phys., 14, 12031–12053, https://doi.org/10.5194/acp-14-12031-2014, https://doi.org/10.5194/acp-14-12031-2014, 2014
Y. Wang, K. N. Sartelet, M. Bocquet, and P. Chazette
Atmos. Chem. Phys., 14, 3511–3532, https://doi.org/10.5194/acp-14-3511-2014, https://doi.org/10.5194/acp-14-3511-2014, 2014
O. Saunier, A. Mathieu, D. Didier, M. Tombette, D. Quélo, V. Winiarek, and M. Bocquet
Atmos. Chem. Phys., 13, 11403–11421, https://doi.org/10.5194/acp-13-11403-2013, https://doi.org/10.5194/acp-13-11403-2013, 2013
M. Bocquet and P. Sakov
Nonlin. Processes Geophys., 20, 803–818, https://doi.org/10.5194/npg-20-803-2013, https://doi.org/10.5194/npg-20-803-2013, 2013
M. R. Koohkan, M. Bocquet, Y. Roustan, Y. Kim, and C. Seigneur
Atmos. Chem. Phys., 13, 5887–5905, https://doi.org/10.5194/acp-13-5887-2013, https://doi.org/10.5194/acp-13-5887-2013, 2013
Y. Wang, K. N. Sartelet, M. Bocquet, and P. Chazette
Atmos. Chem. Phys., 13, 269–283, https://doi.org/10.5194/acp-13-269-2013, https://doi.org/10.5194/acp-13-269-2013, 2013
Related subject area
Subject: Time series, machine learning, networks, stochastic processes, extreme events | Topic: Climate, atmosphere, ocean, hydrology, cryosphere, biosphere | Techniques: Big data and artificial intelligence
Characterisation of Dansgaard–Oeschger events in palaeoclimate time series using the matrix profile method
Evaluation of forecasts by a global data-driven weather model with and without probabilistic post-processing at Norwegian stations
The sampling method for optimal precursors of El Niño–Southern Oscillation events
A comparison of two causal methods in the context of climate analyses
A two-fold deep-learning strategy to correct and downscale winds over mountains
Downscaling of surface wind forecasts using convolutional neural networks
Learning Extreme Vegetation Response to Climate Forcing: A Comparison of Recurrent Neural Network Architectures
Data-driven methods to estimate the committor function in conceptual ocean models
Exploring meteorological droughts' spatial patterns across Europe through complex network theory
Integrated hydrodynamic and machine learning models for compound flooding prediction in a data-scarce estuarine delta
Predicting sea surface temperatures with coupled reservoir computers
Using neural networks to improve simulations in the gray zone
The blessing of dimensionality for the analysis of climate data
Producing realistic climate data with generative adversarial networks
Identification of droughts and heatwaves in Germany with regional climate networks
Extracting statistically significant eddy signals from large Lagrangian datasets using wavelet ridge analysis, with application to the Gulf of Mexico
Ensemble-based statistical interpolation with Gaussian anamorphosis for the spatial analysis of precipitation
Applications of matrix factorization methods to climate data
Detecting dynamical anomalies in time series from different palaeoclimate proxy archives using windowed recurrence network analysis
Remember the past: a comparison of time-adaptive training schemes for non-homogeneous regression
Susana Barbosa, Maria Eduarda Silva, and Denis-Didier Rousseau
Nonlin. Processes Geophys., 31, 433–447, https://doi.org/10.5194/npg-31-433-2024, https://doi.org/10.5194/npg-31-433-2024, 2024
Short summary
Short summary
The characterisation of abrupt transitions in palaeoclimate records allows understanding of millennial climate variability and potential tipping points in the context of current climate change. In our study an algorithmic method, the matrix profile, is employed to characterise abrupt warmings designated as Dansgaard–Oeschger (DO) events and to identify the most similar transitions in the palaeoclimate time series.
John Bjørnar Bremnes, Thomas N. Nipen, and Ivar A. Seierstad
Nonlin. Processes Geophys., 31, 247–257, https://doi.org/10.5194/npg-31-247-2024, https://doi.org/10.5194/npg-31-247-2024, 2024
Short summary
Short summary
During the last 2 years, tremendous progress has been made in global data-driven weather models trained on reanalysis data. In this study, the Pangu-Weather model is compared to several numerical weather prediction models with and without probabilistic post-processing for temperature and wind speed forecasting. The results confirm that global data-driven models are promising for operational weather forecasting and that post-processing can improve these forecasts considerably.
Bin Shi and Junjie Ma
Nonlin. Processes Geophys., 31, 165–174, https://doi.org/10.5194/npg-31-165-2024, https://doi.org/10.5194/npg-31-165-2024, 2024
Short summary
Short summary
Different from traditional deterministic optimization algorithms, we implement the sampling method to compute the conditional nonlinear optimal perturbations (CNOPs) in the realistic and predictive coupled ocean–atmosphere model, which reduces the first-order information to the zeroth-order one, avoiding the high-cost computation of the gradient. The numerical performance highlights the importance of stochastic optimization algorithms to compute CNOPs and capture initial optimal precursors.
David Docquier, Giorgia Di Capua, Reik V. Donner, Carlos A. L. Pires, Amélie Simon, and Stéphane Vannitsem
Nonlin. Processes Geophys., 31, 115–136, https://doi.org/10.5194/npg-31-115-2024, https://doi.org/10.5194/npg-31-115-2024, 2024
Short summary
Short summary
Identifying causes of specific processes is crucial in order to better understand our climate system. Traditionally, correlation analyses have been used to identify cause–effect relationships in climate studies. However, correlation does not imply causation, which justifies the need to use causal methods. We compare two independent causal methods and show that these are superior to classical correlation analyses. We also find some interesting differences between the two methods.
Louis Le Toumelin, Isabelle Gouttevin, Clovis Galiez, and Nora Helbig
Nonlin. Processes Geophys., 31, 75–97, https://doi.org/10.5194/npg-31-75-2024, https://doi.org/10.5194/npg-31-75-2024, 2024
Short summary
Short summary
Forecasting wind fields over mountains is of high importance for several applications and particularly for understanding how wind erodes and disperses snow. Forecasters rely on operational wind forecasts over mountains, which are currently only available on kilometric scales. These forecasts can also be affected by errors of diverse origins. Here we introduce a new strategy based on artificial intelligence to correct large-scale wind forecasts in mountains and increase their spatial resolution.
Florian Dupuy, Pierre Durand, and Thierry Hedde
Nonlin. Processes Geophys., 30, 553–570, https://doi.org/10.5194/npg-30-553-2023, https://doi.org/10.5194/npg-30-553-2023, 2023
Short summary
Short summary
Forecasting near-surface winds over complex terrain requires high-resolution numerical weather prediction models, which drastically increase the duration of simulations and hinder them in running on a routine basis. A faster alternative is statistical downscaling. We explore different ways of calculating near-surface wind speed and direction using artificial intelligence algorithms based on various convolutional neural networks in order to find the best approach for wind downscaling.
Francesco Martinuzzi, Miguel D. Mahecha, Gustau Camps-Valls, David Montero, Tristan Williams, and Karin Mora
EGUsphere, https://doi.org/10.5194/egusphere-2023-2368, https://doi.org/10.5194/egusphere-2023-2368, 2023
Short summary
Short summary
We investigated how machine learning can forecast extreme vegetation responses to weather. Examining four models, no single one stood out as the best, though "echo state networks" showed minor advantages. Our results indicate that while these tools are able to generally model vegetation states, they face challenges under extreme conditions. This underlines the potential of artificial intelligence in ecosystem modeling, also pinpointing areas that need further research.
Valérian Jacques-Dumas, René M. van Westen, Freddy Bouchet, and Henk A. Dijkstra
Nonlin. Processes Geophys., 30, 195–216, https://doi.org/10.5194/npg-30-195-2023, https://doi.org/10.5194/npg-30-195-2023, 2023
Short summary
Short summary
Computing the probability of occurrence of rare events is relevant because of their high impact but also difficult due to the lack of data. Rare event algorithms are designed for that task, but their efficiency relies on a score function that is hard to compute. We compare four methods that compute this function from data and measure their performance to assess which one would be best suited to be applied to a climate model. We find neural networks to be most robust and flexible for this task.
Domenico Giaquinto, Warner Marzocchi, and Jürgen Kurths
Nonlin. Processes Geophys., 30, 167–181, https://doi.org/10.5194/npg-30-167-2023, https://doi.org/10.5194/npg-30-167-2023, 2023
Short summary
Short summary
Despite being among the most severe climate extremes, it is still challenging to assess droughts’ features for specific regions. In this paper we study meteorological droughts in Europe using concepts derived from climate network theory. By exploring the synchronization in droughts occurrences across the continent we unveil regional clusters which are individually examined to identify droughts’ geographical propagation and source–sink systems, which could potentially support droughts’ forecast.
Joko Sampurno, Valentin Vallaeys, Randy Ardianto, and Emmanuel Hanert
Nonlin. Processes Geophys., 29, 301–315, https://doi.org/10.5194/npg-29-301-2022, https://doi.org/10.5194/npg-29-301-2022, 2022
Short summary
Short summary
In this study, we successfully built and evaluated machine learning models for predicting water level dynamics as a proxy for compound flooding hazards in a data-scarce delta. The issues that we tackled here are data scarcity and low computational resources for building flood forecasting models. The proposed approach is suitable for use by local water management agencies in developing countries that encounter these issues.
Benjamin Walleshauser and Erik Bollt
Nonlin. Processes Geophys., 29, 255–264, https://doi.org/10.5194/npg-29-255-2022, https://doi.org/10.5194/npg-29-255-2022, 2022
Short summary
Short summary
As sea surface temperature (SST) is vital for understanding the greater climate of the Earth and is also an important variable in weather prediction, we propose a model that effectively capitalizes on the reduced complexity of machine learning models while still being able to efficiently predict over a large spatial domain. We find that it is proficient at predicting the SST at specific locations as well as over the greater domain of the Earth’s oceans.
Raphael Kriegmair, Yvonne Ruckstuhl, Stephan Rasp, and George Craig
Nonlin. Processes Geophys., 29, 171–181, https://doi.org/10.5194/npg-29-171-2022, https://doi.org/10.5194/npg-29-171-2022, 2022
Short summary
Short summary
Our regional numerical weather prediction models run at kilometer-scale resolutions. Processes that occur at smaller scales not yet resolved contribute significantly to the atmospheric flow. We use a neural network (NN) to represent the unresolved part of physical process such as cumulus clouds. We test this approach on a simplified, yet representative, 1D model and find that the NN corrections vastly improve the model forecast up to a couple of days.
Bo Christiansen
Nonlin. Processes Geophys., 28, 409–422, https://doi.org/10.5194/npg-28-409-2021, https://doi.org/10.5194/npg-28-409-2021, 2021
Short summary
Short summary
In geophysics we often need to analyse large samples of high-dimensional fields. Fortunately but counterintuitively, such high dimensionality can be a blessing, and we demonstrate how this allows simple analytical results to be derived. These results include estimates of correlations between sample members and how the sample mean depends on the sample size. We show that the properties of high dimensionality with success can be applied to climate fields, such as those from ensemble modelling.
Camille Besombes, Olivier Pannekoucke, Corentin Lapeyre, Benjamin Sanderson, and Olivier Thual
Nonlin. Processes Geophys., 28, 347–370, https://doi.org/10.5194/npg-28-347-2021, https://doi.org/10.5194/npg-28-347-2021, 2021
Short summary
Short summary
This paper investigates the potential of a type of deep generative neural network to produce realistic weather situations when trained from the climate of a general circulation model. The generator represents the climate in a compact latent space. It is able to reproduce many aspects of the targeted multivariate distribution. Some properties of our method open new perspectives such as the exploration of the extremes close to a given state or how to connect two realistic weather states.
Gerd Schädler and Marcus Breil
Nonlin. Processes Geophys., 28, 231–245, https://doi.org/10.5194/npg-28-231-2021, https://doi.org/10.5194/npg-28-231-2021, 2021
Short summary
Short summary
We used regional climate networks (RCNs) to identify past heatwaves and droughts in Germany. RCNs provide information for whole areas and can provide many details of extreme events. The RCNs were constructed on the grid of the E-OBS data set. Time series correlation was used to construct the networks. Network metrics were compared to standard extreme indices and differed considerably between normal and extreme years. The results show that RCNs can identify severe and moderate extremes.
Jonathan M. Lilly and Paula Pérez-Brunius
Nonlin. Processes Geophys., 28, 181–212, https://doi.org/10.5194/npg-28-181-2021, https://doi.org/10.5194/npg-28-181-2021, 2021
Short summary
Short summary
Long-lived eddies are an important part of the ocean circulation. Here a dataset for studying eddies in the Gulf of Mexico is created through the analysis of trajectories of drifting instruments. The method involves the identification of quasi-periodic signals, characteristic of particles trapped in eddies, from the displacement records, followed by the creation of a measure of statistical significance. It is expected that this dataset will be of use to other authors studying this region.
Cristian Lussana, Thomas N. Nipen, Ivar A. Seierstad, and Christoffer A. Elo
Nonlin. Processes Geophys., 28, 61–91, https://doi.org/10.5194/npg-28-61-2021, https://doi.org/10.5194/npg-28-61-2021, 2021
Short summary
Short summary
An unprecedented amount of rainfall data is available nowadays, such as ensemble model output, weather radar estimates, and in situ observations from networks of both traditional and opportunistic sensors. Nevertheless, the exact amount of precipitation, to some extent, eludes our knowledge. The objective of our study is precipitation reconstruction through the combination of numerical model outputs with observations from multiple data sources.
Dylan Harries and Terence J. O'Kane
Nonlin. Processes Geophys., 27, 453–471, https://doi.org/10.5194/npg-27-453-2020, https://doi.org/10.5194/npg-27-453-2020, 2020
Short summary
Short summary
Different dimension reduction methods may produce profoundly different low-dimensional representations of multiscale systems. We perform a set of case studies to investigate these differences. When a clear scale separation is present, similar bases are obtained using all methods, but when this is not the case some methods may produce representations that are poorly suited for describing features of interest, highlighting the importance of a careful choice of method when designing analyses.
Jaqueline Lekscha and Reik V. Donner
Nonlin. Processes Geophys., 27, 261–275, https://doi.org/10.5194/npg-27-261-2020, https://doi.org/10.5194/npg-27-261-2020, 2020
Moritz N. Lang, Sebastian Lerch, Georg J. Mayr, Thorsten Simon, Reto Stauffer, and Achim Zeileis
Nonlin. Processes Geophys., 27, 23–34, https://doi.org/10.5194/npg-27-23-2020, https://doi.org/10.5194/npg-27-23-2020, 2020
Short summary
Short summary
Statistical post-processing aims to increase the predictive skill of probabilistic ensemble weather forecasts by learning the statistical relation between historical pairs of observations and ensemble forecasts within a given training data set. This study compares four different training schemes and shows that including multiple years of data in the training set typically yields a more stable post-processing while it loses the ability to quickly adjust to temporal changes in the underlying data.
Cited articles
Alain, G. and Bengio, Y.: What Regularized Auto-Encoders Learn from the Data-Generating Distribution, J. Mach. Learn. Res., 15, 3563–3593, 2014. a
Arcomano, T., Szunyogh, I., Pathak, J., Wikner, A., Hunt, B. R., and Ott, E.: A Machine Learning-Based Global Atmospheric Forecast Model, Geophys. Res. Lett., 47, e2020GL087776, https://doi.org/10.1029/2020GL087776, 2020. a
Arnold, H. M., Moroz, I. M., and Palmer, T. N.: Stochastic Parametrizations and Model Uncertainty in the Lorenz '96 System, Philos. T. Roy. Soc. A, 371, 20110479, https://doi.org/10.1098/rsta.2011.0479, 2013. a
Bao, F., Zhang, Z., and Zhang, G.: A Score-based Nonlinear Filter for Data Assimilation, arXiv [preprint], https://doi.org/10.48550/arXiv.2306.09282, 2023. a
Baranchuk, D., Rubachev, I., Voynov, A., Khrulkov, V., and Babenko, A.: Label-Efficient Semantic Segmentation with Diffusion Models, arXiv [preprint], https://doi.org/10.48550/arXiv.2112.03126, 2022. a
Bauer, P., Thorpe, A., and Brunet, G.: The Quiet Revolution of Numerical Weather Prediction, Nature, 525, 47–55, https://doi.org/10.1038/nature14956, 2015. a
Bauer, P., Dueben, P. D., Hoefler, T., Quintino, T., Schulthess, T. C., and Wedi, N. P.: The Digital Revolution of Earth-system Science, Nat. Comput. Sci., 1, 104–113, https://doi.org/10.1038/s43588-021-00023-0, 2021a. a
Bauer, P., Stevens, B., and Hazeleger, W.: A Digital Twin of Earth for the Green Transition, Nat. Clim. Change, 11, 80–83, https://doi.org/10.1038/s41558-021-00986-y, 2021b. a
Bengio, Y., Yao, L., Alain, G., and Vincent, P.: Generalized Denoising Auto-Encoders as Generative Models, in: Advances in Neural Information Processing Systems, vol. 26, Curran Associates, Inc., ISBN 9781713845393, 2013. a
Bishop, C. H., Etherton, B. J., and Majumdar, S. J.: Adaptive Sampling with the Ensemble Transform Kalman Filter. Part I: Theoretical Aspects, Mon. Weather Rev., 129, 420–436, https://doi.org/10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2, 2001. a
Bocquet, M.: Ensemble Kalman filtering without the intrinsic need for inflation, Nonlin. Processes Geophys., 18, 735–750, https://doi.org/10.5194/npg-18-735-2011, 2011. a, b
Bonavita, M. and Laloyaux, P.: Machine Learning for Model Error Inference and Correction, J. Adv. Model. Earth Sy., 12, e2020MS002232, https://doi.org/10.1029/2020MS002232, 2020. a
Bortoli, V. D., Thornton, J., Heng, J., and Doucet, A.: Diffusion schrödinger bridge with applications to score-based generative modeling, Advances in neural information processing systems, Curran Associates, vol. 34, 17695–17709, ISBN 9781713845393, 2021. a
Buizza, R., Houtekamer, P. L., Pellerin, G., Toth, Z., Zhu, Y., and Wei, M.: A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems, Mon. Weather Rev., 133, 1076–1097, https://doi.org/10.1175/MWR2905.1, 2005. a
Cachay, S. R., Zhao, B., James, H., and Yu, R.: DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting, arXiv [preprint], https://doi.org/10.48550/arXiv.2306.01984, 2023. a
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G.: A Simple Framework for Contrastive Learning of Visual Representations, in: Proceedings of the 37th International Conference on Machine Learning, 1597–1607, PMLR, 119, ISSN 2640-3498, 2020. a
Chen, T., Liu, G.-H., and Theodorou, E. A.: Likelihood Training of Schrödinger Bridge Using Forward-Backward SDEs Theory, arXiv [preprint], https://doi.org/10.48550/arXiv.2110.11291, 2023. a
Chen, T.-C., Penny, S. G., Whitaker, J. S., Frolov, S., Pincus, R., and Tulich, S.: Correcting Systematic and State-Dependent Errors in the NOAA FV3-GFS Using Neural Networks, J. Adv. Model. Earth Sy., 14, e2022MS003309, https://doi.org/10.1029/2022MS003309, 2022. a
De Bortoli, V.: Convergence of Denoising Diffusion Models under the Manifold Hypothesis, arXiv [preprint], https://doi.org/10.48550/arXiv.2208.05314, 2022. a
Demaeyer, J., Penny, S. G., and Vannitsem, S.: Identifying Efficient Ensemble Perturbations for Initializing Subseasonal-To-Seasonal Prediction, J. Adv. Model. Earth Sy., 14, e2021MS002828, https://doi.org/10.1029/2021MS002828, 2022. a
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186, https://doi.org/10.18653/v1/N19-1423, 2019. a
Dockhorn, T., Vahdat, A., and Kreis, K.: Score-Based Generative Modeling with Critically-Damped Langevin Diffusion, arXiv [preprint], https://doi.org/10.48550/arXiv.2112.07068, 2022. a
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., and Hon, H.-W.: Unified Language Model Pre-training for Natural Language Understanding and Generation, in: Advances in Neural Information Processing Systems, Curran Associates, Inc., vol. 32, ISBN 9781713807933, 2019. a
Efron, B.: Tweedie's Formula and Selection Bias, J. Am. Stat. Assoc., 106, 1602–1614, 2011. a
Evensen, G.: The Ensemble Kalman Filter: Theoretical Formulation and Practical Implementation, Ocean Dynam., 53, 343–367, https://doi.org/10.1007/s10236-003-0036-9, 2003. a, b
Falcon, W., Borovec, J., Wälchli, A., Eggert, N., Schock, J., Jordan, J., Skafte, N., Ir1dXD, Bereznyuk, V., Harris, E., Murrell, T., Yu, P., Præsius, S., Addair, T., Zhong, J., Lipin, D., Uchida, S., Bapat, S., Schröter, H., Dayma, B., Karnachev, A., Kulkarni, A., Komatsu, S., Martin.B, SCHIRATTI, J.-B., Mary, H., Byrne, D., Eyzaguirre, C., cinjon, and Bakhtin, A.: PyTorchLightning: 0.7.6 Release, Zenodo [code], https://doi.org/10.5281/zenodo.3828935, 2020. a
Farchi, A., Laloyaux, P., Bonavita, M., and Bocquet, M.: Using Machine Learning to Correct Model Error in Data Assimilation and Forecast Applications, Q. J. Roy. Meteor. Soc., 147, 3067–3084, https://doi.org/10.1002/qj.4116, 2021. a
Finn, T.: Ddm-attractor (Version initial_submission), Zenodo [code], https://doi.org/10.5281/zenodo.8406184, 2023. a, b
Finn, T. S., Durand, C., Farchi, A., Bocquet, M., Chen, Y., Carrassi, A., and Dansereau, V.: Deep learning subgrid-scale parametrisations for short-term forecasting of sea-ice dynamics with a Maxwell elasto-brittle rheology, The Cryosphere, 17, 2965–2991, https://doi.org/10.5194/tc-17-2965-2023, 2023. a
Gagne II, D. J., Christensen, H. M., Subramanian, A. C., and Monahan, A. H.: Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model, J. Adv. Model. Earth Sy., 12, e2019MS001896, https://doi.org/10.1029/2019MS001896, 2020. a
Grooms, I.: Analog Ensemble Data Assimilation and a Method for Constructing Analogs with Variational Autoencoders, Q. J. Roy. Meteor. Soc., 147, 139–149, https://doi.org/10.1002/qj.3910, 2021. a
Grooms, I., Renaud, C., Stanley, Z., and Yang, L. M.: Analog Ensemble Data Assimilation in a Quasigeostrophic Coupled Model, Q. J. Roy. Meteo. Soc., 149, 1018–1037, https://doi.org/10.1002/qj.4446, 2023. a
Hamill, T. M. and Snyder, C.: A Hybrid Ensemble Kalman Filter–3D Variational Analysis Scheme, Mon. Weather Rev., 128, 2905–2919, https://doi.org/10.1175/1520-0493(2000)128<2905:AHEKFV>2.0.CO;2, 2000. a
He, K., Zhang, X., Ren, S., and Sun, J.: Deep Residual Learning for Image Recognition, arXiv [preprint], https://doi.org/10.48550/arXiv.1512.03385, 2015. a
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R.: Masked Autoencoders Are Scalable Vision Learners, arXiv [preprint], https://doi.org/10.48550/arXiv.2111.06377, 2021. a
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S.: GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, arXiv [preprint], https://doi.org/10.48550/arXiv.1706.08500, 2017. a
Ho, J., Jain, A., and Abbeel, P.: Denoising diffusion probabilistic models, in: Advances in neural information processing systems, edited by: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H., Curran Associates, Inc., ISBN 9781713829546, Vol. 33, 6840–6851, 2020. a, b, c, d, e, f, g
Hoffmann, S. and Lessig, C.: AtmoDist: Self-supervised Representation Learning for Atmospheric Dynamics, Environmental Data Sci., 2, e6, https://doi.org/10.1017/eds.2023.1, 2023. a
Holzschuh, B., Vegetti, S., and Thuerey, N.: Solving inverse physics problems with score matching, in: Advances in neural information processing systems, edited by: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S., Curran Associates, Inc., Vol. 36, 61888–61922, ISBN 9781713899921, 2023. a
Hunt, B. R., Kostelich, E. J., and Szunyogh, I.: Efficient Data Assimilation for Spatiotemporal Chaos: A Local Ensemble Transform Kalman Filter, Physica D, 230, 112–126, https://doi.org/10.1016/j.physd.2006.11.008, 2007. a
Hyvärinen, A.: Estimation of Non-Normalized Statistical Models by Score Matching, J. Mach. Learn. Res., 6, 695–709, 2005. a
Jolicoeur-Martineau, A., Li, K., Piché-Taillefer, R., Kachman, T., and Mitliagkas, I.: Gotta Go Fast When Generating Data with Score-Based Models, arXiv [preprint], https://doi.org/10.48550/arXiv.2105.14080, 2022. a
Kingma, D. P. and Ba, J.: Adam: A Method for Stochastic Optimization, arXiv [preprint], https://doi.org/10.48550/arXiv.1412.6980, 2017. a
Kretschmer, M., Hunt, B. R., and Ott, E.: Data Assimilation Using a Climatologically Augmented Local Ensemble Transform Kalman Filter, Tellus A, 67, 26617, https://doi.org/10.3402/tellusa.v67.26617, 2015. a
Latif, M.: The Roadmap of Climate Models, Nat. Comput. Sci., 2, 536–538, https://doi.org/10.1038/s43588-022-00322-0, 2022. a
Lguensat, R., Tandeo, P., Ailliot, P., Pulido, M., and Fablet, R.: The Analog Data Assimilation, Mon. Weather Rev., 145, 4093–4107, https://doi.org/10.1175/MWR-D-16-0441.1, 2017. a
Li, L., Carver, R., Lopez-Gomez, I., Sha, F., and Anderson, J.: SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models, arXiv [preprint], https://doi.org/10.48550/arXiv.2306.14066, 2023a. a
Li, X., Feng, M., Ran, Y., Su, Y., Liu, F., Huang, C., Shen, H., Xiao, Q., Su, J., Yuan, S., and Guo, H.: Big Data in Earth System Science and Progress towards a Digital Twin, Nature Rev. Earth Environ., 4, 319–332, https://doi.org/10.1038/s43017-023-00409-w, 2023b. a
Lorenc, A. C.: The Potential of the Ensemble Kalman Filter for NWP – a Comparison with 4D-Var, Q. J. Roy. Meteor. Soc., 129, 3183–3203, https://doi.org/10.1256/qj.02.132, 2003. a
Lorenz, E. N.: Deterministic Nonperiodic Flow, J. Atmos. Sci., 20, 130–141, https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2, 1963. a, b
Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., and Zhu, J.: DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps, arXiv [preprint], https://doi.org/10.48550/arXiv.2206.00927, 2022. a
Luo, C.: Understanding Diffusion Models: A Unified Perspective, arXiv [preprint], https://doi.org/10.48550/arXiv.2208.11970, 2022. a
Luo, G., Dunlap, L., Park, D. H., Holynski, A., and Darrell, T.: Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence, arXiv [preprint], https://doi.org/10.48550/arXiv.2305.14334, 2023. a
Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., and Ermon, S.: SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations, arXiv [preprint], https://doi.org/10.48550/arXiv.2108.01073, 2022. a, b
Mittal, S., Abstreiter, K., Bauer, S., Schölkopf, B., and Mehrjou, A.: Diffusion Based Representation Learning, in: Proceedings of the 40th International Conference on Machine Learning, PMLR, 202, 24963–24982, ISSN 2640-3498, 2023. a
Molteni, F., Buizza, R., Palmer, T. N., and Petroliagis, T.: The ECMWF Ensemble Prediction System: Methodology and Validation, Q. J. Roy. Meteor. Soc., 122, 73–119, https://doi.org/10.1002/qj.49712252905, 1996. a
Nguyen, T., Brandstetter, J., Kapoor, A., Gupta, J. K., and Grover, A.: ClimaX: A Foundation Model for Weather and Climate, arXiv [preprint], https://doi.org/10.48550/arXiv.2301.10343, 2023. a
Oke, P. R., Allen, J. S., Miller, R. N., Egbert, G. D., and Kosro, P. M.: Assimilation of Surface Velocity Data into a Primitive Equation Coastal Ocean Model, J. Geophys. Res.-Oceans, 107, 5-1–5-25, https://doi.org/10.1029/2000JC000511, 2002. a, b
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S.: PyTorch: An Imperative Style, High-Performance Deep Learning Library, in: Advances in Neural Information Processing Systems 32, edited by: Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., and Garnett, R., Curran Associates, Inc., 8024–8035, ISBN 9781713807933, 2019. a
Peebles, W. and Xie, S.: Scalable Diffusion Models with Transformers, arXiv [preprint], https://doi.org/10.48550/arXiv.2212.09748, 2023. a
Perez, E., Strub, F., de Vries, H., Dumoulin, V., and Courville, A.: FiLM: Visual Reasoning with a General Conditioning Layer, arXiv [preprint], https://doi.org/10.48550/arXiv.1709.07871, 2017. a
Price, I., Sanchez-Gonzalez, A., Alet, F., Andersson, T. R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., Lam, R., and Willson, M.: GenCast: Diffusion-based Ensemble Forecasting for Medium-Range Weather, arXiv [preprint], https://doi.org/10.48550/arXiv.2312.15796, 2024. a
Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I.: Improving Language Understanding by Generative Pre-Training, https://openai.com/research/language-unsupervised (last access: 18 September 2024), 2018. a
Rahimi, A. and Recht, B.: Random Features for Large-Scale Kernel Machines, in: Advances in Neural Information Processing Systems, 1177–1184, ISBN 9781605603520, 2007. a
Reich, S.: Data Assimilation: The Schrödinger Perspective, Acta Numerica, 28, 635–711, https://doi.org/10.1017/S0962492919000011, 2019. a
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B.: High-Resolution Image Synthesis With Latent Diffusion Models, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, 10674–10685, https://doi.org/10.1109/CVPR52688.2022.01042, 2022. a
Rozet, F. and Louppe, G.: Score-Based Data Assimilation, Adv. Neur. Inf., 36, 40521–40541, 2023. a
Salimans, T. and Ho, J.: Progressive Distillation for Fast Sampling of Diffusion Models, arXiv [preprint], https://doi.org/10.48550/arXiv.2202.00512, 2022. a, b, c, d
Scher, S. and Messori, G.: Generalization properties of feed-forward neural networks trained on Lorenz systems, Nonlin. Processes Geophys., 26, 381–399, https://doi.org/10.5194/npg-26-381-2019, 2019. a, b
Schraff, C., Reich, H., Rhodin, A., Schomburg, A., Stephan, K., Periáñez, A., and Potthast, R.: Kilometre-Scale Ensemble Data Assimilation for the COSMO Model (KENDA), Q. J. Roy. Meteor. Soc., 142, 1453–1472, https://doi.org/10.1002/qj.2748, 2016. a
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., and Ganguli, S.: Deep Unsupervised Learning Using Nonequilibrium Thermodynamics, arXiv [preprint], https://doi.org/10.48550/arXiv.1503.03585, 2015. a
Song, Y., Garg, S., Shi, J., and Ermon, S.: Sliced Score Matching: A Scalable Approach to Density and Score Estimation, in: Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, PMLR, 115, 574–584, ISSN 2640-3498, 2019. a
Sutherland, D. J. and Schneider, J.: On the Error of Random Fourier Features, arXiv [preprint], https://doi.org/10.48550/arXiv.1506.02785, 2015. a
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A.: Going Deeper with Convolutions, arXiv [preprint], https://doi.org/10.48550/arXiv.1409.4842, 2014. a
Tandeo, P., Ailliot, P., and Sévellec, F.: Data-driven reconstruction of partially observed dynamical systems, Nonlin. Processes Geophys., 30, 129–137, https://doi.org/10.5194/npg-30-129-2023, 2023. a
Toth, Z. and Kalnay, E.: Ensemble Forecasting at NMC: The Generation of Perturbations, B. Am. Meteorol. Soc., 74, 2317–2330, https://doi.org/10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2, 1993. a
Van Rossum, G.: Python Tutorial, Technical Report CS-R9526, Tech. rep., Centrum voor Wiskunde en Informatica (CWI), Amsterdam, ISSN 0169-118X, 1995. a
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I.: Attention Is All You Need, arXiv [preprint], arXiv:1706.03762, https://doi.org/10.48550/arXiv.1706.03762, 2017. a, b
Vincent, P.: A Connection Between Score Matching and Denoising Autoencoders, Neural Comput., 23, 1661–1674, https://doi.org/10.1162/NECO_a_00142, 2011. a
Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.-A.: Extracting and Composing Robust Features with Denoising Autoencoders, Proceedings of the 25th international conference on Machine learning, 1096–1103, https://doi.org/10.1145/1390156.1390294, 2008. a
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.-A., and Bottou, L.: Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion., J. Mach. Learn. Res., 11, 3371−-3408, 2010. a
Vlachas, P. R., Pathak, J., Hunt, B. R., Sapsis, T. P., Girvan, M., Ott, E., and Koumoutsakos, P.: Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics, Neural Networks, 126, 191–217, https://doi.org/10.1016/j.neunet.2020.02.016, 2020. a
Xiang, W., Yang, H., Huang, D., and Wang, Y.: Denoising Diffusion Autoencoders Are Unified Self-supervised Learners, arXiv [preprint], https://doi.org/10.48550/arXiv.2303.09769, 2023. a
Yang, G. and Sommer, S.: A Denoising Diffusion Model for Fluid Field Prediction, arXiv [preprint], https://doi.org/10.48550/arXiv.2301.11661, 2023. a
Yang, L. M. and Grooms, I.: Machine Learning Techniques to Construct Patched Analog Ensembles for Data Assimilation, J. Comput. Phys., 443, 110532, https://doi.org/10.1016/j.jcp.2021.110532, 2021. a
Yang, X. and Wang, X.: Diffusion Model as Representation Learner, 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 18892–18903, https://doi.org/10.1109/ICCV51070.2023.01736, 2023. a
Zhang, Z., Zhao, Z., and Lin, Z.: Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models, Adv. Neur. Inf., 35, 22117–22130, 2022. a
Executive editor
This paper tests the ability of Artificial Intelligence methods, and more specifically Deep Learning, to eliminate the Gaussian noise that disturbs the data of a dynamic system. The authors demonstrate this using a highly chaotic model as a hard test case.
This paper tests the ability of Artificial Intelligence methods, and more specifically Deep...
Short summary
We train neural networks as denoising diffusion models for state generation in the Lorenz 1963 system and demonstrate that they learn an internal representation of the system. We make use of this learned representation and the pre-trained model in two downstream tasks: surrogate modelling and ensemble generation. For both tasks, the diffusion model can outperform other more common approaches. Thus, we see a potential of representation learning with diffusion models for dynamical systems.
We train neural networks as denoising diffusion models for state generation in the Lorenz 1963...
