Articles | Volume 23, issue 6
https://doi.org/10.5194/npg-23-391-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
https://doi.org/10.5194/npg-23-391-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
Research article
| 
04 Nov 2016
Research article |
| 04 Nov 2016
A local particle filter for high-dimensional geophysical systems
Stephen G. Penny
CORRESPONDING AUTHOR
Department of Atmospheric and Oceanic Science, University of Maryland, College Park, MD, USA
National Centers for Environmental Prediction, College Park, MD, USA
RIKEN Advanced Institute for Computational Science, Kobe, Japan
Takemasa Miyoshi
RIKEN Advanced Institute for Computational Science, Kobe, Japan
Department of Atmospheric and Oceanic Science, University of Maryland, College Park, MD, USA
Related authors
S. G. Penny, E. Kalnay, J. A. Carton, B. R. Hunt, K. Ide, T. Miyoshi, and G. A. Chepurin
Nonlin. Processes Geophys., 20, 1031–1046, https://doi.org/10.5194/npg-20-1031-2013, https://doi.org/10.5194/npg-20-1031-2013, 2013
Tobias Necker, David Hinger, Philipp Johannes Griewank, Takemasa Miyoshi, and Martin Weissmann
Nonlin. Processes Geophys., 30, 13–29, https://doi.org/10.5194/npg-30-13-2023, https://doi.org/10.5194/npg-30-13-2023, 2023
Short summary
Short summary
This study investigates vertical localization based on a convection-permitting 1000-member ensemble simulation. We derive an empirical optimal localization (EOL) that minimizes sampling error in 40-member sub-sample correlations assuming 1000-member correlations as truth. The results will provide guidance for localization in convective-scale ensemble data assimilation systems.
Shun Ohishi, Takemasa Miyoshi, and Misako Kachi
Geosci. Model Dev., 15, 9057–9073, https://doi.org/10.5194/gmd-15-9057-2022, https://doi.org/10.5194/gmd-15-9057-2022, 2022
Short summary
Short summary
An adaptive observation error inflation (AOEI) method was proposed for atmospheric data assimilation to mitigate erroneous analysis updates caused by large observation-minus-forecast differences for satellite brightness temperature around clear- and cloudy-sky boundaries. This study implemented the AOEI with an ocean data assimilation system, leading to an improvement of analysis accuracy and dynamical balance around the frontal regions with large meridional temperature differences.
Shun Ohishi, Tsutomu Hihara, Hidenori Aiki, Joji Ishizaka, Yasumasa Miyazawa, Misako Kachi, and Takemasa Miyoshi
Geosci. Model Dev., 15, 8395–8410, https://doi.org/10.5194/gmd-15-8395-2022, https://doi.org/10.5194/gmd-15-8395-2022, 2022
Short summary
Short summary
We develop an ensemble-Kalman-filter-based regional ocean data assimilation system in which satellite and in situ observations are assimilated at a daily frequency. We find the best setting for dynamical balance and accuracy based on sensitivity experiments focused on how to inflate the ensemble spread and how to apply the analysis update to the model evolution. This study has a broader impact on more general data assimilation systems in which the initial shocks are a significant issue.
Shunji Kotsuki, Takemasa Miyoshi, Keiichi Kondo, and Roland Potthast
Geosci. Model Dev., 15, 8325–8348, https://doi.org/10.5194/gmd-15-8325-2022, https://doi.org/10.5194/gmd-15-8325-2022, 2022
Short summary
Short summary
Data assimilation plays an important part in numerical weather prediction (NWP) in terms of combining forecasted states and observations. While data assimilation methods in NWP usually assume the Gaussian error distribution, some variables in the atmosphere, such as precipitation, are known to have non-Gaussian error statistics. This study extended a widely used ensemble data assimilation algorithm to enable the assimilation of more non-Gaussian observations.
Qiwen Sun, Takemasa Miyoshi, and Serge Richard
Nonlin. Processes Geophys. Discuss., https://doi.org/10.5194/npg-2022-12, https://doi.org/10.5194/npg-2022-12, 2022
Revised manuscript accepted for NPG
Short summary
Short summary
This paper is a follow up of a work by T. Miyoshi and Q. Sun which was published as NPG letters in 2022. The Control Simulation Experiment is applied to the Lorenz 96 model for avoiding extreme events. The results show that extreme events of this partially and imperfectly observed chaotic system can be avoided by applying pre-designed small perturbations. These investigations may be extended to more realistic numerical weather prediction models.
Takemasa Miyoshi and Qiwen Sun
Nonlin. Processes Geophys., 29, 133–139, https://doi.org/10.5194/npg-29-133-2022, https://doi.org/10.5194/npg-29-133-2022, 2022
Short summary
Short summary
The weather is chaotic and hard to predict, but the chaos implies an effective control where a small control signal grows rapidly to make a big difference. This study proposes a control simulation experiment where we apply a small signal to control
naturein a computational simulation. Idealized experiments with a low-order chaotic system show successful results by small control signals of only 3 % of the observation error. This is the first step toward realistic weather simulations.
Juan Ruiz, Guo-Yuan Lien, Keiichi Kondo, Shigenori Otsuka, and Takemasa Miyoshi
Nonlin. Processes Geophys., 28, 615–626, https://doi.org/10.5194/npg-28-615-2021, https://doi.org/10.5194/npg-28-615-2021, 2021
Short summary
Short summary
Effective use of observations with numerical weather prediction models, also known as data assimilation, is a key part of weather forecasting systems. For precise prediction at the scales of thunderstorms, fast nonlinear processes pose a grand challenge because most data assimilation systems are based on linear processes and normal distribution errors. We investigate how, every 30 s, weather radar observations can help reduce the effect of nonlinear processes and nonnormal distributions.
Keiichi Kondo and Takemasa Miyoshi
Nonlin. Processes Geophys., 26, 211–225, https://doi.org/10.5194/npg-26-211-2019, https://doi.org/10.5194/npg-26-211-2019, 2019
Short summary
Short summary
This study investigates non-Gaussian statistics of the data from a 10240-member ensemble Kalman filter. The large ensemble size can resolve the detailed structures of the probability density functions (PDFs) and indicates that the non-Gaussian PDF is caused by multimodality and outliers. While the outliers appear randomly, large multimodality corresponds well with large analysis error, mainly in the tropical regions and storm track regions where highly nonlinear processes appear frequently.
Atsushi Okazaki, Takumi Honda, Shunji Kotsuki, Moeka Yamaji, Takuji Kubota, Riko Oki, Toshio Iguchi, and Takemasa Miyoshi
Atmos. Meas. Tech., 12, 3985–3996, https://doi.org/10.5194/amt-12-3985-2019, https://doi.org/10.5194/amt-12-3985-2019, 2019
Short summary
Short summary
The JAXA is surveying the feasibility of a potential satellite mission equipped with a precipitation radar on a geostationary orbit, as a successor of the GPM Core Observatory. We investigate what kind of observation data will be available from the radar using simulation techniques. Although the quality of the observation depends on the radar specifications and the position of precipitation systems, the results demonstrate that it would be possible to obtain three-dimensional precipitation data.
Guo-Yuan Lien, Daisuke Hotta, Eugenia Kalnay, Takemasa Miyoshi, and Tse-Chun Chen
Nonlin. Processes Geophys., 25, 129–143, https://doi.org/10.5194/npg-25-129-2018, https://doi.org/10.5194/npg-25-129-2018, 2018
Short summary
Short summary
The ensemble forecast sensitivity to observation (EFSO) method can efficiently clarify under what conditions observations are beneficial or detrimental for assimilation. Based on EFSO, an offline assimilation method is proposed to accelerate the development of data selection strategies for new observing systems. The usefulness of this method is demonstrated with the assimilation of global satellite precipitation data.
Hazuki Arakida, Takemasa Miyoshi, Takeshi Ise, Shin-ichiro Shima, and Shunji Kotsuki
Nonlin. Processes Geophys., 24, 553–567, https://doi.org/10.5194/npg-24-553-2017, https://doi.org/10.5194/npg-24-553-2017, 2017
Short summary
Short summary
This is the first study assimilating the satellite-based leaf area index observations every 4 days into a numerical model simulating the growth and death of individual plants. The newly developed data assimilation system successfully reduced the uncertainties of the model parameters related to phenology and carbon dynamics. It also provides better estimates of the present vegetation structure which can be used as the initial states for the simulation of the future vegetation change.
Hisashi Yashiro, Koji Terasaki, Takemasa Miyoshi, and Hirofumi Tomita
Geosci. Model Dev., 9, 2293–2300, https://doi.org/10.5194/gmd-9-2293-2016, https://doi.org/10.5194/gmd-9-2293-2016, 2016
Short summary
Short summary
We propose the design and implementation of an ensemble data assimilation framework for weather prediction at a high resolution and with a large ensemble size. We consider the deployment of this framework on the data throughput of file I/O and multi-node communication. With regard to high-performance computing systems, where data throughput performance increases at a slower rate than computational performance, our new framework promises drastic reduction of total execution time.
X. Han, X. Li, G. He, P. Kumbhar, C. Montzka, S. Kollet, T. Miyoshi, R. Rosolem, Y. Zhang, H. Vereecken, and H.-J. H. Franssen
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmdd-8-7395-2015, https://doi.org/10.5194/gmdd-8-7395-2015, 2015
Revised manuscript not accepted
Short summary
Short summary
DasPy is a ready to use open source parallel multivariate land data assimilation framework with joint state and parameter estimation using Local Ensemble Transform Kalman Filter. The Community Land Model (4.5) was integrated as model operator. The Community Microwave Emission Modelling platform, COsmic-ray Soil Moisture Interaction Code and the Two-Source Formulation were integrated as observation operators for the multivariate assimilation of soil moisture and soil temperature, respectively.
S. G. Penny, E. Kalnay, J. A. Carton, B. R. Hunt, K. Ide, T. Miyoshi, and G. A. Chepurin
Nonlin. Processes Geophys., 20, 1031–1046, https://doi.org/10.5194/npg-20-1031-2013, https://doi.org/10.5194/npg-20-1031-2013, 2013
Related subject area
Subject: Predictability, probabilistic forecasts, data assimilation, inverse problems | Topic: Climate, atmosphere, ocean, hydrology, cryosphere, biosphere
A range of outcomes: the combined effects of internal variability and anthropogenic forcing on regional climate trends over Europe
Extending ensemble Kalman filter algorithms to assimilate observations with an unknown time offset
Guidance on how to improve vertical covariance localization based on a 1000-member ensemble
Weather pattern dynamics over western Europe under climate change: predictability, information entropy and production
Using a hybrid optimal interpolation–ensemble Kalman filter for the Canadian Precipitation Analysis
Applying prior correlations for ensemble-based spatial localization
Control Simulation Experiments of Extreme Events with the Lorenz-96 Model
A stochastic covariance shrinkage approach to particle rejuvenation in the ensemble transform particle filter
Control simulation experiment with Lorenz's butterfly attractor
Ensemble Riemannian data assimilation: towards large-scale dynamical systems
Inferring the instability of a dynamical system from the skill of data assimilation exercises
Reduced non-Gaussianity by 30 s rapid update in convective-scale numerical weather prediction
Multivariate localization functions for strongly coupled data assimilation in the bivariate Lorenz 96 system
A study of capturing Atlantic meridional overturning circulation (AMOC) regime transition through observation-constrained model parameters
Calibrated ensemble forecasts of the height of new snow using quantile regression forests and ensemble model output statistics
Enhancing geophysical flow machine learning performance via scale separation
Improving the potential accuracy and usability of EURO-CORDEX estimates of future rainfall climate using frequentist model averaging
Ensemble Riemannian data assimilation over the Wasserstein space
An early warning sign of critical transition in the Antarctic ice sheet – a data-driven tool for a spatiotemporal tipping point
Training a convolutional neural network to conserve mass in data assimilation
Behavior of the iterative ensemble-based variational method in nonlinear problems
Fast hybrid tempered ensemble transform filter formulation for Bayesian elliptical problems via Sinkhorn approximation
A methodology to obtain model-error covariances due to the discretization scheme from the parametric Kalman filter perspective
A method for predicting the uncompleted climate transition process
Statistical postprocessing of ensemble forecasts for severe weather at Deutscher Wetterdienst
Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network
From research to applications – examples of operational ensemble post-processing in France using machine learning
Correcting for model changes in statistical postprocessing – an approach based on response theory
Brief communication: Residence time of energy in the atmosphere
Simulating model uncertainty of subgrid-scale processes by sampling model errors at convective scales
Data-driven versus self-similar parameterizations for stochastic advection by Lie transport and location uncertainty
Seasonal statistical–dynamical prediction of the North Atlantic Oscillation by probabilistic post-processing and its evaluation
Application of a local attractor dimension to reduced space strongly coupled data assimilation for chaotic multiscale systems
Order of operation for multi-stage post-processing of ensemble wind forecast trajectories
Generalization properties of feed-forward neural networks trained on Lorenz systems
Revising the stochastic iterative ensemble smoother
Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data
Non-Gaussian statistics in global atmospheric dynamics: a study with a 10 240-member ensemble Kalman filter using an intermediate atmospheric general circulation model
Fluctuations of finite-time Lyapunov exponents in an intermediate-complexity atmospheric model: a multivariate and large-deviation perspective
Data assimilation using adaptive, non-conservative, moving mesh models
Data assimilation as a learning tool to infer ordinary differential equation representations of dynamical models
A Bayesian approach to multivariate adaptive localization in ensemble-based data assimilation with time-dependent extensions
Exploring the sensitivity of Northern Hemisphere atmospheric circulation to different surface temperature forcing using a statistical–dynamical atmospheric model
Review article: Comparison of local particle filters and new implementations
Data assimilation of radar reflectivity volumes in a LETKF scheme
Application of ensemble transform data assimilation methods for parameter estimation in reservoir modeling
Nonlinear effects in 4D-Var
A novel approach for solving CNOPs and its application in identifying sensitive regions of tropical cyclone adaptive observations
Chaotic dynamics and the role of covariance inflation for reduced rank Kalman filters with model error
Ensemble variational assimilation as a probabilistic estimator – Part 1: The linear and weak non-linear case
Clara Deser and Adam S. Phillips
Nonlin. Processes Geophys., 30, 63–84, https://doi.org/10.5194/npg-30-63-2023, https://doi.org/10.5194/npg-30-63-2023, 2023
Short summary
Short summary
Past and future climate change at regional scales is a result of both human influences and natural (internal) variability. Here, we provide an overview of recent advances in climate modeling and physical understanding that has led to new insights into their respective roles, illustrated with original results for the European climate. Our findings highlight the confounding role of internal variability in attribution, climate model evaluation, and accuracy of future projections.
Elia Gorokhovsky and Jeffrey L. Anderson
Nonlin. Processes Geophys., 30, 37–47, https://doi.org/10.5194/npg-30-37-2023, https://doi.org/10.5194/npg-30-37-2023, 2023
Short summary
Short summary
Older observations of the Earth system sometimes lack information about the time they were taken, posing problems for analyses of past climate. To begin to ameliorate this problem, we propose new methods of varying complexity, including methods to estimate the distribution of the offsets between true and reported observation times. The most successful method accounts for the nonlinearity in the system, but even the less expensive ones can improve data assimilation in the presence of time error.
Tobias Necker, David Hinger, Philipp Johannes Griewank, Takemasa Miyoshi, and Martin Weissmann
Nonlin. Processes Geophys., 30, 13–29, https://doi.org/10.5194/npg-30-13-2023, https://doi.org/10.5194/npg-30-13-2023, 2023
Short summary
Short summary
This study investigates vertical localization based on a convection-permitting 1000-member ensemble simulation. We derive an empirical optimal localization (EOL) that minimizes sampling error in 40-member sub-sample correlations assuming 1000-member correlations as truth. The results will provide guidance for localization in convective-scale ensemble data assimilation systems.
Stéphane Vannitsem
Nonlin. Processes Geophys., 30, 1–12, https://doi.org/10.5194/npg-30-1-2023, https://doi.org/10.5194/npg-30-1-2023, 2023
Short summary
Short summary
The impact of climate change on weather pattern dynamics over the North Atlantic is explored through the lens of information theory. These tools allow the predictability of the succession of weather patterns and the irreversible nature of the dynamics to be clarified. It is shown that the predictability is increasing in the observations, while the opposite trend is found in model projections. The irreversibility displays an overall increase in time in both the observations and the model runs.
Dikraa Khedhaouiria, Stéphane Bélair, Vincent Fortin, Guy Roy, and Franck Lespinas
Nonlin. Processes Geophys., 29, 329–344, https://doi.org/10.5194/npg-29-329-2022, https://doi.org/10.5194/npg-29-329-2022, 2022
Short summary
Short summary
This study introduces a well-known use of hybrid methods in data assimilation (DA) algorithms that has not yet been explored for precipitation analyses. Our approach combined an ensemble-based DA approach with an existing deterministically based DA. Both DA scheme families have desirable aspects that can be leveraged if combined. The DA hybrid method showed better precipitation analyses in regions with a low rate of assimilated surface observations, which is typically the case in winter.
Chu-Chun Chang and Eugenia Kalnay
Nonlin. Processes Geophys., 29, 317–327, https://doi.org/10.5194/npg-29-317-2022, https://doi.org/10.5194/npg-29-317-2022, 2022
Short summary
Short summary
This study introduces a new approach for enhancing the ensemble data assimilation (DA), a technique that combines observations and forecasts to improve numerical weather predictions. Our method uses the prescribed correlations to suppress spurious errors, improving the accuracy of DA. The experiments on the simplified atmosphere model show that our method has comparable performance to the traditional method but is superior in the early stage and is more computationally efficient.
Qiwen Sun, Takemasa Miyoshi, and Serge Richard
Nonlin. Processes Geophys. Discuss., https://doi.org/10.5194/npg-2022-12, https://doi.org/10.5194/npg-2022-12, 2022
Revised manuscript accepted for NPG
Short summary
Short summary
This paper is a follow up of a work by T. Miyoshi and Q. Sun which was published as NPG letters in 2022. The Control Simulation Experiment is applied to the Lorenz 96 model for avoiding extreme events. The results show that extreme events of this partially and imperfectly observed chaotic system can be avoided by applying pre-designed small perturbations. These investigations may be extended to more realistic numerical weather prediction models.
Andrey A. Popov, Amit N. Subrahmanya, and Adrian Sandu
Nonlin. Processes Geophys., 29, 241–253, https://doi.org/10.5194/npg-29-241-2022, https://doi.org/10.5194/npg-29-241-2022, 2022
Short summary
Short summary
Numerical weather prediction requires the melding of both computational model and data obtained from sensors such as satellites. We focus on one algorithm to accomplish this. We aim to aid its use by additionally supplying it with data obtained from separate models that describe the average behavior of the computational model at any given time. We show that our approach outperforms the standard approaches to this problem.
Takemasa Miyoshi and Qiwen Sun
Nonlin. Processes Geophys., 29, 133–139, https://doi.org/10.5194/npg-29-133-2022, https://doi.org/10.5194/npg-29-133-2022, 2022
Short summary
Short summary
The weather is chaotic and hard to predict, but the chaos implies an effective control where a small control signal grows rapidly to make a big difference. This study proposes a control simulation experiment where we apply a small signal to control
naturein a computational simulation. Idealized experiments with a low-order chaotic system show successful results by small control signals of only 3 % of the observation error. This is the first step toward realistic weather simulations.
Sagar K. Tamang, Ardeshir Ebtehaj, Peter Jan van Leeuwen, Gilad Lerman, and Efi Foufoula-Georgiou
Nonlin. Processes Geophys., 29, 77–92, https://doi.org/10.5194/npg-29-77-2022, https://doi.org/10.5194/npg-29-77-2022, 2022
Short summary
Short summary
The outputs from Earth system models are optimally combined with satellite observations to produce accurate forecasts through a process called data assimilation. Many existing data assimilation methodologies have some assumptions regarding the shape of the probability distributions of model output and observations, which results in forecast inaccuracies. In this paper, we test the effectiveness of a newly proposed methodology that relaxes such assumptions about high-dimensional models.
Yumeng Chen, Alberto Carrassi, and Valerio Lucarini
Nonlin. Processes Geophys., 28, 633–649, https://doi.org/10.5194/npg-28-633-2021, https://doi.org/10.5194/npg-28-633-2021, 2021
Short summary
Short summary
Chaotic dynamical systems are sensitive to the initial conditions, which are crucial for climate forecast. These properties are often used to inform the design of data assimilation (DA), a method used to estimate the exact initial conditions. However, obtaining the instability properties is burdensome for complex problems, both numerically and analytically. Here, we suggest a different viewpoint. We show that the skill of DA can be used to infer the instability properties of a dynamical system.
Juan Ruiz, Guo-Yuan Lien, Keiichi Kondo, Shigenori Otsuka, and Takemasa Miyoshi
Nonlin. Processes Geophys., 28, 615–626, https://doi.org/10.5194/npg-28-615-2021, https://doi.org/10.5194/npg-28-615-2021, 2021
Short summary
Short summary
Effective use of observations with numerical weather prediction models, also known as data assimilation, is a key part of weather forecasting systems. For precise prediction at the scales of thunderstorms, fast nonlinear processes pose a grand challenge because most data assimilation systems are based on linear processes and normal distribution errors. We investigate how, every 30 s, weather radar observations can help reduce the effect of nonlinear processes and nonnormal distributions.
Zofia Stanley, Ian Grooms, and William Kleiber
Nonlin. Processes Geophys., 28, 565–583, https://doi.org/10.5194/npg-28-565-2021, https://doi.org/10.5194/npg-28-565-2021, 2021
Short summary
Short summary
In weather forecasting, observations are incorporated into a model of the atmosphere through a process called data assimilation. Sometimes observations in one location may impact the weather forecast in another faraway location in undesirable ways. The impact of distant observations on the forecast is mitigated through a process called localization. We propose a new method for localization when a model has multiple length scales, as in a model spanning both the ocean and the atmosphere.
Zhao Liu, Shaoqing Zhang, Yang Shen, Yuping Guan, and Xiong Deng
Nonlin. Processes Geophys., 28, 481–500, https://doi.org/10.5194/npg-28-481-2021, https://doi.org/10.5194/npg-28-481-2021, 2021
Short summary
Short summary
A general methodology is introduced to capture regime transitions of the Atlantic meridional overturning circulation (AMOC). The assimilation models with different parameters simulate different paths for the AMOC to switch between equilibrium states. Constraining model parameters with observations can significantly mitigate the model deviations, thus capturing AMOC regime transitions. This simple model study serves as a guideline for improving coupled general circulation models.
Guillaume Evin, Matthieu Lafaysse, Maxime Taillardat, and Michaël Zamo
Nonlin. Processes Geophys., 28, 467–480, https://doi.org/10.5194/npg-28-467-2021, https://doi.org/10.5194/npg-28-467-2021, 2021
Short summary
Short summary
Forecasting the height of new snow is essential for avalanche hazard surveys, road and ski resort management, tourism attractiveness, etc. Météo-France operates a probabilistic forecasting system using a numerical weather prediction system and a snowpack model. It provides better forecasts than direct diagnostics but exhibits significant biases. Post-processing methods can be applied to provide automatic forecasting products from this system.
Davide Faranda, Mathieu Vrac, Pascal Yiou, Flavio Maria Emanuele Pons, Adnane Hamid, Giulia Carella, Cedric Ngoungue Langue, Soulivanh Thao, and Valerie Gautard
Nonlin. Processes Geophys., 28, 423–443, https://doi.org/10.5194/npg-28-423-2021, https://doi.org/10.5194/npg-28-423-2021, 2021
Short summary
Short summary
Machine learning approaches are spreading rapidly in climate sciences. They are of great help in many practical situations where using the underlying equations is difficult because of the limitation in computational power. Here we use a systematic approach to investigate the limitations of the popular echo state network algorithms used to forecast the long-term behaviour of chaotic systems, such as the weather. Our results show that noise and intermittency greatly affect the performances.
Stephen Jewson, Giuliana Barbato, Paola Mercogliano, Jaroslav Mysiak, and Maximiliano Sassi
Nonlin. Processes Geophys., 28, 329–346, https://doi.org/10.5194/npg-28-329-2021, https://doi.org/10.5194/npg-28-329-2021, 2021
Short summary
Short summary
Climate model simulations are uncertain. In some cases this makes it difficult to know how to use them. Significance testing is often used to deal with this issue but has various shortcomings. We describe two alternative ways to manage uncertainty in climate model simulations that avoid these shortcomings. We test them on simulations of future rainfall over Europe and show they produce more accurate projections than either using unadjusted climate model output or statistical testing.
Sagar K. Tamang, Ardeshir Ebtehaj, Peter J. van Leeuwen, Dongmian Zou, and Gilad Lerman
Nonlin. Processes Geophys., 28, 295–309, https://doi.org/10.5194/npg-28-295-2021, https://doi.org/10.5194/npg-28-295-2021, 2021
Short summary
Short summary
Data assimilation aims to improve hydrologic and weather forecasts by combining available information from Earth system models and observations. The classical approaches to data assimilation usually proceed with some preconceived assumptions about the shape of their probability distributions. As a result, when such assumptions are invalid, the forecast accuracy suffers. In the proposed methodology, we relax such assumptions and demonstrate improved performance.
Abd AlRahman AlMomani and Erik Bollt
Nonlin. Processes Geophys., 28, 153–166, https://doi.org/10.5194/npg-28-153-2021, https://doi.org/10.5194/npg-28-153-2021, 2021
Short summary
Short summary
This paper introduces a tool for data-driven discovery of early warning signs of critical transitions in ice shelves from remote sensing data. Our directed spectral clustering method considers an asymmetric affinity matrix along with the associated directed graph Laplacian. We applied our approach to reprocessing the ice velocity data and remote sensing satellite images of the Larsen C ice shelf.
Yvonne Ruckstuhl, Tijana Janjić, and Stephan Rasp
Nonlin. Processes Geophys., 28, 111–119, https://doi.org/10.5194/npg-28-111-2021, https://doi.org/10.5194/npg-28-111-2021, 2021
Short summary
Short summary
The assimilation of observations using standard algorithms can lead to a violation of physical laws (e.g. mass conservation), which is shown to have a detrimental impact on the system's forecast. We use a neural network (NN) to correct this mass violation, using training data generated from expensive algorithms that can constrain such physical properties. We found that, in an idealized set-up, the NN can match the performance of these expensive algorithms at negligible computational costs.
Shin'ya Nakano
Nonlin. Processes Geophys., 28, 93–109, https://doi.org/10.5194/npg-28-93-2021, https://doi.org/10.5194/npg-28-93-2021, 2021
Short summary
Short summary
The ensemble-based variational method is a method for solving nonlinear data assimilation problems by using an ensemble of multiple simulation results. Although this method is derived based on a linear approximation, highly uncertain problems, in which system nonlinearity is significant, can also be solved by applying this method iteratively. This paper reformulated this iterative algorithm to analyze its behavior in high-dimensional nonlinear problems and discuss the convergence.
Sangeetika Ruchi, Svetlana Dubinkina, and Jana de Wiljes
Nonlin. Processes Geophys., 28, 23–41, https://doi.org/10.5194/npg-28-23-2021, https://doi.org/10.5194/npg-28-23-2021, 2021
Short summary
Short summary
To infer information of an unknown quantity that helps to understand an associated system better and to predict future outcomes, observations and a physical model that connects the data points to the unknown parameter are typically used as information sources. Yet this problem is often very challenging due to the fact that the unknown is generally high dimensional, the data are sparse and the model can be non-linear. We propose a novel approach to address these challenges.
Olivier Pannekoucke, Richard Ménard, Mohammad El Aabaribaoune, and Matthieu Plu
Nonlin. Processes Geophys., 28, 1–22, https://doi.org/10.5194/npg-28-1-2021, https://doi.org/10.5194/npg-28-1-2021, 2021
Short summary
Short summary
Numerical weather prediction involves numerically solving the mathematical equations, which describe the geophysical flow, by transforming them so that they can be computed. Through this transformation, it appears that the equations actually solved by the machine are then a modified version of the original equations, introducing an error that contributes to the model error. This work helps to characterize the covariance of the model error that is due to this modification of the equations.
Pengcheng Yan, Guolin Feng, Wei Hou, and Ping Yang
Nonlin. Processes Geophys., 27, 489–500, https://doi.org/10.5194/npg-27-489-2020, https://doi.org/10.5194/npg-27-489-2020, 2020
Short summary
Short summary
A system transiting from one stable state to another has to experience a period. Can we predict the end moment (state) if the process has not been completed? This paper presents a method to solve this problem. This method is based on the quantitative relationship among the parameters, which is used to describe the transition process of the abrupt change. By using the historical data, we extract some parameters for predicting the uncompleted climate transition process.
Reinhold Hess
Nonlin. Processes Geophys., 27, 473–487, https://doi.org/10.5194/npg-27-473-2020, https://doi.org/10.5194/npg-27-473-2020, 2020
Short summary
Short summary
Forecasts of ensemble systems are statistically aligned to synoptic observations at DWD in order to provide support for warning decision management. Motivation and design consequences for extreme and rare meteorological events are presented. Especially for probabilities of severe wind gusts global logistic parameterisations are developed that generate robust statistical forecasts for extreme events, while local characteristics are preserved.
Ashesh Chattopadhyay, Pedram Hassanzadeh, and Devika Subramanian
Nonlin. Processes Geophys., 27, 373–389, https://doi.org/10.5194/npg-27-373-2020, https://doi.org/10.5194/npg-27-373-2020, 2020
Short summary
Short summary
The performance of three machine-learning methods for data-driven modeling of a multiscale chaotic Lorenz 96 system is examined. One of the methods is found to be able to predict the future evolution of the chaotic system well from just knowing the past observations of the large-scale component of the multiscale state vector. Potential applications to data-driven and data-assisted surrogate modeling of complex dynamical systems such as weather and climate are discussed.
Maxime Taillardat and Olivier Mestre
Nonlin. Processes Geophys., 27, 329–347, https://doi.org/10.5194/npg-27-329-2020, https://doi.org/10.5194/npg-27-329-2020, 2020
Short summary
Short summary
Statistical post-processing of ensemble forecasts is now a well-known procedure in order to correct biased and misdispersed ensemble weather predictions. But practical application in European national weather services is in its infancy. Different applications of ensemble post-processing using machine learning at an industrial scale are presented. Forecast quality and value are improved compared to the raw ensemble, but several facilities have to be made to adjust to operational constraints.
Jonathan Demaeyer and Stéphane Vannitsem
Nonlin. Processes Geophys., 27, 307–327, https://doi.org/10.5194/npg-27-307-2020, https://doi.org/10.5194/npg-27-307-2020, 2020
Short summary
Short summary
Postprocessing schemes used to correct weather forecasts are no longer efficient when the model generating the forecasts changes. An approach based on response theory to take the change into account without having to recompute the parameters based on past forecasts is presented. It is tested on an analytical model and a simple model of atmospheric variability. We show that this approach is effective and discuss its potential application for an operational environment.
Carlos Osácar, Manuel Membrado, and Amalio Fernández-Pacheco
Nonlin. Processes Geophys., 27, 235–237, https://doi.org/10.5194/npg-27-235-2020, https://doi.org/10.5194/npg-27-235-2020, 2020
Short summary
Short summary
We deduce that after a global thermal perturbation, the Earth's
atmosphere would need about a couple of months to come back to equilibrium.
Michiel Van Ginderachter, Daan Degrauwe, Stéphane Vannitsem, and Piet Termonia
Nonlin. Processes Geophys., 27, 187–207, https://doi.org/10.5194/npg-27-187-2020, https://doi.org/10.5194/npg-27-187-2020, 2020
Short summary
Short summary
A generic methodology is developed to estimate the model error and simulate the model uncertainty related to a specific physical process. The method estimates the model error by comparing two different representations of the physical process in otherwise identical models. The found model error can then be used to perturb the model and simulate the model uncertainty. When applying this methodology to deep convection an improvement in the probabilistic skill of the ensemble forecast is found.
Valentin Resseguier, Wei Pan, and Baylor Fox-Kemper
Nonlin. Processes Geophys., 27, 209–234, https://doi.org/10.5194/npg-27-209-2020, https://doi.org/10.5194/npg-27-209-2020, 2020
Short summary
Short summary
Geophysical flows span a broader range of temporal and spatial scales than can be resolved numerically. One way to alleviate the ensuing numerical errors is to combine simulations with measurements, taking account of the accuracies of these two sources of information. Here we quantify the distribution of numerical simulation errors without relying on high-resolution numerical simulations. Specifically, small-scale random vortices are added to simulations while conserving energy or circulation.
André Düsterhus
Nonlin. Processes Geophys., 27, 121–131, https://doi.org/10.5194/npg-27-121-2020, https://doi.org/10.5194/npg-27-121-2020, 2020
Short summary
Short summary
Seasonal prediction of the of the North Atlantic Oscillation (NAO) has been improved in recent years by improving dynamical models and ensemble predictions. One step therein was the so-called sub-sampling, which combines statistical and dynamical predictions. This study generalises this approach and makes it much more accessible. Furthermore, it presents a new verification approach for such predictions.
Courtney Quinn, Terence J. O'Kane, and Vassili Kitsios
Nonlin. Processes Geophys., 27, 51–74, https://doi.org/10.5194/npg-27-51-2020, https://doi.org/10.5194/npg-27-51-2020, 2020
Short summary
Short summary
This study presents a novel method for reduced-rank data assimilation of multiscale highly nonlinear systems. Time-varying dynamical properties are used to determine the rank and projection of the system onto a reduced subspace. The variable reduced-rank method is shown to succeed over other fixed-rank methods. This work provides implications for performing strongly coupled data assimilation with a limited number of ensemble members on high-dimensional coupled climate models.
Nina Schuhen
Nonlin. Processes Geophys., 27, 35–49, https://doi.org/10.5194/npg-27-35-2020, https://doi.org/10.5194/npg-27-35-2020, 2020
Short summary
Short summary
We present a new way to adaptively improve weather forecasts by incorporating last-minute observation information. The recently measured error, together with a statistical model, gives us an indication of the expected future error of wind speed forecasts, which are then adjusted accordingly. This new technique can be especially beneficial for customers in the wind energy industry, where it is important to have reliable short-term forecasts, as well as providers of extreme weather warnings.
Sebastian Scher and Gabriele Messori
Nonlin. Processes Geophys., 26, 381–399, https://doi.org/10.5194/npg-26-381-2019, https://doi.org/10.5194/npg-26-381-2019, 2019
Short summary
Short summary
Neural networks are a technique that is widely used to predict the time evolution of physical systems. For this the past evolution of the system is shown to the neural network – it is
trained– and then can be used to predict the evolution in the future. We show some limitations in this approach for certain systems that are important to consider when using neural networks for climate- and weather-related applications.
Patrick Nima Raanes, Andreas Størksen Stordal, and Geir Evensen
Nonlin. Processes Geophys., 26, 325–338, https://doi.org/10.5194/npg-26-325-2019, https://doi.org/10.5194/npg-26-325-2019, 2019
Short summary
Short summary
A popular variational ensemble smoother for data assimilation and history matching is simplified. An exact relationship between ensemble linearizations (linear regression) and adjoints (analytic derivatives) is established.
Fei Lu, Nils Weitzel, and Adam H. Monahan
Nonlin. Processes Geophys., 26, 227–250, https://doi.org/10.5194/npg-26-227-2019, https://doi.org/10.5194/npg-26-227-2019, 2019
Short summary
Short summary
ll-posedness of the inverse problem and sparse noisy data are two major challenges in the modeling of high-dimensional spatiotemporal processes. We present a Bayesian inference method with a strongly regularized posterior to overcome these challenges, enabling joint state-parameter estimation and quantifying uncertainty in the estimation. We demonstrate the method on a physically motivated nonlinear stochastic partial differential equation arising from paleoclimate construction.
Keiichi Kondo and Takemasa Miyoshi
Nonlin. Processes Geophys., 26, 211–225, https://doi.org/10.5194/npg-26-211-2019, https://doi.org/10.5194/npg-26-211-2019, 2019
Short summary
Short summary
This study investigates non-Gaussian statistics of the data from a 10240-member ensemble Kalman filter. The large ensemble size can resolve the detailed structures of the probability density functions (PDFs) and indicates that the non-Gaussian PDF is caused by multimodality and outliers. While the outliers appear randomly, large multimodality corresponds well with large analysis error, mainly in the tropical regions and storm track regions where highly nonlinear processes appear frequently.
Frank Kwasniok
Nonlin. Processes Geophys., 26, 195–209, https://doi.org/10.5194/npg-26-195-2019, https://doi.org/10.5194/npg-26-195-2019, 2019
Short summary
Short summary
The stability properties as characterized by finite-time Lyapunov exponents are investigated in an intermediate-complexity atmospheric model. Firstly, the dominant patterns of collective excitation are identified by an empirical orthogonal function analysis of the fluctuation field of all of the finite-time Lyapunov exponents. Secondly, a large-deviation principle is established for all of the Lyapunov exponents and the large-deviation rate functions are estimated.
Ali Aydoğdu, Alberto Carrassi, Colin T. Guider, Chris K. R. T Jones, and Pierre Rampal
Nonlin. Processes Geophys., 26, 175–193, https://doi.org/10.5194/npg-26-175-2019, https://doi.org/10.5194/npg-26-175-2019, 2019
Short summary
Short summary
Computational models involving adaptive meshes can both evolve dynamically and be remeshed. Remeshing means that the state vector dimension changes in time and across ensemble members, making the ensemble Kalman filter (EnKF) unsuitable for assimilation of observational data. We develop a modification in which analysis is performed on a fixed uniform grid onto which the ensemble is mapped, with resolution relating to the remeshing criteria. The approach is successfully tested on two 1-D models.
Marc Bocquet, Julien Brajard, Alberto Carrassi, and Laurent Bertino
Nonlin. Processes Geophys., 26, 143–162, https://doi.org/10.5194/npg-26-143-2019, https://doi.org/10.5194/npg-26-143-2019, 2019
Short summary
Short summary
This paper describes an innovative way to use data assimilation to infer the dynamics of a physical system from its observation only. The method can operate with noisy and partial observation of the physical system. It acts as a deep learning technique specialised to dynamical models without the need for machine learning tools. The method is successfully tested on chaotic dynamical systems: the Lorenz-63, Lorenz-96, and Kuramoto–Sivashinski models and a two-scale Lorenz model.
Andrey A. Popov and Adrian Sandu
Nonlin. Processes Geophys., 26, 109–122, https://doi.org/10.5194/npg-26-109-2019, https://doi.org/10.5194/npg-26-109-2019, 2019
Short summary
Short summary
This work has to do with a small part of existing algorithms that are used in applications such as predicting the weather. We provide empirical evidence that our new technique works well on small but representative models. This might lead to creation of a better weather forecast and potentially save lives as in the case of hurricane prediction.
Sonja Totz, Stefan Petri, Jascha Lehmann, Erik Peukert, and Dim Coumou
Nonlin. Processes Geophys., 26, 1–12, https://doi.org/10.5194/npg-26-1-2019, https://doi.org/10.5194/npg-26-1-2019, 2019
Alban Farchi and Marc Bocquet
Nonlin. Processes Geophys., 25, 765–807, https://doi.org/10.5194/npg-25-765-2018, https://doi.org/10.5194/npg-25-765-2018, 2018
Short summary
Short summary
Data assimilation looks for an optimal way to learn from observations of a dynamical system to improve the quality of its predictions. The goal is to filter out the noise (both observation and model noise) to retrieve the true signal. Among all possible methods, particle filters are promising; the method is fast and elegant, and it allows for a Bayesian analysis. In this review paper, we discuss implementation techniques for (local) particle filters in high-dimensional systems.
Thomas Gastaldo, Virginia Poli, Chiara Marsigli, Pier Paolo Alberoni, and Tiziana Paccagnella
Nonlin. Processes Geophys., 25, 747–764, https://doi.org/10.5194/npg-25-747-2018, https://doi.org/10.5194/npg-25-747-2018, 2018
Short summary
Short summary
Accuracy of numerical weather prediction forecasts is strongly related to the quality of initial conditions employed. To improve them, it seems advantageous to use radar reflectivity observations because of their high spatial and temporal resolution. This is tested in a high-resolution model whose domain covers Italy. Results show that the employment of reflectivity observations improves precipitation forecast accuracy, but the positive impact is lost after a few hours of forecast.
Sangeetika Ruchi and Svetlana Dubinkina
Nonlin. Processes Geophys., 25, 731–746, https://doi.org/10.5194/npg-25-731-2018, https://doi.org/10.5194/npg-25-731-2018, 2018
Short summary
Short summary
Accurate estimation of subsurface geological parameters is essential for the oil industry. This is done by combining observations with an estimation from a model. Ensemble Kalman filter is a widely used method for inverse modeling, while ensemble transform particle filtering is a recently developed method that has been applied to estimate only a small number of parameters and in fluids. We show that for a high-dimensional inverse problem it is superior to an ensemble Kalman filter.
Massimo Bonavita, Peter Lean, and Elias Holm
Nonlin. Processes Geophys., 25, 713–729, https://doi.org/10.5194/npg-25-713-2018, https://doi.org/10.5194/npg-25-713-2018, 2018
Short summary
Short summary
This paper deals with the effects of nonlinearity in a state-of-the-art atmospheric global data assimilation system. It is shown that these effects have become increasingly important over the years due to increased model resolution and use of nonlinear observations. The ability to deal with nonlinearities has thus become a crucial asset of data assimilation algorithms. At ECMWF this is done in a perturbative fashion. Advantages and limitations of this technique are discussed.
Linlin Zhang, Bin Mu, Shijin Yuan, and Feifan Zhou
Nonlin. Processes Geophys., 25, 693–712, https://doi.org/10.5194/npg-25-693-2018, https://doi.org/10.5194/npg-25-693-2018, 2018
Short summary
Short summary
We propose a novel approach to solve conditional nonlinear optimal perturbation for identifying sensitive areas for tropical cyclone adaptive observations. This method is free of adjoint models and overcomes two obstacles, not having adjoint models and having dimensions higher than the problem space. All experimental results prove that it is a meaningful and effective method for solving CNOP and provides a new way for such research. This work aims to solve CNOP and identify sensitive areas.
Colin Grudzien, Alberto Carrassi, and Marc Bocquet
Nonlin. Processes Geophys., 25, 633–648, https://doi.org/10.5194/npg-25-633-2018, https://doi.org/10.5194/npg-25-633-2018, 2018
Short summary
Short summary
Using the framework Lyapunov vectors, we analyze the asymptotic properties of ensemble based Kalman filters and how these are influenced by dynamical chaos, especially in the context of random model errors and small ensemble sizes. Particularly, we show a novel derivation of the evolution of forecast uncertainty for ensemble-based Kalman filters with weakly-nonlinear error growth, and discuss its impact for filter design in geophysical models.
Mohamed Jardak and Olivier Talagrand
Nonlin. Processes Geophys., 25, 565–587, https://doi.org/10.5194/npg-25-565-2018, https://doi.org/10.5194/npg-25-565-2018, 2018
Short summary
Short summary
Ensemble variational assimilation (EnsVAR) has been implemented on two small-dimension non-linear chaotic toy models, as well as on a linearized version of those models. In the linear case, EnsVAR is exactly Bayesian and produced
highly reliable ensembles. In the non-linear case, EnsVAR, implemented on temporal windows on the order of magnitude of the predictability time of the systems, shows as good performance as in the exactly linear case. EnsVar is as good an estimator as EnKF and PF.
Cited articles
Abarbanel, H. D. I., Creveling, D. R., Farsian, R., and Kostuk, M.: Dynamical State and Parameter Estimation, SIAM J. Appl. Dyn. Syst., 8, 1341–1381, https://doi.org/10.1137/090749761, 2009.
Ades, M. and van Leeuwen, P. J.: An exploration of the equivalent weights particle filter, Q. J. Roy. Meteorol. Soc., 139, 820–840, 2013.
Anderson, J.: An ensemble adjustment kalman filter for data assimilation, Mon. Weather Rev., 129, 2884–2903, 2001.
Atkins, E., Morzfeld, M., and Chorin, A. J.: Implicit Particle Methods and their Connection with Variational Data Assimilation, Mon. Weather Rev., 141, 1786–1803, 2013.
Bengtsson, T., Snyder, C., and Nychka, D.: Toward a nonlinear ensemble filter for high-dimensional systems, J. Geophys. Res., 108, STS2.1–STS2.10, 2003.
Berner, J., Shutts, G. J., Leutbecher, M., and Palmer, T. N.: A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system, J. Atmos. Sci., 66, 603–626, https://doi.org/10.1175/2008JAS2677.1, 2009.
Beskos, A., Crisan, D., and Jasra, A.: On the Stability of Sequential Monte Carlo Methods in High Dimensions, arXiv:1103.3965v2 [stat.CO], 2012.
Bishop, C. H., Etherton, B. J., and Majumdar, S. J.: Adaptive Sampling with Ensemble Transform Kalman Filter, Part I: Theoretical Aspects, Mon. Weather Rev., 129, 420–436, 2001.
Bocquet, M., Pires, C. A., and Wu, L.: Beyond Gaussian Statistical Modeling in Geophysical Data Assimilation, Mon. Weather Rev., 138, 2997–3023, https://doi.org/10.1175/2010MWR3164.1, 2010.
Bontempi, G.: Simulating continuous dynamical systems under conditions of uncertainty: the probability and the possibility approaches. Fuzzy Partial Differential Equations and Relational Equations, in: Vol. 142 of the series Studies in Fuzziness and Soft Computing, 130–152, https://doi.org/10.1007/978-3-540-39675-8_4, available at: http://www.ulb.ac.be/di/map/gbonte/ftp/bontempi_fpde.pdf (last access: November 2016), 2004.
Bowler, N.: Comparison of error breeding, singular vectors, random perturbations and ensemble Kalman filter perturbation strategies on a simple model, Tellus A, 58, 538–548, 2006.
Doucet, A., De Freitas, N., and Gordon, N. J.: An introduction to Sequential Monte Carlo Methods, in: SMC in Practice, http://www.stats.ox.ac.uk/~doucet/smc_resources.html (last access: 31 October 2016), 2001.
Evenson, G.: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res., 99, 10143–10162, 1994.
Fowler, A. and van Leeuwen, N. J.: Observation impact in data assimilation: the effect of non-Gaussian observation error, Tellus A, 65, 20035, https://doi.org/10.3402/tellusa.v65i0.20035, 2013.
Gordon, N. J., Salmond, D., and Smith, A. F. M.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation, IEEE Proc.-F., 140, 107, https://doi.org/10.1049/ip-f-2.1993.0015, 1993.
Hamrud, M., Bonavita, M., and Isaksen, L.: EnKF and Hybrid Gain Ensemble Data Assimilation, ECMWF Technical Report #733, http://www.ecmwf.int/sites/default/files/elibrary/2014/9766-enkf-and-hybrid-gain-ensemble-data-assimilation.pdf (last access: 31 October 2016), 2014.
Hou, D., Toth, Z., and Zhu, Y.: A Stochastic Parameterization Scheme Within NCEP Global Ensemble Forecast System, in: 18th Conference on Probability and Statistics in the Atmospheric Sciences, Am. Met. Soc., http://ams.confex.com/ams/pdfpapers/101401.pdf (last access: 31 October 2016), 2006.
Hou, D., Toth, Z., Zhu, Y., Yang, W., and Wobus, R.: A Stochastic Total Tendency Perturbation Scheme Representing Model-Related Uncertainties in the NCEP Global Ensemble Forecast System, NOAA/NCEP/EMC, http://www.emc.ncep.noaa.gov/gmb/yzhu/gif/pub/Manuscript_STTP_Tellus_A_HOU-1.pdf (last access: 31 October 2016), 2010.
Houtekamer, P. L. and Mitchell, H. L.: Data Assimilation Using an Ensemble Kalman Filter Technique, Mon. Weather Rev., 126, 796–811, https://doi.org/10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2, 1998.
Hunt, B. R., Kostelich, E. J., and Szunyogh, I.: Efficient Data Assimilation for Spatiotemporal Chaos: A Local Ensemble Transform Kalman Filter, Physica D, 230, 112–126, 2007.
Hurzeler, M. and Kunsch, H.: Monte Carlo approximations for general state-space models, J. Comput. Graph. Stat., 7, 175–193, 1998.
Jardak, M., Navon, I. M., and Zupanski, M.: Comparison of sequential data assimilation methods for the Kuramoto–Sivashinsky equation, Int. J. Num. Meth. Fluids, 62, 374–402, 2010.
Kitagawa, G.: Monte Carlo filter and smoother for non-Gaussian non-linear state space models, J. Comput. Graph. Stat., 5, 1–25, 1996.
Kleist, D. T.: An evaluation of hybrid variational-ensemble data assimilation for the NCEP GFS, PhD dissertation, University of Maryland, Maryland, 2012.
Kolczynski, W., Pegion, P., Hamill, T., Whitaker, J. S., Hou, D., Zhu, Y., and Zhou, X.: Investigating a New Stochastic Physics Suite for Use in the NCEP Global Ensemble, in: 27th Conference On Weather Analysis And Forecasting/23rd Conference On Numerical Weather Prediction, Am. Met. Soc., https://ams.confex.com/ams/27WAF23NWP/webprogram/Paper273838.html (last access: 31 October 2016), 2015.
Lei, J. and Bickel, P.: A Moment Matching Ensemble Filter for Nonlinear Non-Gaussian Data Assimilation, Mon. Weather Rev., 139, 3964–3973, 2011.
Lien, G.-Y., Kalnay, E., and Miyoshi, T.: Effective assimilation of global precipitation: Simulation experiments, Tellus A, 65, 19915, https://doi.org/10.3402/tellusa.v65i0.19915, 2013.
Lien, G.-Y., Miyoshi, T., and Kalnay, E.: Assimilation of TRMM Multisatellite Precipitation Analysis with a low-resolution NCEP Global Forecasting System, Mon. Weather Rev., 144, 643–661, https://doi.org/10.1175/MWR-D-15-0149.1, 2016.
Liu, J. S. and Chen, R.: Sequential Monte Carlo methods for dynamic systems, J. Am. Stat. Assoc., 93, 1032–1044, 1998.
Lorenz, E. N.: Predictability – A problem partly solved, Proceedings of a Seminar Held at ECMWF on Predictability, ECMWF Seminar Proceedings, Vol. 1, ECMWF, 1–18, 1996.
Metref, S., Cosme, E., Snyder, C., and Brasseur, P.: A non-Gaussian analysis scheme using rank histograms for ensemble data assimilation, Nonlin. Processes Geophys., 21, 869–885, https://doi.org/10.5194/npg-21-869-2014, 2014.
Miyamoto, Y., Kajikawa, Y., Yoshida, R., Yamaura, T., Yashiro, H., and Tomita, H.: Deep moist atmospheric convection in a subkilometer global simulation, Geophys. Res. Lett., 40, 4922–4926, https://doi.org/10.1002/grl.50944, 2013.
Miyoshi, T., Kondo, K., and Imamura, T.: The 10240-member ensemble Kalman filtering with an intermediate AGCM, Geophys. Res. Lett., 41, 5264–5271, https://doi.org/10.1002/2014GL060863, 2014.
Miyoshi, T., Kondo, K., and Terasaki, K.: Big Ensemble Data Assimilation in Numerical Weather Prediction, Computer, 48, https://doi.org/10.1109/MC.2015.332, 2015.
Nakano, S., Ueno, G., and Higuchi, T.: Merging particle filter for sequential data assimilation, Nonlin. Processes Geophys., 14, 395–408, https://doi.org/10.5194/npg-14-395-2007, 2007.
Nerger, L.: On Serial Observation Processing in Localized Ensemble Kalman Filters, Mon. Weather Rev., 143, 1554–1567, https://doi.org/10.1175/MWR-D-14-00182.1, 2015.
Ochatta, T., Gebhardt, C., Saupe, D., and Wergen, W.: Adaptive thinning of atmospheric observations in data assimilation with vector quantization and filtering methods, Q. J. Roy. Meteorol. Soc., 131, 3427–3437, 2005.
Pazo, D., Rodriguez, M. A., and Lopez, J. M.: Spatio-temporal evolution of perturbations in ensembles initialized by bred, Lyapunov, and singular vectors, Tellus A, 62, 10–23, 2010.
Penny, S. G.: The Hybrid Local Ensemble Transform Kalman Filter, Mon. Weather Rev., 142, 2139–2149, https://doi.org/10.1175/MWR-D-13-00131.1, 2014.
Penny, S. G., Kalnay, E., Carton, J. A., Hunt, B. R., Ide, K., Miyoshi, T., and Chepurin, G. A.: The local ensemble transform Kalman filter and the running-in-place algorithm applied to a global ocean general circulation model, Nonlin. Processes Geophys., 20, 1031–1046, https://doi.org/10.5194/npg-20-1031-2013, 2013.
Penny, S. G., Behringer, D., Carton, J. A., and Kalnay, E.: A Hybrid Global Ocean Data Assimilation System at NCEP, Mon. Weather Rev., 143, 4660–4677, https://doi.org/10.1175/MWR-D-14-00376.1, 2015.
Reich, S.: A nonparametric ensemble transform method for Bayesian inference, SIAM J. Sci. Comput., 35, A2013–A2024, https://doi.org/10.1137/130907367, 2013.
Sanchez, C., Williams, K. D., Shutts, G., and Collins, M.: Impact of a Stochastic Kinetic Energy Backscatter scheme across time-scales and resolutions, Q. J. Roy. Meteorol. Soc., 140, 2625–2637, 2014.
Shutts, G.: A stochastic convective backscatter scheme for use in ensemble prediction systems, Q. J. Roy. Meteorol. Soc., 141, 2602–2616, https://doi.org/10.1002/qj.2547, 2015.
Shutts, G. and Pallarès, A. C.: Assessing parametrization uncertainty associated with horizontal resolution in numerical weather prediction models, Philos. T. Roy. Soc. A, 372, 20130284, https://doi.org/10.1098/rsta.2013.0284, 2014.
Snyder, C., Bengtsson, T., Bickel, P., and Anderson, J.: Obstacles to High-Dimensional Particle Filtering, Mon. Weather Rev., 136, 4629–4640, 2008.
Snyder, C., Bengtsson, T., and Morzfeld, M.: Performance bounds for particle filters using the optimal proposal, Mon. Weather Rev., 143, 4750–4761, https://doi.org/10.1175/MWR-D-15-0144.1, 2015.
Stewart, L. and McCarty, P.: The use of Bayesian Belief Networks to fuse continuous and discrete information for target recognition and discrete information for target recognition, tracking, and situation assessment, in: Proc. SPIE, 1699, 177–185, 1992.
Tennant, W. J., Shutts, G. J., Arribas, A., and Thompson, S. A.: Using a Stochastic Kinetic Energy Backscatter Scheme to Improve MOGREPS Probabilistic Forecast Skill, Mon. Weather Rev., 139, 1190–1206, https://doi.org/10.1175/2010MWR3430.1, 2011.
Tippett, M. K., Anderson, J. L., Bishop, C. H., Hamill, T. M., and Whitaker, J. S.: Ensemble square root filters, Mon. Weather Rev., 131, 1485–1490, 2003.
van Leeuwen, P. J.: A variance-minimizing filter for large-scale applications, Mon. Weather Rev., 131, 2071–2084, 2003.
van Leeuwen, P. J.: Nonlinear data assimilation in geosciences: an extremely efficient particle filter, Q. J. Roy. Meteorol. Soc., 136, 1991–1999, 2010.
Watson, P. A. G., Christensen, H. M., and Palmer, T. N.: Does the ECMWF IFS Convection Parameterization with Stochastic Physics Correctly Reproduce Relationships between Convection and the Large-Scale State?, J. Atmos. Sci., 72, 236–242, https://doi.org/10.1175/JAS-D-14-0252.1, 2015.
Weisheimer, A., Corti, S., Palmer, T., and Vitart, F.: Addressing model error through atmospheric stochastic physical parametrizations: impact on the coupled ECMWF seasonal forecasting system, Philos. T. Roy. Soc. A, 372, 20130290, https://doi.org/10.1098/rsta.2013.0290, 2014.
Whartenby, W. G., Quinn, J. C., and Abarbanel, H. D. I.: The number of required observations in data assimilation for a shallow-water flow, Mon. Weather Rev., 141, 2502–2518, 2013.
Whitaker, J. S. and Hamill, T. M.: Ensemble Data Assimilation without Perturbed Observations, Mon. Weather Rev., 130, 1913–1924, 2002.
Xiong, X., Navon, I. M., and Uzunoglu, B.: A note on the particle filter with posterior Gaussian resampling, Tellus A, 58, 456–460, 2006.
Yang, S.-C., Kalnay, E., and Hunt, B. R.: Weight interpolation for efficient data assimilation with the Local Ensemble Transform Kalman Filter, Q. J. Roy. Meteorol. Soc., 135, 251–262, 2009.
Yang, S.-C., Kalnay, E., and Hunt, B. R.: Handling nonlinearity in Ensemble Kalman Filter: Experiments with the three-variable Lorenz model, Mon. Weather Rev., 140, 2628–2646, https://doi.org/10.1175/MWR-D-11-00313.1, 2012a.
Yang, S.-C., Kalnay, E., and Miyoshi, T.: Improving EnKF spin-up for typhoon assimilation and prediction, Weather Forecast., 27, 878–897, 2012b.
Short summary
Particle filters in their basic form have been shown to be unusable for large geophysical systems because the number of required particles grows exponentially with the size of the system. We have applied the ideas of localized analyses at each model grid point and use ensemble weight smoothing to blend each local analysis with its neighbors. This new local particle filter (LPF) makes large geophysical applications tractable for particle filters and is competitive with a popular EnKF alternative.
Particle filters in their basic form have been shown to be unusable for large geophysical...
