Articles | Volume 30, issue 1
https://doi.org/10.5194/npg-30-13-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/npg-30-13-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Research article
| 
09 Jan 2023
Research article |
| 09 Jan 2023
| 09 Jan 2023
Guidance on how to improve vertical covariance localization based on a 1000-member ensemble
Institut für Meteorologie und Geophysik, Universität Wien, Vienna, Austria
David Hinger
Institut für Meteorologie und Geophysik, Universität Wien, Vienna, Austria
Philipp Johannes Griewank
Institut für Meteorologie und Geophysik, Universität Wien, Vienna, Austria
Takemasa Miyoshi
RIKEN Center for Computational Science, Kobe, Japan
Martin Weissmann
Institut für Meteorologie und Geophysik, Universität Wien, Vienna, Austria
Related authors
No articles found.
Michael Goodliff and Takemasa Miyoshi
EGUsphere, https://doi.org/10.5194/egusphere-2025-933, https://doi.org/10.5194/egusphere-2025-933, 2025
Short summary
Short summary
Data-driven models (DDMs) learn from large datasets to make predictions, but data limitations affect reliability. Data assimilation (DA) improves accuracy by combining real-world observations with computational models. This research explores how DA enhances DDMs despite limited data. We propose an algorithm using DA to refine DDM training iteratively. This work has broad implications for fields like meteorology, engineering, and environmental science, where accurate prediction is critical.
Konstantin Krüger, Andreas Schäfler, Martin Weissmann, and George C. Craig
Weather Clim. Dynam., 5, 491–509, https://doi.org/10.5194/wcd-5-491-2024, https://doi.org/10.5194/wcd-5-491-2024, 2024
Short summary
Short summary
Initial conditions of current numerical weather prediction models insufficiently represent the sharp vertical gradients across the midlatitude tropopause. Observation-space data assimilation output is used to study the influence of assimilated radiosondes on the tropopause. The radiosondes reduce systematic biases of the model background and sharpen temperature and wind gradients in the analysis. Tropopause sharpness is still underestimated in the analysis, which may impact weather forecasts.
Maurus Borne, Peter Knippertz, Martin Weissmann, Benjamin Witschas, Cyrille Flamant, Rosimar Rios-Berrios, and Peter Veals
Atmos. Meas. Tech., 17, 561–581, https://doi.org/10.5194/amt-17-561-2024, https://doi.org/10.5194/amt-17-561-2024, 2024
Short summary
Short summary
This study assesses the quality of Aeolus wind measurements over the tropical Atlantic. The results identified the accuracy and precision of the Aeolus wind measurements and the potential source of errors. For instance, the study revealed atmospheric conditions that can deteriorate the measurement quality, such as weaker laser signal in cloudy or dusty conditions, and confirmed the presence of an orbital-dependant bias. These results can help to improve the Aeolus wind measurement algorithm.
Kenta Kurosawa, Shunji Kotsuki, and Takemasa Miyoshi
Nonlin. Processes Geophys., 30, 457–479, https://doi.org/10.5194/npg-30-457-2023, https://doi.org/10.5194/npg-30-457-2023, 2023
Short summary
Short summary
This study aimed to enhance weather and hydrological forecasts by integrating soil moisture data into a global weather model. By assimilating atmospheric observations and soil moisture data, the accuracy of forecasts was improved, and certain biases were reduced. The method was found to be particularly beneficial in areas like the Sahel and equatorial Africa, where precipitation patterns vary seasonally. This new approach has the potential to improve the precision of weather predictions.
Qiwen Sun, Takemasa Miyoshi, and Serge Richard
Nonlin. Processes Geophys., 30, 117–128, https://doi.org/10.5194/npg-30-117-2023, https://doi.org/10.5194/npg-30-117-2023, 2023
Short summary
Short summary
This paper is a follow-up of a work by Miyoshi and Sun which was published in NPG Letters in 2022. The control simulation experiment is applied to the Lorenz-96 model for avoiding extreme events. The results show that extreme events of this partially and imperfectly observed chaotic system can be avoided by applying pre-designed small perturbations. These investigations may be extended to more realistic numerical weather prediction models.
Anne Martin, Martin Weissmann, and Alexander Cress
Weather Clim. Dynam., 4, 249–264, https://doi.org/10.5194/wcd-4-249-2023, https://doi.org/10.5194/wcd-4-249-2023, 2023
Short summary
Short summary
Global wind profiles from the Aeolus satellite mission are an important recent substitute for the Global Observing System, showing an overall positive impact on numerical weather prediction forecasts. This study highlights atmospheric dynamic phenomena constituting pathways for significant improvement of Aeolus for future studies, including large-scale tropical circulation systems and the interaction of tropical cyclones undergoing an extratropical transition with the midlatitude waveguide.
Shun Ohishi, Takemasa Miyoshi, and Misako Kachi
Geosci. Model Dev., 15, 9057–9073, https://doi.org/10.5194/gmd-15-9057-2022, https://doi.org/10.5194/gmd-15-9057-2022, 2022
Short summary
Short summary
An adaptive observation error inflation (AOEI) method was proposed for atmospheric data assimilation to mitigate erroneous analysis updates caused by large observation-minus-forecast differences for satellite brightness temperature around clear- and cloudy-sky boundaries. This study implemented the AOEI with an ocean data assimilation system, leading to an improvement of analysis accuracy and dynamical balance around the frontal regions with large meridional temperature differences.
Konstantin Krüger, Andreas Schäfler, Martin Wirth, Martin Weissmann, and George C. Craig
Atmos. Chem. Phys., 22, 15559–15577, https://doi.org/10.5194/acp-22-15559-2022, https://doi.org/10.5194/acp-22-15559-2022, 2022
Short summary
Short summary
A comprehensive data set of airborne lidar water vapour profiles is compared with ERA5 reanalyses for a robust characterization of the vertical structure of the mid-latitude lower-stratospheric moist bias. We confirm a moist bias of up to 55 % at 1.3 km altitude above the tropopause and uncover a decreasing bias beyond. Collocated O3 and H2O observations reveal a particularly strong bias in the mixing layer, indicating insufficiently modelled transport processes fostering the bias.
Shun Ohishi, Tsutomu Hihara, Hidenori Aiki, Joji Ishizaka, Yasumasa Miyazawa, Misako Kachi, and Takemasa Miyoshi
Geosci. Model Dev., 15, 8395–8410, https://doi.org/10.5194/gmd-15-8395-2022, https://doi.org/10.5194/gmd-15-8395-2022, 2022
Short summary
Short summary
We develop an ensemble-Kalman-filter-based regional ocean data assimilation system in which satellite and in situ observations are assimilated at a daily frequency. We find the best setting for dynamical balance and accuracy based on sensitivity experiments focused on how to inflate the ensemble spread and how to apply the analysis update to the model evolution. This study has a broader impact on more general data assimilation systems in which the initial shocks are a significant issue.
Shunji Kotsuki, Takemasa Miyoshi, Keiichi Kondo, and Roland Potthast
Geosci. Model Dev., 15, 8325–8348, https://doi.org/10.5194/gmd-15-8325-2022, https://doi.org/10.5194/gmd-15-8325-2022, 2022
Short summary
Short summary
Data assimilation plays an important part in numerical weather prediction (NWP) in terms of combining forecasted states and observations. While data assimilation methods in NWP usually assume the Gaussian error distribution, some variables in the atmosphere, such as precipitation, are known to have non-Gaussian error statistics. This study extended a widely used ensemble data assimilation algorithm to enable the assimilation of more non-Gaussian observations.
Takemasa Miyoshi and Qiwen Sun
Nonlin. Processes Geophys., 29, 133–139, https://doi.org/10.5194/npg-29-133-2022, https://doi.org/10.5194/npg-29-133-2022, 2022
Short summary
Short summary
The weather is chaotic and hard to predict, but the chaos implies an effective control where a small control signal grows rapidly to make a big difference. This study proposes a control simulation experiment where we apply a small signal to control
naturein a computational simulation. Idealized experiments with a low-order chaotic system show successful results by small control signals of only 3 % of the observation error. This is the first step toward realistic weather simulations.
Juan Ruiz, Guo-Yuan Lien, Keiichi Kondo, Shigenori Otsuka, and Takemasa Miyoshi
Nonlin. Processes Geophys., 28, 615–626, https://doi.org/10.5194/npg-28-615-2021, https://doi.org/10.5194/npg-28-615-2021, 2021
Short summary
Short summary
Effective use of observations with numerical weather prediction models, also known as data assimilation, is a key part of weather forecasting systems. For precise prediction at the scales of thunderstorms, fast nonlinear processes pose a grand challenge because most data assimilation systems are based on linear processes and normal distribution errors. We investigate how, every 30 s, weather radar observations can help reduce the effect of nonlinear processes and nonnormal distributions.
Stefan Geiss, Leonhard Scheck, Alberto de Lozar, and Martin Weissmann
Atmos. Chem. Phys., 21, 12273–12290, https://doi.org/10.5194/acp-21-12273-2021, https://doi.org/10.5194/acp-21-12273-2021, 2021
Short summary
Short summary
This study demonstrates the benefits of using both visible and infrared satellite channels to evaluate clouds in numerical weather prediction models. Combining these highly resolved observations provides significantly more and complementary information than using only infrared observations. The visible observations are particularly sensitive to subgrid water clouds, which are not well constrained by other observations.
Anne Martin, Martin Weissmann, Oliver Reitebuch, Michael Rennie, Alexander Geiß, and Alexander Cress
Atmos. Meas. Tech., 14, 2167–2183, https://doi.org/10.5194/amt-14-2167-2021, https://doi.org/10.5194/amt-14-2167-2021, 2021
Short summary
Short summary
This study provides an overview of validation activities to determine the Aeolus HLOS wind errors and to understand the biases by investigating possible dependencies and testing bias correction approaches. To ensure meaningful validation statistics, collocated radiosondes and two different global NWP models, the ECMWF IFS and the ICON model (DWD), are used as reference data. To achieve an estimate for the Aeolus instrumental error the representativeness errors for the comparisons are evaluated.
Philipp J. Griewank, Thijs Heus, Neil P. Lareau, and Roel A. J. Neggers
Atmos. Chem. Phys., 20, 10211–10230, https://doi.org/10.5194/acp-20-10211-2020, https://doi.org/10.5194/acp-20-10211-2020, 2020
Short summary
Short summary
The idea that larger shallow cumulus clouds have stronger updrafts than small shallow cumulus clouds is as intuitive as it is old. In this paper we gather years of upward-pointing laser measurements from a plain in Oklahoma and combine them with 28 d of high-resolution simulations. Our approach, which has much more data than previous studies, confirms that updraft strength and cloud size are linked and that the simulations reproduce the observed cloud wind and moisture structure.
Keiichi Kondo and Takemasa Miyoshi
Nonlin. Processes Geophys., 26, 211–225, https://doi.org/10.5194/npg-26-211-2019, https://doi.org/10.5194/npg-26-211-2019, 2019
Short summary
Short summary
This study investigates non-Gaussian statistics of the data from a 10240-member ensemble Kalman filter. The large ensemble size can resolve the detailed structures of the probability density functions (PDFs) and indicates that the non-Gaussian PDF is caused by multimodality and outliers. While the outliers appear randomly, large multimodality corresponds well with large analysis error, mainly in the tropical regions and storm track regions where highly nonlinear processes appear frequently.
Atsushi Okazaki, Takumi Honda, Shunji Kotsuki, Moeka Yamaji, Takuji Kubota, Riko Oki, Toshio Iguchi, and Takemasa Miyoshi
Atmos. Meas. Tech., 12, 3985–3996, https://doi.org/10.5194/amt-12-3985-2019, https://doi.org/10.5194/amt-12-3985-2019, 2019
Short summary
Short summary
The JAXA is surveying the feasibility of a potential satellite mission equipped with a precipitation radar on a geostationary orbit, as a successor of the GPM Core Observatory. We investigate what kind of observation data will be available from the radar using simulation techniques. Although the quality of the observation depends on the radar specifications and the position of precipitation systems, the results demonstrate that it would be possible to obtain three-dimensional precipitation data.
Guo-Yuan Lien, Daisuke Hotta, Eugenia Kalnay, Takemasa Miyoshi, and Tse-Chun Chen
Nonlin. Processes Geophys., 25, 129–143, https://doi.org/10.5194/npg-25-129-2018, https://doi.org/10.5194/npg-25-129-2018, 2018
Short summary
Short summary
The ensemble forecast sensitivity to observation (EFSO) method can efficiently clarify under what conditions observations are beneficial or detrimental for assimilation. Based on EFSO, an offline assimilation method is proposed to accelerate the development of data selection strategies for new observing systems. The usefulness of this method is demonstrated with the assimilation of global satellite precipitation data.
Hazuki Arakida, Takemasa Miyoshi, Takeshi Ise, Shin-ichiro Shima, and Shunji Kotsuki
Nonlin. Processes Geophys., 24, 553–567, https://doi.org/10.5194/npg-24-553-2017, https://doi.org/10.5194/npg-24-553-2017, 2017
Short summary
Short summary
This is the first study assimilating the satellite-based leaf area index observations every 4 days into a numerical model simulating the growth and death of individual plants. The newly developed data assimilation system successfully reduced the uncertainties of the model parameters related to phenology and carbon dynamics. It also provides better estimates of the present vegetation structure which can be used as the initial states for the simulation of the future vegetation change.
Stephen G. Penny and Takemasa Miyoshi
Nonlin. Processes Geophys., 23, 391–405, https://doi.org/10.5194/npg-23-391-2016, https://doi.org/10.5194/npg-23-391-2016, 2016
Short summary
Short summary
Particle filters in their basic form have been shown to be unusable for large geophysical systems because the number of required particles grows exponentially with the size of the system. We have applied the ideas of localized analyses at each model grid point and use ensemble weight smoothing to blend each local analysis with its neighbors. This new local particle filter (LPF) makes large geophysical applications tractable for particle filters and is competitive with a popular EnKF alternative.
Hisashi Yashiro, Koji Terasaki, Takemasa Miyoshi, and Hirofumi Tomita
Geosci. Model Dev., 9, 2293–2300, https://doi.org/10.5194/gmd-9-2293-2016, https://doi.org/10.5194/gmd-9-2293-2016, 2016
Short summary
Short summary
We propose the design and implementation of an ensemble data assimilation framework for weather prediction at a high resolution and with a large ensemble size. We consider the deployment of this framework on the data throughput of file I/O and multi-node communication. With regard to high-performance computing systems, where data throughput performance increases at a slower rate than computational performance, our new framework promises drastic reduction of total execution time.
X. Han, X. Li, G. He, P. Kumbhar, C. Montzka, S. Kollet, T. Miyoshi, R. Rosolem, Y. Zhang, H. Vereecken, and H.-J. H. Franssen
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmdd-8-7395-2015, https://doi.org/10.5194/gmdd-8-7395-2015, 2015
Revised manuscript not accepted
Short summary
Short summary
DasPy is a ready to use open source parallel multivariate land data assimilation framework with joint state and parameter estimation using Local Ensemble Transform Kalman Filter. The Community Land Model (4.5) was integrated as model operator. The Community Microwave Emission Modelling platform, COsmic-ray Soil Moisture Interaction Code and the Two-Source Formulation were integrated as observation operators for the multivariate assimilation of soil moisture and soil temperature, respectively.
S. G. Penny, E. Kalnay, J. A. Carton, B. R. Hunt, K. Ide, T. Miyoshi, and G. A. Chepurin
Nonlin. Processes Geophys., 20, 1031–1046, https://doi.org/10.5194/npg-20-1031-2013, https://doi.org/10.5194/npg-20-1031-2013, 2013
Related subject area
Subject: Predictability, probabilistic forecasts, data assimilation, inverse problems | Topic: Climate, atmosphere, ocean, hydrology, cryosphere, biosphere | Techniques: Big data and artificial intelligence
Selecting and weighting dynamical models using data-driven approaches
A quest for precipitation attractors in weather radar archives
Robust weather-adaptive post-processing using model output statistics random forests
Weather pattern dynamics over western Europe under climate change: predictability, information entropy and production
Calibrated ensemble forecasts of the height of new snow using quantile regression forests and ensemble model output statistics
Enhancing geophysical flow machine learning performance via scale separation
Training a convolutional neural network to conserve mass in data assimilation
Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network
From research to applications – examples of operational ensemble post-processing in France using machine learning
Pierre Le Bras, Florian Sévellec, Pierre Tandeo, Juan Ruiz, and Pierre Ailliot
Nonlin. Processes Geophys., 31, 303–317, https://doi.org/10.5194/npg-31-303-2024, https://doi.org/10.5194/npg-31-303-2024, 2024
Short summary
Short summary
The goal of this paper is to weight several dynamic models in order to improve the representativeness of a system. It is illustrated using a set of versions of an idealized model describing the Atlantic Meridional Overturning Circulation. The low-cost method is based on data-driven forecasts. It enables model performance to be evaluated on their dynamics. Taking into account both model performance and codependency, the derived weights outperform benchmarks in reconstructing a model distribution.
Loris Foresti, Bernat Puigdomènech Treserras, Daniele Nerini, Aitor Atencia, Marco Gabella, Ioannis V. Sideris, Urs Germann, and Isztar Zawadzki
Nonlin. Processes Geophys., 31, 259–286, https://doi.org/10.5194/npg-31-259-2024, https://doi.org/10.5194/npg-31-259-2024, 2024
Short summary
Short summary
We compared two ways of defining the phase space of low-dimensional attractors describing the evolution of radar precipitation fields. The first defines the phase space by the domain-scale statistics of precipitation fields, such as their mean, spatial and temporal correlations. The second uses principal component analysis to account for the spatial distribution of precipitation. To represent different climates, radar archives over the United States and the Swiss Alpine region were used.
Thomas Muschinski, Georg J. Mayr, Achim Zeileis, and Thorsten Simon
Nonlin. Processes Geophys., 30, 503–514, https://doi.org/10.5194/npg-30-503-2023, https://doi.org/10.5194/npg-30-503-2023, 2023
Short summary
Short summary
Statistical post-processing is necessary to generate probabilistic forecasts from physical numerical weather prediction models. To allow for more flexibility, there has been a shift in post-processing away from traditional parametric regression models towards modern machine learning methods. By fusing these two approaches, we developed model output statistics random forests, a new post-processing method that is highly flexible but at the same time also very robust and easy to interpret.
Stéphane Vannitsem
Nonlin. Processes Geophys., 30, 1–12, https://doi.org/10.5194/npg-30-1-2023, https://doi.org/10.5194/npg-30-1-2023, 2023
Short summary
Short summary
The impact of climate change on weather pattern dynamics over the North Atlantic is explored through the lens of information theory. These tools allow the predictability of the succession of weather patterns and the irreversible nature of the dynamics to be clarified. It is shown that the predictability is increasing in the observations, while the opposite trend is found in model projections. The irreversibility displays an overall increase in time in both the observations and the model runs.
Guillaume Evin, Matthieu Lafaysse, Maxime Taillardat, and Michaël Zamo
Nonlin. Processes Geophys., 28, 467–480, https://doi.org/10.5194/npg-28-467-2021, https://doi.org/10.5194/npg-28-467-2021, 2021
Short summary
Short summary
Forecasting the height of new snow is essential for avalanche hazard surveys, road and ski resort management, tourism attractiveness, etc. Météo-France operates a probabilistic forecasting system using a numerical weather prediction system and a snowpack model. It provides better forecasts than direct diagnostics but exhibits significant biases. Post-processing methods can be applied to provide automatic forecasting products from this system.
Davide Faranda, Mathieu Vrac, Pascal Yiou, Flavio Maria Emanuele Pons, Adnane Hamid, Giulia Carella, Cedric Ngoungue Langue, Soulivanh Thao, and Valerie Gautard
Nonlin. Processes Geophys., 28, 423–443, https://doi.org/10.5194/npg-28-423-2021, https://doi.org/10.5194/npg-28-423-2021, 2021
Short summary
Short summary
Machine learning approaches are spreading rapidly in climate sciences. They are of great help in many practical situations where using the underlying equations is difficult because of the limitation in computational power. Here we use a systematic approach to investigate the limitations of the popular echo state network algorithms used to forecast the long-term behaviour of chaotic systems, such as the weather. Our results show that noise and intermittency greatly affect the performances.
Yvonne Ruckstuhl, Tijana Janjić, and Stephan Rasp
Nonlin. Processes Geophys., 28, 111–119, https://doi.org/10.5194/npg-28-111-2021, https://doi.org/10.5194/npg-28-111-2021, 2021
Short summary
Short summary
The assimilation of observations using standard algorithms can lead to a violation of physical laws (e.g. mass conservation), which is shown to have a detrimental impact on the system's forecast. We use a neural network (NN) to correct this mass violation, using training data generated from expensive algorithms that can constrain such physical properties. We found that, in an idealized set-up, the NN can match the performance of these expensive algorithms at negligible computational costs.
Ashesh Chattopadhyay, Pedram Hassanzadeh, and Devika Subramanian
Nonlin. Processes Geophys., 27, 373–389, https://doi.org/10.5194/npg-27-373-2020, https://doi.org/10.5194/npg-27-373-2020, 2020
Short summary
Short summary
The performance of three machine-learning methods for data-driven modeling of a multiscale chaotic Lorenz 96 system is examined. One of the methods is found to be able to predict the future evolution of the chaotic system well from just knowing the past observations of the large-scale component of the multiscale state vector. Potential applications to data-driven and data-assisted surrogate modeling of complex dynamical systems such as weather and climate are discussed.
Maxime Taillardat and Olivier Mestre
Nonlin. Processes Geophys., 27, 329–347, https://doi.org/10.5194/npg-27-329-2020, https://doi.org/10.5194/npg-27-329-2020, 2020
Short summary
Short summary
Statistical post-processing of ensemble forecasts is now a well-known procedure in order to correct biased and misdispersed ensemble weather predictions. But practical application in European national weather services is in its infancy. Different applications of ensemble post-processing using machine learning at an industrial scale are presented. Forecast quality and value are improved compared to the raw ensemble, but several facilities have to be made to adjust to operational constraints.
Cited articles
Anderson, J.: Exploring the need for localization in ensemble data
assimilation using a hierarchical ensemble filter, Physica D, 230, 99–111, https://doi.org/10.1016/j.physd.2006.02.011, 2007. a, b, c
Anderson, J. and Lei, L.: Empirical Localization of Observation Impact in
Ensemble Kalman Filters, Mon. Weather Rev., 141, 4140–4153,
https://doi.org/10.1175/MWR-D-12-00330.1, 2013. a, b, c
Anderson, J., Hoar, T., Raeder, K., Liu, H., Collins, N., Torn, R., and
Avellano, A.: The Data Assimilation Research Testbed: A Community Facility,
B. Am. Meteor. Soc., 90, 1283–1296,
https://doi.org/10.1175/2009BAMS2618.1, 2009. a, b
Anderson, J. L.: An Ensemble Adjustment Kalman Filter for Data Assimilation,
Mon. Weather Rev., 129, 2884–2903,
https://doi.org/10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2, 2001. a
Anderson, J. L.: Reducing Correlation Sampling Error in Ensemble Kalman Filter
Data Assimilation, Mon. Weather Rev., 144, 913–925,
https://doi.org/10.1175/MWR-D-15-0052.1, 2016. a, b, c, d
Bannister, R. N.: A review of forecast error covariance statistics in
atmospheric variational data assimilation. II: Modelling the forecast error
covariance statistics,
Q. J. Roy. Meteor. Soc.,
134, 1971–1996, https://doi.org/10.1002/qj.340, 2008. a
Bannister, R. N.: A review of operational methods of variational and
ensemble-variational data assimilation, Q. J. R. Meteorol. Soc., 143,
607–633, https://doi.org/10.1002/qj.2982, 2017. a, b
Bannister, R. N., Migliorini, S., Rudd, A. C., and Baker, L. H.: Methods of investigating forecast error sensitivity to ensemble size in a limited-area convection-permitting ensemble, Geosci. Model Dev. Discuss. [preprint], https://doi.org/10.5194/gmd-2017-260, in review, 2017. a, b
Bishop, C. H. and Hodyss, D.: Ensemble covariances adaptively localized with
ECO-RAP. Part 1: tests on simple error models, Tellus A, 61, 84–96,
https://doi.org/10.1111/j.1600-0870.2008.00371.x, 2009a. a
Bishop, C. H. and Hodyss, D.: Ensemble covariances adaptively localized with
ECO-RAP. Part 2: a strategy for the atmosphere, Tellus A, 61, 97–111,
https://doi.org/10.1111/j.1600-0870.2008.00372.x, 2009b. a
Bolin, D. and Wallin, J.: Spatially adaptive covariance tapering, Spatial
Statistics, 18, 163–178, https://doi.org/10.1016/j.spasta.2016.03.003,
2016. a, b
Bonavita, M., Hólm, E., Isaksen, L., and Fisher, M.: The evolution of the
ECMWF hybrid data assimilation system, Q. J. Roy. Meteor. Soc., 142, 287–303, https://doi.org/10.1002/qj.2652,
2016. a
Bouttier, F., Raynaud, L., Nuissier, O., and Menetrier, B.: Sensitivity of the
AROME ensemble to initial and surface perturbations during HyMeX, Q. J. Roy. Meteor. Soc., 142, 390–403,
https://doi.org/10.1002/qj.2622, 2016. a
Craig, G. C., Puh, M., Keil, C., Tempest, K., Necker, T., Ruiz, J., Weissmann,
M., and Miyoshi, T.: Distributions and convergence of forecast variables in
a 1000 member convection-permitting ensemble, Q. J. Roy.
Meteor. Soc., 148, 2325–2343, https://doi.org/10.1002/qj.4305, 2022. a, b
Daley, D., Porcu, E., and Bevilacqua, M.: Classes of compactly supported
covariance functions for multivariate random fields,
Stoch. Env. Res. Risk A., 29, 1249–1263,
https://doi.org/10.1007/s00477-014-0996-y, 2015. a
Destouches, M., Montmerle, T., Michel, Y., and Ménétrier, B.: Estimating
optimal localization for sampled background-error covariances of hydrometeor
variables, Q. J. Roy. Meteor. Soc., 147,
74–93, https://doi.org/10.1002/qj.3906, 2021. a, b
Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic
model using Monte Carlo methods to forecast error statistics, J.
Geophys. Res.-Oceans, 99, 10143–10162, https://doi.org/10.1029/94JC00572,
1994. a
Flowerdew, J.: Towards a theory of optimal localisation, Tellus A, 67, 25257, https://doi.org/10.3402/tellusa.v67.25257,
2015. a, b, c, d
Gaspari, G., Cohn, S. E., Guo, J., and Pawson, S.: Construction and application
of covariance functions with variable length-fields, Q. J.
Roy. Meteor. Soc., 132, 1815–1838,
https://doi.org/10.1256/qj.05.08, 2006. a
Greybush, S. J., Kalnay, E., Miyoshi, T., Ide, K., and Hunt, B. R.: Balance and
Ensemble Kalman Filter Localization Techniques, Mon. Weather Rev., 139,
511–522, https://doi.org/10.1175/2010MWR3328.1, 2011. a
Gustafsson, N., Janjić, T., Schraff, C., Leuenberger, D., Weissman, M.,
Reich, H., Brousseau, P., Montmerle, T., Wattrelot, E., Bučánek,
A., Mile, M., Hamdi, R., Lindskog, M., Barkmeijer, J., Dahlbom, M.,
Macpherson, B., Ballard, S., Inverarity, G., Carley, J., Alexander, C.,
Dowell, D., Liu, S., Ikuta, Y., and Fujita, T.: Survey of data assimilation
methods for convective-scale numerical weather prediction at operational
centres, Q. J. Roy. Meteor. Soc., 144,
1218–1256, https://doi.org/10.1002/qj.3179, 2018. a, b
Hagelin, S., Son, J., Swinbank, R., McCabe, A., Roberts, N., and Tennant, W.:
The Met Office convective-scale ensemble, MOGREPS-UK, Q. J. Roy. Meteor. Soc., 143, 2846–2861, https://doi.org/10.1002/qj.3135, 2017. a
Hamill, T., Whitaker, J., and Snyder, C.: Distance-Dependent Filtering of
Background Error Covariance Estimates in an Ensemble Kalman Filter, Mon. Weather Rev., 129, 2776–2790,
https://doi.org/10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2, 2001. a, b, c
Higham, N. J.: Computing the nearest correlation matrix–a problem from
finance, IMA J. Numer. Anal., 22, 329–343,
https://doi.org/10.1093/imanum/22.3.329, 2002. a, b, c, d
Hohenegger, C. and Schaer, C.: Predictability and Error Growth Dynamics in
Cloud-Resolving Models, J. Atmos. Sci., 64, 4467–4478,
https://doi.org/10.1175/2007JAS2143.1, 2007. a
Horn, R. A. and Johnson, C. R.: Matrix Analysis, Cambridge University Press, 2
edn., https://doi.org/10.1017/CBO9781139020411, 2012. a
Hotta, D. and Ota, Y.: Why does EnKF suffer from analysis overconfidence? An
insight into exploiting the ever-increasing volume of observations, Q. J. Roy. Meteor. Soc., 147, 1258–1277,
https://doi.org/10.1002/qj.3970, 2021. a, b
Houtekamer, P. L. and Mitchell, H. L.: A Sequential Ensemble Kalman Filter for
Atmospheric Data Assimilation, Mon. Weather Rev., 129, 123–137,
https://doi.org/10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2, 2001. a, b
Hoyer, S. and Hamman, J. J.: xarray: N-D labeled Arrays and Datasets in
Python, Journal of Open Research Software, 5, 1–6, https://doi.org/10.5334/jors.148,
2017. a
Hu, G., Dance, S. L., Bannister, R. N., Chipilski, H., Guillet, O., Macpherson,
B., Weissmann, M., and Yussouf, N.: Progress, challenges, and future steps in data assimilation for convection-permitting numerical weather prediction: Report on the virtual meeting held on 10 and 12 November 2021, Atmos. Sci. Lett., 24, e1130, https://doi.org/10.1002/asl.1130, 2023. a
Hunt, B. R., Kostelich, E. J., and Szunyogh, I.: Efficient data assimilation
for spatiotemporal chaos: A local ensemble transform Kalman filter, Physica
D, 230, 112–126, https://doi.org/10.1016/j.physd.2006.11.008,
2007. a
Kepert, J. D.: Covariance localisation and balance in an Ensemble Kalman
Filter, Q. J. Roy. Meteor. Soc., 135,
1157–1176, https://doi.org/10.1002/qj.443, 2009. a
Kirchgessner, P., Nerger, L., and Bunse-Gerstner, A.: On the Choice of an
Optimal Localization Radius in Ensemble Kalman Filter Methods, Mon. Weather Rev., 142, 2165–2175, https://doi.org/10.1175/MWR-D-13-00246.1, 2014. a
Kondo, K. and Miyoshi, T.: Impact of Removing Covariance Localization in an
Ensemble Kalman Filter: Experiments with 10 240 Members Using an Intermediate
AGCM, Mon. Weather Rev., 144, 4849–4865, https://doi.org/10.1175/MWR-D-15-0388.1, 2016. a, b
Kunii, M.: The 1000-Member Ensemble Kalman Filtering with the JMA
Nonhydrostatic Mesoscale Model on the K Computer,
J. Meteorol. Soc. Jpn., 92, 623–633,
https://doi.org/10.2151/jmsj.2014-607, 2014. a
Lei, L. and Anderson, J. L.: Comparisons of Empirical Localization Techniques
for Serial Ensemble Kalman Filters in a Simple Atmospheric General
Circulation Model, Mon. Weather Rev., 142, 739–754,
https://doi.org/10.1175/MWR-D-13-00152.1, 2014. a, b, c
Lei, L., Anderson, J. L., and Romine, G. S.: Empirical Localization Functions
for Ensemble Kalman Filter Data Assimilation in Regions with and without
Precipitation, Mon. Weather Rev., 143, 3664–3679,
https://doi.org/10.1175/MWR-D-14-00415.1, 2015. a, b
Lei, L., Anderson, J. L., and Whitaker, J. S.: Localizing the impact of
satellite radiance observations using a global group ensemble filter, J. Adv. Model. Earth Sy., 8, 719–734,
https://doi.org/10.1002/2016MS000627, 2016. a, b
Lei, L., Whitaker, J. S., and Bishop, C.: Improving Assimilation of Radiance
Observations by Implementing Model Space Localization in an Ensemble Kalman
Filter, J. Adv. Model. Earth Sy., 10, 3221–3232,
https://doi.org/10.1029/2018MS001468, 2018. a
Lei, L., Whitaker, J. S., Anderson, J. L., and Tan, Z.: Adaptive Localization
for Satellite Radiance Observations in an Ensemble Kalman Filter, J.
Adv. Model. Earth Sy., 12, e2019MS001693,
https://doi.org/10.1029/2019MS001693, 2020. a
Lien, G.-Y., Miyoshi, T., Nishizawa, S., Yoshida, R., Yashiro, H., Adachi,
S. A., Yamaura, T., and Tomita, H.: The Near-Real-Time SCALE-LETKF System: A
Case of the September 2015 Kanto-Tohoku Heavy Rainfall, SOLA, 13, 1–6,
https://doi.org/10.2151/sola.2017-001, 2017. a
Michel, Y., Auligné, T., and Montmerle, T.: Heterogeneous Convective-Scale
Background Error Covariances with the Inclusion of Hydrometeor Variables,
Mon. Weather Rev., 139, 2994–3015, https://doi.org/10.1175/2011MWR3632.1, 2011. a
Miyoshi, T. and Yamane, S.: Local Ensemble Transform Kalman Filtering with an
AGCM at a T159/L48 Resolution, Mon. Weather Rev., 135, 3841–3861,
https://doi.org/10.1175/2007MWR1873.1, 2007. a
Miyoshi, T., Kondo, K., and Imamura, T.: The 10,240-member ensemble Kalman
filtering with an intermediate AGCM, Geophys. Res. Lett., 41,
5264–5271, https://doi.org/10.1002/2014GL060863, 2014. a, b
Miyoshi, T., Kondo, K., and Terasaki, K.: Big Ensemble Data Assimilation in
Numerical Weather Prediction, Computer, 48, 15–21,
https://doi.org/10.1109/MC.2015.332, 2015. a
Miyoshi, T., Kunii, M., Ruiz, J., Lien, G.-Y., Satoh, S., Ushio, T., Bessho,
K., Seko, H., Tomita, H., and Ishikawa, Y.: Big Data Assimilation
Revolutionizing Severe Weather Prediction, B. Am. Meteor. Soc., 97, 1347–1354, https://doi.org/10.1175/BAMS-D-15-00144.1,
2016a. a
Miyoshi, T., Lien, G., Satoh, S., Ushio, T., Bessho, K., Tomita, H., Nishizawa,
S., Yoshida, R., Adachi, S. A., Liao, J., Gerofi, B., Ishikawa, Y., Kunii,
M., Ruiz, J., Maejima, Y., Otsuka, S., Otsuka, M., Okamoto, K., and Seko, H.:
“Big Data Assimilation” Toward Post-Petascale Severe Weather Prediction:
An Overview and Progress, P. IEEE, 104, 2155–2179,
https://doi.org/10.1109/JPROC.2016.2602560, 2016b. a
Ménétrier, B., Montmerle, T., Berre, L., and Michel, Y.: Estimation and
diagnosis of heterogeneous flow-dependent background-error covariances at the
convective scale using either large or small ensembles, Q. J.
Roy. Meteor. Soc., 140, 2050–2061, https://doi.org/10.1002/qj.2267,
2014. a, b
Ménétrier, B., Montmerle, T., Michel, Y., and Berre, L.: Linear Filtering of
Sample Covariances for Ensemble-Based Data Assimilation. Part I: Optimality
Criteria and Application to Variance Filtering and Covariance Localization,
Mon. Weather Rev., 143, 1622–1643, https://doi.org/10.1175/MWR-D-14-00157.1,
2015a. a, b
Ménétrier, B., Montmerle, T., Michel, Y., and Berre, L.: Linear Filtering of
Sample Covariances for Ensemble-Based Data Assimilation. Part II: Application
to a Convective-Scale NWP Model, Mon. Weather Rev., 143, 1644–1664,
https://doi.org/10.1175/MWR-D-14-00156.1, 2015b. a
Necker, T.: Empirical optimal localization/Vertical covariance
localization, Zenodo [data set, code], https://doi.org/10.5281/zenodo.7254119, 2022. a
Nomokonova, T., Griewank, P. J., Löhnert, U., Miyoshi, T., Necker, T., and
Weissmann, M.: Estimating the benefit of Doppler wind lidars for short-term low-level wind ensemble forecasts, Q. J. Roy. Meteor. Soc., 1–19, https://doi.org/10.1002/qj.4402, 2022. a
Perianez, A., Reich, H., and Potthast, R.: Optimal Localization for Ensemble
Kalman Filter Systems, J. Meteor. Soc. Jpn., 92, 585–597, https://doi.org/10.2151/jmsj.2014-605, 2014. a
Piper, D., Kunz, M., Ehmele, F., Mohr, S., Mühr, B., Kron, A., and Daniell, J.: Exceptional sequence of severe thunderstorms and related flash floods in May and June 2016 in Germany – Part 1: Meteorological background, Nat. Hazards Earth Syst. Sci., 16, 2835–2850, https://doi.org/10.5194/nhess-16-2835-2016, 2016.
a
Poterjoy, J., Zhang, F., and Weng, Y.: The Effects of Sampling Errors on the
EnKF Assimilation of Inner-Core Hurricane Observations, Mon. Weather Rev., 142, 1609–1630, https://doi.org/10.1175/MWR-D-13-00305.1, 2014. a, b
Scheck, L., Weissmann, M., and Bach, L.: Assimilating visible satellite images
for convective-scale numerical weather prediction: A case-study, Q. J. Roy. Meteor. Soc., 146, 3165–3186,
https://doi.org/10.1002/qj.3840, 2020. a
Schraff, C., Reich, H., Rhodin, A., Schomburg, A., Stephan, K., Periáñez, A.,
and Potthast, R.: Kilometre-scale ensemble data assimilation for the COSMO
model (KENDA), Q. J. Roy. Meteor. Soc., 142,
1453–1472, https://doi.org/10.1002/qj.2748, 2016. a
Stanley, Z., Grooms, I., and Kleiber, W.: Multivariate localization functions for strongly coupled data assimilation in the bivariate Lorenz 96 system, Nonlin. Processes Geophys., 28, 565–583, https://doi.org/10.5194/npg-28-565-2021, 2021. a, b
Tabeart, J. M., Dance, S. L., Lawless, A. S., Nichols, N. K., and Waller,
J. A.: Improving the condition number of estimated covariance matrices,
Tellus, 72, 1–19, https://doi.org/10.1080/16000870.2019.1696646, 2019. a
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T.,
Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van
der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson,
A. R. J., Jones, E., Kern, R., Larson, E., Carey, C. J., Polat, I., Feng, Y.,
Moore, E. W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R.,
Henriksen, I., Quintero, E. A., Harris, C. R., Archibald, A. M., Ribeiro,
A. H., Pedregosa, F., van Mulbregt, P., and SciPy 1.0 Contributors:
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,
Nat. Methods, 17, 261–272, https://doi.org/10.1038/s41592-019-0686-2, 2020. a
Whitaker, J. S. and Hamill, T. M.: Evaluating Methods to Account for System
Errors in Ensemble Data Assimilation, Mon. Weather Rev., 140, 3078–3089, https://doi.org/10.1175/MWR-D-11-00276.1, 2012. a
Wu, P.-Y., Yang, S.-C., Tsai, C.-C., and Cheng, H.-W.: Convective-Scale
Sampling Error and Its Impact on the Ensemble Radar Data Assimilation System:
A Case Study of a Heavy Rainfall Event on 16 June 2008 in Taiwan, Mon. Weather Rev., 148, 3631–3652, https://doi.org/10.1175/MWR-D-19-0319.1, 2020. a, b
Short summary
This study investigates vertical localization based on a convection-permitting 1000-member ensemble simulation. We derive an empirical optimal localization (EOL) that minimizes sampling error in 40-member sub-sample correlations assuming 1000-member correlations as truth. The results will provide guidance for localization in convective-scale ensemble data assimilation systems.
This study investigates vertical localization based on a convection-permitting 1000-member...
