Articles | Volume 30, issue 4
https://doi.org/10.5194/npg-30-503-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/npg-30-503-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Research article
| 
20 Nov 2023
Research article |
| 20 Nov 2023
| 20 Nov 2023
Robust weather-adaptive post-processing using model output statistics random forests
Thomas Muschinski
CORRESPONDING AUTHOR
Department of Atmospheric and Cryospheric Sciences, Universität Innsbruck, Innsbruck, Austria
Department of Economics, Statistical Methods and Econometrics, Karlsruhe Institute of Technology, Karlsruhe, Germany
Georg J. Mayr
Department of Atmospheric and Cryospheric Sciences, Universität Innsbruck, Innsbruck, Austria
Achim Zeileis
Department of Statistics, Universität Innsbruck, Innsbruck, Austria
Thorsten Simon
Department of Statistics, Universität Innsbruck, Innsbruck, Austria
Related authors
Thomas Muschinski, Moritz N. Lang, Georg J. Mayr, Jakob W. Messner, Achim Zeileis, and Thorsten Simon
Wind Energ. Sci., 7, 2393–2405, https://doi.org/10.5194/wes-7-2393-2022, https://doi.org/10.5194/wes-7-2393-2022, 2022
Short summary
Short summary
The power generated by offshore wind farms can vary greatly within a couple of hours, and failing to anticipate these ramp events can lead to costly imbalances in the electrical grid. A novel multivariate Gaussian regression model helps us to forecast not just the means and variances of the next day's hourly wind speeds, but also their corresponding correlations. This information is used to generate more realistic scenarios of power production and accurate estimates for ramp probabilities.
Helen Claire Ward, Mathias Walter Rotach, Alexander Gohm, Martin Graus, Thomas Karl, Maren Haid, Lukas Umek, and Thomas Muschinski
Atmos. Chem. Phys., 22, 6559–6593, https://doi.org/10.5194/acp-22-6559-2022, https://doi.org/10.5194/acp-22-6559-2022, 2022
Short summary
Short summary
This study examines how cities and their surroundings influence turbulent exchange processes responsible for weather and climate. Analysis of a 4-year observational dataset for the Alpine city of Innsbruck reveals several similarities with other (flat) city centre sites. However, the mountain setting leads to characteristic daily and seasonal flow patterns (valley winds) and downslope windstorms that have a marked effect on temperature, wind speed, turbulence and pollutant concentration.
Fiona Fix-Hewitt, Achim Zeileis, Isabell Stucke, Reto Stauffer, and Georg J. Mayr
EGUsphere, https://doi.org/10.5194/egusphere-2025-3552, https://doi.org/10.5194/egusphere-2025-3552, 2025
This preprint is open for discussion and under review for Weather and Climate Dynamics (WCD).
Short summary
Short summary
Atmospheric deserts are air masses from the boundary layer of desert regions. This study tracked them from North Africa to Europe during a two-year period (May 2022–April 2024). They can occur up to 60 % of the time, and can span large parts of Europe. They typically last about a day, and can alter the atmosphere throughout the free troposphere. On its way from Africa the air either rises and may become even warmer and drier, or it stays at mid-altitudes and cools, or retains its properties.
Gregor Ehrensperger, Thorsten Simon, Georg J. Mayr, and Tobias Hell
Geosci. Model Dev., 18, 1141–1153, https://doi.org/10.5194/gmd-18-1141-2025, https://doi.org/10.5194/gmd-18-1141-2025, 2025
Short summary
Short summary
As lightning is a brief and localized event, it is not explicitly resolved in atmospheric models. Instead, expert-based auxiliary descriptions are used to assess it. This study explores how AI can improve our understanding of lightning without relying on traditional expert knowledge. We reveal that AI independently identified the key factors known to experts as essential for lightning in the Alps region. This shows how knowledge discovery could be sped up in areas with limited expert knowledge.
Fiona Fix, Georg Mayr, Achim Zeileis, Isabell Stucke, and Reto Stauffer
Weather Clim. Dynam., 5, 1545–1560, https://doi.org/10.5194/wcd-5-1545-2024, https://doi.org/10.5194/wcd-5-1545-2024, 2024
Short summary
Short summary
Atmospheric deserts (ADs) are air masses that are transported away from hot, dry regions. Our study introduces this new concept. ADs can suppress or boost thunderstorms and potentially contribute to the formation of heat waves, which makes them relevant for forecasting extreme events. Using a novel detection method, we follow an AD directly from North Africa to Europe for a case in June 2022, allowing us to analyse the air mass at any time and investigate how it is modified along the way.
Deborah Morgenstern, Isabell Stucke, Georg J. Mayr, Achim Zeileis, and Thorsten Simon
Weather Clim. Dynam., 4, 489–509, https://doi.org/10.5194/wcd-4-489-2023, https://doi.org/10.5194/wcd-4-489-2023, 2023
Short summary
Short summary
Two thunderstorm environments are described for Europe: mass-field thunderstorms, which occur mostly in summer, over land, and under similar meteorological conditions, and wind-field thunderstorms, which occur mostly in winter, over the sea, and under more diverse meteorological conditions. Our descriptions are independent of static thresholds and help to understand why thunderstorms in unfavorable seasons for lightning pose a particular risk to tall infrastructure such as wind turbines.
Thomas Muschinski, Moritz N. Lang, Georg J. Mayr, Jakob W. Messner, Achim Zeileis, and Thorsten Simon
Wind Energ. Sci., 7, 2393–2405, https://doi.org/10.5194/wes-7-2393-2022, https://doi.org/10.5194/wes-7-2393-2022, 2022
Short summary
Short summary
The power generated by offshore wind farms can vary greatly within a couple of hours, and failing to anticipate these ramp events can lead to costly imbalances in the electrical grid. A novel multivariate Gaussian regression model helps us to forecast not just the means and variances of the next day's hourly wind speeds, but also their corresponding correlations. This information is used to generate more realistic scenarios of power production and accurate estimates for ramp probabilities.
Helen Claire Ward, Mathias Walter Rotach, Alexander Gohm, Martin Graus, Thomas Karl, Maren Haid, Lukas Umek, and Thomas Muschinski
Atmos. Chem. Phys., 22, 6559–6593, https://doi.org/10.5194/acp-22-6559-2022, https://doi.org/10.5194/acp-22-6559-2022, 2022
Short summary
Short summary
This study examines how cities and their surroundings influence turbulent exchange processes responsible for weather and climate. Analysis of a 4-year observational dataset for the Alpine city of Innsbruck reveals several similarities with other (flat) city centre sites. However, the mountain setting leads to characteristic daily and seasonal flow patterns (valley winds) and downslope windstorms that have a marked effect on temperature, wind speed, turbulence and pollutant concentration.
Deborah Morgenstern, Isabell Stucke, Thorsten Simon, Georg J. Mayr, and Achim Zeileis
Weather Clim. Dynam., 3, 361–375, https://doi.org/10.5194/wcd-3-361-2022, https://doi.org/10.5194/wcd-3-361-2022, 2022
Short summary
Short summary
Wintertime lightning in central Europe is rare but has a large damage potential for tall structures such as wind turbines. We use a data-driven approach to explain why it even occurs when the meteorological processes causing thunderstorms in summer are absent. In summer, with strong solar input, thunderclouds have a large vertical extent, whereas in winter, thunderclouds are shallower in the vertical but tilted and elongated in the horizontal by strong winds that increase with altitude.
Cited articles
Athey, S., Tibshirani, J., and Wager, S.: Generalized random forests, Ann. Stat., 47, 1148–1178, https://doi.org/10.1214/18-AOS1709, 2019. a
Baran, S. and Nemoda, D.: Censored and shifted gamma distribution based EMOS model for probabilistic quantitative precipitation forecasting, Environmetrics, 27, 280–292, https://doi.org/10.1002/env.2391, 2016. a
Bauer, P., Thorpe, A., and Brunet, G.: The Quiet Revolution of Numerical Weather Prediction, Nature, 525, 47–55, https://doi.org/10.1038/nature14956, 2015. a
Breiman, L.: Bagging Predictors, Mach. Learn., 24, 123–140, https://doi.org/10.1007/bf00058655, 1996. a
Breiman, L.: Random Forests, Mach. Learn., 45, 5–32, https://doi.org/10.1023/a:1010933404324, 2001. a, b
Bremnes, J. B.: Ensemble postprocessing using quantile function regression based on neural networks and Bernstein polynomials, Mon. Weather Rev., 148, 403–414, https://doi.org/10.1175/MWR-D-19-0227.1, 2020. a
Evin, G., Lafaysse, M., Taillardat, M., and Zamo, M.: Calibrated ensemble forecasts of the height of new snow using quantile regression forests and ensemble model output statistics, Nonlin. Processes Geophys., 28, 467–480, https://doi.org/10.5194/npg-28-467-2021, 2021. a
Glahn, H. R. and Lowry, D. A.: The Use of Model Output Statistics (MOS) in Objective Weather Forecasting, J. Appl. Meteorol., 11, 1203–1211, https://doi.org/10.1175/1520-0450(1972)011<1203:tuomos>2.0.co;2, 1972. a, b, c
Gneiting, T. and Raftery, A. E.: Strictly Proper Scoring Rules, Prediction, and Estimation, J. Am. Stat. Assoc., 102, 359–378, https://doi.org/10.1198/016214506000001437, 2007. a
Gneiting, T., Raftery, A. E., Westveld III, A. H., and Goldman, T.: Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation, Mon. Weather Rev., 133, 1098–1118, https://doi.org/10.1175/mwr2904.1, 2005. a, b, c
Hamill, T. M., Bates, G. T., Whitaker, J. S., Murray, D. R., Fiorino, M., Galarneau, T. J., Zhu, Y., and Lapenta, W.: NOAA's second-generation global medium-range ensemble reforecast dataset, B. Am. Meteorol. Soc., 94, 1553–1565, https://doi.org/10.1175/BAMS-D-12-00014.1, 2013. a
Hothorn, T., Hornik, K., and Zeileis, A.: Unbiased Recursive Partitioning: A Conditional Inference Framework, J. Comput. Graph. Stat., 15, 651–674, https://doi.org/10.1198/106186006X133933, 2006. a
Hothorn, T., Hornik, K., Van De Wiel, M. A., and Zeileis, A.: Implementing a class of permutation tests: the coin package, J. Stat. Softw., 28, 1–23, 2008. a
Jordan, A. I., Krueger, F., Lerch, S., Allen, S., and Graeter, M.: scoringRules: Scoring Rules for Parametric and Simulated Distribution Forecasts, R package version 1.1.1, https://cran.r-project.org/web/packages/scoringRules/, 2023. a
Kneib, T., Silbersdorff, A., and Säfken, B.: Rage against the mean–a review of distributional regression approaches, Econometrics and Statistics, 26, 99–123, https://doi.org/10.1016/j.ecosta.2021.07.006, 2021. a
Lang, M. N., Lerch, S., Mayr, G. J., Simon, T., Stauffer, R., and Zeileis, A.: Remember the past: a comparison of time-adaptive training schemes for non-homogeneous regression, Nonlin. Processes Geophys., 27, 23–34, https://doi.org/10.5194/npg-27-23-2020, 2020. a
Lerch, S. and Thorarinsdottir, T. L.: Comparison of non-homogeneous regression models for probabilistic wind speed forecasting, Tellus A, 65, 21206, https://doi.org/10.3402/tellusa.v65i0.21206, 2013. a
Matheson, J. E. and Winkler, R. L.: Scoring rules for continuous probability distributions, Manage. Sci., 22, 1087–1096, 1976. a
Meinshausen, N. and Ridgeway, G.: Quantile regression forests, J. Mach. Learn. Res., 7, 983–999, 2006. a
Messner, J. W., Mayr, G. J., and Zeileis, A.: Heteroscedastic Censored and Truncated Regression with crch, R J., 8, 173–181, https://doi.org/10.32614/RJ-2016-012, 2016. a, b, c
Messner, J. W., Mayr, G. J., and Zeileis, A.: Nonhomogeneous boosting for predictor selection in ensemble postprocessing, Mon. Weather Rev., 145, 137–147, https://doi.org/10.1175/MWR-D-16-0088.1, 2017. a
Rasp, S. and Lerch, S.: Neural networks for postprocessing ensemble weather forecasts, Mon. Weather Rev., 146, 3885–3900, https://doi.org/10.1175/MWR-D-18-0187.1, 2018. a
Rigby, R. A. and Stasinopoulos, D. M.: Generalized additive models for location, scale and shape, J. Roy. Stat. Soc. C-App., 54, 507–554, https://doi.org/10.1111/j.1467-9876.2005.00510.x, 2005. a
Scheuerer, M.: Probabilistic Quantitative Precipitation Forecasting Using Ensemble Model Output Statistics, Q. J. Roy. Meteor. Soc., 140, 1086–1096, https://doi.org/10.1002/qj.2183, 2014. a
Schlosser, L., Stauffer, R., and Zeileis, A.: RainTyrol: Precipitation Observations and NWP Forecasts from GEFS, R package version 0.2-0/r2952, https://R-Forge.R-project.org/projects/partykit/ (last access: 15 November 2023), 2020. a
Schlosser, L., Lang, M. N., Hothorn, T., and Zeileis, A.: disttree: Trees and Forests for Distributional Regression, R package version 0.2-0/r3189, https://R-Forge.R-project.org/projects/partykit/ (last access: 15 November 2023), 2021. a
Schoenach, D., Simon, T., and Mayr, G. J.: Postprocessing ensemble forecasts of vertical temperature profiles, Adv. Stat. Clim. Meteorol. Oceanogr., 6, 45–60, https://doi.org/10.5194/ascmo-6-45-2020, 2020. a
Schulz, B. and Lerch, S.: Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison, Mon. Weather Rev., 150, 235–257, https://doi.org/10.1175/MWR-D-21-0150.1, 2022. a
Seibold, H., Zeileis, A., and Hothorn, T.: Individual treatment effect prediction for amyotrophic lateral sclerosis patients, Stat. Methods Med. Res., 27, 3104–3125, https://doi.org/10.1177/0962280217693034, 2018. a
Seibold, H., Zeileis, A., and Hothorn, T.: model4you: an R package for personalised treatment effect estimation, J. Open Res. Softw., 7, 17, https://doi.org/10.5334/jors.219, 2019. a, b
Simon, T., Mayr, G. J., Umlauf, N., and Zeileis, A.: NWP-based lightning prediction using flexible count data regression, Adv. Stat. Clim. Meteorol. Oceanogr., 5, 1–16, https://doi.org/10.5194/ascmo-5-1-2019, 2019. a
Stauffer, R., Mayr, G. J., Messner, J. W., Umlauf, N., and Zeileis, A.: Spatio-temporal precipitation climatology over complex terrain using a censored additive regression model, Int. J. Climatol., 37, 3264–3275, https://doi.org/10.1002/joc.4913, 2017a. a
Stauffer, R., Umlauf, N., Messner, J. W., Mayr, G. J., and Zeileis, A.: Ensemble postprocessing of daily precipitation sums over complex terrain using censored high-resolution standardized anomalies, Mon. Weather Rev., 145, 955–969, https://doi.org/10.1175/MWR-D-16-0260.1, 2017b. a
Taillardat, M., Mestre, O., Zamo, M., and Naveau, P.: Calibrated ensemble forecasts using quantile regression forests and ensemble model output statistics, Mon. Weather Rev., 144, 2375–2393, https://doi.org/10.1175/MWR-D-15-0260.1, 2016. a, b
Taillardat, M., Fougères, A.-L., Naveau, P., and Mestre, O.: Forest-based and semiparametric methods for the postprocessing of rainfall ensemble forecasting, Weather Forecast., 34, 617–634, https://doi.org/10.1175/WAF-D-18-0149.1, 2019. a
Thorarinsdottir, T. L. and Gneiting, T.: Probabilistic forecasts of wind speed: Ensemble model output statistics by using heteroscedastic censored regression, J. Roy. Stat. Soc. A Sta., 173, 371–388, https://doi.org/10.1111/j.1467-985X.2009.00616.x, 2010. a
Vannitsem, S., Bremnes, J. B., Demaeyer, J., Evans, G. R., Flowerdew, J., Hemri, S., Lerch, S., Roberts, N., Theis, S., Atencia, A., Bouallègue, Z. B., Bhend, J., Dabernig, M., De Cruz, L., Hieta, L., Mestre, O., Moret, L., Plenković, I. O., Schmeits, M., Taillardat, M., Van den Bergh, J., Van Schaeybroeck, B., Whan, K., and Ylhaisi, J.: Statistical postprocessing for weather forecasts: Review, challenges, and avenues in a big data world, B. Am. Meteorol. Soc., 102, E681–E699, https://doi.org/10.1175/BAMS-D-19-0308.1, 2021. a
Zeileis, A., Hothorn, T., and Hornik, K.: Model-based recursive partitioning, J. Computat. Graph. Stat., 17, 492–514, https://doi.org/10.1198/106186008X319331, 2008. a
Short summary
Statistical post-processing is necessary to generate probabilistic forecasts from physical numerical weather prediction models. To allow for more flexibility, there has been a shift in post-processing away from traditional parametric regression models towards modern machine learning methods. By fusing these two approaches, we developed model output statistics random forests, a new post-processing method that is highly flexible but at the same time also very robust and easy to interpret.
Statistical post-processing is necessary to generate probabilistic forecasts from physical...
