Journal cover Journal topic
Nonlinear Processes in Geophysics An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 1.558 IF 1.558
  • IF 5-year value: 1.475 IF 5-year
    1.475
  • CiteScore value: 2.8 CiteScore
    2.8
  • SNIP value: 0.921 SNIP 0.921
  • IPP value: 1.56 IPP 1.56
  • SJR value: 0.571 SJR 0.571
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 55 Scimago H
    index 55
  • h5-index value: 22 h5-index 22
Preprints
https://doi.org/10.5194/npg-2019-63
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/npg-2019-63
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

  20 Jan 2020

20 Jan 2020

Review status
A revised version of this preprint was accepted for the journal NPG and is expected to appear here in due course.

Beyond Univariate Calibration: Verifying Spatial Structure in Ensembles of Forecast Fields

Joshuah Jacobson1, William Kleiber1, Michael Scheuerer2,3, and Joseph Bellier2,3 Joshuah Jacobson et al.
  • 1Department of Applied Mathematics, University of Colorado Boulder
  • 2Cooperative Institute for Research in Environmental Sciences, University of Colorado Boulder
  • 3Physical Sciences Division, NOAA/Earth System Research Laboratory, Boulder, Colorado

Abstract. Most available verification metrics for ensemble forecasts focus on univariate quantities. That is, they assess whether the ensemble provides an adequate representation of the forecast uncertainty about the quantity of interest at a particular location and time. For spatially-indexed ensemble forecasts, however, it is also important that forecast fields reproduce the spatial structure of the observed field, and represent the uncertainty about spatial properties such as the size of the area for which heavy precipitation, high winds, critical fire weather conditions, etc. are expected. In this article we study the properties of a new diagnostic tool designed for spatially-indexed ensemble forecast fields. The metric is based on a level-crossing statistic that we term the fraction of threshold exceedance (FTE), and is calculated for the verification field, and separately for each ensemble member. The FTE yields a projection of a – possibly high-dimensional – multivariate quantity onto a univariate quantity that can be studied with standard tools like verification rank histograms. This projection is appealing since it reflects a spatial property that is intuitive and directly relevant in applications, though it is not obvious whether the FTE is sufficiently sensitive to misrepresentation of spatial structure in the ensemble. In a comprehensive simulation study we find that departures from uniformity of these so called FTE histograms can be indeed be related to forecast ensembles with biased spatial variability, and that these histograms detect shortcomings in the spatial structure of ensemble forecast fields that are not obvious by eye. For demonstration, FTE histograms are applied in the context of spatially downscaled ensemble precipitation forecast fields from NOAA's Global Ensemble Forecast System.

Joshuah Jacobson et al.

Interactive discussion

Status: closed
Status: closed
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
Printer-friendly Version - Printer-friendly version Supplement - Supplement

Interactive discussion

Status: closed
Status: closed
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
Printer-friendly Version - Printer-friendly version Supplement - Supplement

Joshuah Jacobson et al.

Data sets

Verifying Spatial Structure in Ensembles of Forecast Fields J. Jacobson, J. Bellier, M. Scheuerer, and W. Kleiber https://doi.org/10.5281/zenodo.3592495

Model code and software

FTE J. Jacobson https://doi.org/10.5281/zenodo.3592506

Joshuah Jacobson et al.

Viewed

Total article views: 388 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
262 106 20 388 28 27
  • HTML: 262
  • PDF: 106
  • XML: 20
  • Total: 388
  • BibTeX: 28
  • EndNote: 27
Views and downloads (calculated since 20 Jan 2020)
Cumulative views and downloads (calculated since 20 Jan 2020)

Viewed (geographical distribution)

Total article views: 356 (including HTML, PDF, and XML) Thereof 355 with geography defined and 1 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 

Cited

Saved

No saved metrics found.

Discussed

No discussed metrics found.
Latest update: 08 Aug 2020
Publications Copernicus
Download
Short summary
Most verification metrics for ensemble forecasts assess the representation of uncertainty at a particular location and time. We study a new diagnostic tool based on fractions of threshold exceedance (FTE) which evaluates an additional important attribute: the ability of ensemble forecast fields to reproduce the spatial structure of observed fields. The utility of this diagnostic tool is demonstrated through simulations and an application to ensemble precipitation forecasts.
Most verification metrics for ensemble forecasts assess the representation of uncertainty at a...
Citation