Hourly precipitation over a region is often simultaneously simulated by numerical models and observed by multiple data sources. An accurate precipitation representation based on all available information is a valuable result for numerous applications and a critical aspect of climate monitoring. The inverse problem theory offers an ideal framework for the combination of observations with a numerical model background. In particular, we have considered a modified ensemble optimal interpolation scheme. The deviations between background and observations are used to adjust for deficiencies in the ensemble. A data transformation based on Gaussian anamorphosis has been used to optimally exploit the potential of the spatial analysis, given that precipitation is approximated with a gamma distribution and the spatial analysis requires normally distributed variables. For each point, the spatial analysis returns the shape and rate parameters of its gamma distribution. The ensemble-based statistical interpolation scheme with Gaussian anamorphosis for precipitation (EnSI-GAP) is implemented in a way that the covariance matrices are locally stationary, and the background error covariance matrix undergoes a localization process. Concepts and methods that are usually found in data assimilation are here applied to spatial analysis, where they have been adapted in an original way to represent precipitation at finer spatial scales than those resolved by the background, at least where the observational network is dense enough. The EnSI-GAP setup requires the specification of a restricted number of parameters, and specifically, the explicit values of the error variances are not needed, since they are inferred from the available data. The examples of applications presented over Norway provide a better understanding of EnSI-GAP. The data sources considered are those typically used at national meteorological services, such as local area models, weather radars, and in situ observations. For this last data source, measurements from both traditional and opportunistic sensors have been considered.

Precipitation amounts are measured or estimated simultaneously by multiple observing systems, such as networks of automated weather stations and remote sensing instruments. At the same time, sophisticated numerical models simulating the evolution of the atmospheric state provide a realistic precipitation representation over regular grids with the spacing of a few kilometers. An unprecedented amount of rainfall data is available nowadays at very short sampling rates of 1 h or less. Nevertheless, it is a common experience within national meteorological services that the exact amount of precipitation, to some extent, eludes our knowledge. There may be numerous reasons for this uncertainty. For example, a thunderstorm triggering a landslide may have occurred in a region of complex topography where in situ observations are available but not exactly at the landslide spot; thus, weather radars may cover the region in a patchy way because of obstacles blocking the beam, and numerical weather prediction forecasts are likely to misplace precipitation maxima. Another typical situation is when an intense and localized summer thunderstorm hits a city. In this case, several observation systems are measuring the event and more than one numerical model may provide precipitation totals. From this plurality of data, a detailed reconstruction of the event is possible, provided that the data agree both in terms of the event intensity and on its spatial features. This is not always the case, and sometimes meteorologists and hydrologists are left with a number of slightly different but plausible scenarios.

The objective of our study is the precipitation reconstruction through a combination of numerical model output with observations from multiple data sources. The aim is that the combined fields will provide a more skillful representation than any of the original data sources. As remarked above, any improvement in the accuracy and precision of precipitation can be of great help for monitoring the weather, but it is not only that. Snow- and hydrological- modeling will benefit from improvements in the quality of precipitation, which is one of the atmospheric forcing variables

The data sources considered in our study are precipitation ensemble forecasts, observations from in situ measurement stations, and estimates derived from weather radars. Numerical model fields are available everywhere, and the quality of their output is constantly increasing over the years. The weather-dependent uncertainty is often delivered in the form of an ensemble. At present, assessments using hydrological models have shown that input from numerical models “may be comparable or preferable compared to gauge observations to drive a hydrologic and/or snow model in complex terrain”, as stated by

Inverse problem theory

The innovative part of the presented approach to statistical interpolation is in the application to spatial analysis of concepts that are usually encountered in DA. The formulation of the problem is adapted to our aim, which is improving precipitation representation instead of providing initial conditions for a physical model, as it is for DA. In the literature, there are a number of articles describing similar approaches applied to precipitation analysis, such as

The remainder of the paper is organized as follows. Section

We assume that the marginal probability density function (PDF) for the hourly precipitation at a point in time follows a gamma distribution

Precipitation fields are regarded as realizations of locally stationary and transformed Gaussian random fields, where each hour is considered independently from the others. The time sequence of EnSI-GAP simulated precipitation fields shows temporal continuity because this is present in both observations and background fields. Transformed Gaussian random fields are used for the production of observational precipitation gridded data sets by

An implementation of EnSI-GAP is reported in Algorithm 1.

Overview of variables and notation for global variables. All the vectors are column vectors unless otherwise specified. If

Overview of variables and notation for local variables. All variables are specified in the transformed space. All the vectors are column vectors unless otherwise specified. If

The data transformation chosen is a Gaussian anamorphosis

Algorithm 1 can be divided into the following three parts that are described in the next sections: the data transformation in Sect.

The Gaussian anamorphosis maps a gamma distribution into a standard Gaussian.

The hourly precipitation background and observations,

In this paragraph, the procedure used in Sect.

In Gaussian anamorphosis, zero precipitation values must be treated as special cases, as explained by

The transformation function

For the presented implementation of EnSI-GAP, the Gaussian anamorphosis is based on the constant parameters of

The spatial analysis in Algorithm 1 has been divided into three parts. In Sect.

In Bayesian statistics, according to

The analysis is the best estimate of the truth, in the sense that it is the linear, unbiased estimator with the minimum error variance. The analysis is defined as

As for linear filtering theory

The

Our definitions of the error covariance matrices follow from a few general principles that we have formulated. For P1 (i.e., general principle 1; hereinafter the same definition applies for other references to P), background and observation uncertainties are weather and location dependent. For P2, the background is more uncertain, where either the forecast is more uncertain or observations and forecasts disagree the most. For P3, observations are a more accurate estimate of the true state than the background. We want to specify how much more we trust the observations than the background in a simple way, such as, for example, “we trust the observations twice as much as the background”. For P4, the local observation density must be used optimally to ensure a higher effective resolution, as it has been defined in Sect.

P1 and P4 led to our choice of implementing Algorithm 1 by means of a loop over grid points. P2 will lead us to the identification of the regions in which the uncertainty on the input data is greatest. P3 will be used to define the observational uncertainty with respect to that of the background.

A distinctive feature of our spatial analysis method is that the background error covariance matrix

The local observation error covariance matrix

The spatial structures of the error covariance matrices are determined through the matrices in Eqs. (

As a final step, to set

The second situation is when the ensemble spread is consistent with the empirical estimate of

The third situation is the special case in which the background is deemed as perfect; that is, when all the observed values and all the forecasts, at all observation locations, have the same value. In practice, this occurs in the case of no precipitation. In this case,

With reference to the working assumptions stated at the beginning of this section, they can now be reformulated in more precise mathematical terms by referring to the above definitions and equations. P1 led us to Eqs. (

The expressions for the analysis and its error variance are direct results of the linear filter theory

The special case of a perfect background, as introduced in Sect.

The inverse transformation

The inverse transformation at the

Given

The aim of this section is to provide guidance on the implementation of EnSI-GAP for some applications that we consider important or useful for understanding how it works.

In Sect.

In Sect.

Section

The aim of this section is to show how EnSI-GAP works and to assess its performances with different configurations under idealized conditions. The impacts of Gaussian anamorphosis and different specifications of background error covariances are also investigated. The functioning of the algorithm is shown with the example application to a single simulation. The conclusions on the EnSI-GAP pros and cons are based on the statistics collected over 100 simulations.

A one-dimensional grid with 400 points and a spacing of 1 spatial unit, or

One-dimensional simulation.

The simulation presented here is shown in Fig.

The ensemble background (gray lines in Fig.

The number of observations (blue dots in Fig.

The effect of the Gaussian anamorphosis is shown in Fig.

An example application of EnSI-GAP is presented in Algorithm 1. The choices that are kept fixed and that will not vary for the whole Sect.

The evaluation of analysis versus truth at grid points are evaluated using two scores that are applied over precipitation values. The mean squared error skill score (MSESS) quantifies the agreement between the analysis expected value and the truth. The continuous ranked probability score (CRPS) is a much used measure of performance for probabilistic forecasts. The definitions of both scores can be found, for example, in

A sensitivity analysis on variations in the scaling parameters

The sensitivity study considers three situations which are also used in Figs.

One-dimensional simulation. Error variances (dimensionless quantities) for different configurations of the scaling parameters. The variances shown are

The scaling of the covariances, which in turn determines the weights used in the analysis, is determined by

One-dimensional simulation in the transformed precipitation space. Analyses at grid points with different EnSI-GAP configurations. For all panels,

The transformed precipitation analysis is shown in Fig.

One-dimensional simulation in the original precipitation space (in millimeters). Analyses at grid points with different EnSI-GAP configurations. The layout is the same as in Fig.

By comparing Figs.

The analysis of over 100 simulations confirms the considerations we have made above on the basis of a single simulation. If we consider 100 simulations, the results are shown in Table

Summary statistics on the evaluation of the 100 one-dimensional simulations. Results are presented for three modes, namely EnSI-GAP, no transformation, which is EnSI-GAP without applying the Gaussian anamorphosis, and no ensemble, which is EnSI-GAP where the background is the ensemble mean, and the background error covariance matrix is determined solely by the scale matrix. The configurations listed are the same as those that have been used in Figs.

In Fig.

One-dimensional simulation in the original precipitation space (in millimeters). Analyses at grid points with different EnSI-GAP configurations, without applying the data transformation. The layout is the same as in Fig.

The comparison of the analysis spread between Figs.

If we consider 100 simulations, the results are reported in the column “no transformation” of Table

In Fig.

One-dimensional simulation in the original precipitation space (in millimeters). Analyses at grid points with different EnSI-GAP configurations, without considering the whole ensemble. Specification of the scale matrix

When comparing the three different configurations, the general considerations are the same as in Sects.

If we consider 100 simulations, the results are reported in the column labeled no ensemble in Table

If we consider the 100 simulations on the one-dimensional grid, the comparison of results in Table

The comparison between exponential and Gaussian correlation functions in

The EnSI-GAP implementation in Algorithm 1 requires the specification of four parameters, namely

The data used in this section are those used in the operational daily routine at the Norwegian Meteorological Institute (MET Norway). The forecasts are from the MetCoOp Ensemble Prediction System

A mass of moist air from the ocean, moving towards the Norwegian mountains, originated from several intense showers over western Norway on 30 July 2019. South Norway, the domain considered, is shown in Fig.

South Norway domain used in the simulations of Sects.

The two domains of South Norway and Sogn og Fjordane have been chosen to showcase two typical situations that can be found in an operations center. In both domains, the focus is on the representation of hourly precipitation patterns at the mesoscale, as defined by

Algorithm 1 has been used over a grid with 2.5

As an example of application, the Gaussian anamorphosis described in Sect.

Data transformation procedure. Example for 30 July 2019, 15:00 UTC, hourly precipitation totals over Sogn og Fjordane (see Fig.

The four different steps of the data transformation for an arbitrary value, at approximately 2

Figure

Hourly precipitation totals for 30 July 2019, 15:00 UTC (

Hourly precipitation totals for 30 July 2019, 15:00 UTC (

One of the main innovations of EnSI-GAP, compared to traditional spatial analysis methods

Background error correlations for 30 July 2019, 15:00 UTC,

The evolution in time of the hourly precipitation fields is shown in Fig.

Hourly precipitation totals (in

The time series of hourly precipitation at points A and B are shown in Fig.

Time series of hourly precipitation totals for the period 30 July 2019, 10:00 to 23:00 UTC, at points A

EnSI-GAP can support weather forecasters and civil protection by filling in the empty spaces in the observational networks. The analysis seamlessly merges the high-resolution NWP models with observations, and it remains closer to the observed values where they are available. The predicted fields are easy to interpret by experienced staff that are aware of the spatial distribution of the observations and the characteristics of the NWP considered. The analysis is more precise and accurate than the background where observations are available, as at point A in Sect.

In Sect.

The cross-validation experiments have been conducted over the South Norway domain shown in Fig.

The EnSI-GAP Algorithm 1 has been used. The spatial analysis predicts values at those station locations used for cross-validation. The fixed parameters in this implementation are

The parameters that are allowed to vary and that are the objective of the sensitivity analysis that follows are

There is an important difference here with respect to Sect.

Summer 2019 hourly precipitation statistics for the cross-validation experiments.

Figure

Figure

Equitable threat score (ETS) for summer 2019 hourly precipitation, as obtained through the cross-validation experiments. The black lines are the ETS curves for the analysis mean values, as indicated in the legend. The ETS curve for the background is the gray line. The precipitation thresholds defining the “yes” events are reported on the

The ensemble-based statistical interpolation with Gaussian anamorphosis (EnSI-GAP) applies the inverse problem theory to the spatial analysis of hourly precipitation. Numerical model output provides the prior information, and specifically, we have considered ensemble forecasts that have been combined with radar-derived estimates and in situ observations. EnSI-GAP has been applied on data sets that are typically available within national meteorological services. In addition, opportunistic sensing networks based on citizen observations have been considered. The precipitation representation is a synthesis of all the data available. Thanks to the diffusion of open data policies, the same data sets are nowadays also available in real time to the general public. For instance, MET Norway provides free access to the weather forecasts and the radar data used in this article via

EnSI-GAP assumes the precipitation fields to be locally stationary and transformed Gaussian random fields. The marginal distribution of precipitation at a point is a gamma distribution, which is different for each point. Gaussian anamorphosis is used to preprocess data in order to better comply with the requirements of linear filtering. A special case is considered where uncertainties are so small that the returned analysis values have delta functions as their marginal distributions.

EnSI-GAP considers each hour independently, and it requires the specification of four parameters that can vary across the domain. The implementation is designed to run in parallel on a grid point by grid point basis. Despite the small number of parameters to optimize, the spatial analysis scheme is flexible enough that it can also be applied when the background ensemble is not representing the truth satisfactorily. An important case is when, in a region, all the ensemble members show no precipitation, while the observations report precipitation. By adding a scale matrix to the flow-dependent background error covariance matrix, the analysis can predict precipitation – even where the background is sure that it is not occurring.

The examples of the applications presented allow for a better understanding of the characteristics of EnSI-GAP, and they show how the statistical interpolation can be adapted to meet specific requirements. It can be used to fill in the gaps between observation-rich regions to obtain a continuous precipitation field. The analysis expected value is available everywhere, as it is the background, and in observation-dense regions it can be as accurate as the observations. Thanks to the data transformation, the spread of the analysis PDF is less likely to become unrealistically large either because of large model errors or large variability in observed small-scale precipitation. Within certain limits determined by the spatial distribution of the observational network, the analysis envelope at a point can be tuned such that it is representative of the distribution of precipitation values determined by atmospheric processes occurring at smaller spatial scales than those resolved by the background. For instance, in an observation-void region, the EnSI-GAP analysis PDF at a point provides a better estimate than the background for the probability of precipitation exceeding a threshold by an observation hypothetically placed at that point. This is an important result, especially when high-impact weather is involved.

Some of the data sets used in Sects.

CL developed EnSI-GAP, tested it on the case studies, and prepared the paper with contributions from all coauthors. TNN and IAS configured EnSI-GAP to work with MET Norway's data sets, collected in situ observations from opportunistic sensing networks, and quality controlled them. CAE prepared the radar data.

The authors declare that they have no conflict of interest.

This research was partially supported by RADPRO (Radar for Improving Precipitation Forecast and Hydropower Energy Production), an innovative industry project funded by the Research Council of Norway (NFR) and partnering hydropower industries.

This research has been supported by the Norwegian Water Resources and Energy Directorate and the Norwegian Meteorological Institute (project “Felles aktiviteter NVE-MET tilknyttet: nasjonal flom- og skredvarslingstjeneste”).

This paper was edited by Alberto Carrassi and reviewed by two anonymous referees.