11 February 2023

Archives of composite weather radar images represent an invaluable resource to study the predictability of precipitation. In this paper, we compare two distinct approaches to construct empirical low-dimensional attractors from radar precipitation fields. In the first approach, the phase space variables of the attractor are defined using the domain-scale statistics of precipitation fields, such as the mean precipitation, fraction of rain, and spatial and temporal correlations. The second type of attractor considers the spatial distribution of precipitation and is built by principal component analysis (PCA). For both attractors, we investigate the density of trajectories in phase space, growth of errors from analogue states, and fractal properties. To represent different scales and climatic and orographic conditions, the analyses are done using multi-year radar archives over the continental United States (

Precipitation is challenging to forecast. The difficulty is due to its large space–time variability (e.g.

As a result, a rapid loss of precipitation predictability is observed for both extrapolation-based nowcasting and numerical weather prediction (NPW)-based forecasting (e.g.

Studies on atmospheric predictability are either model-based or observation-based: see a review in

Observation-based predictability studies comprise statistical extrapolation methods (e.g.

Inspired by

In this paper, we want to answer the following questions.

What do we learn about the predictability of precipitation from weather radar archives?

How do we define the phase space of the attractor?

What is the typical growth of errors from analogues?

How does predictability depend on scale?

In which way is the attractor relevant for short-term precipitation forecasting and stochastic simulation?

The paper is structured as follows. Section

The US and Swiss radar datasets are described in Table

Characteristics of the Swiss and US radar datasets.

The Swiss data comprise the quantitative precipitation estimation (QPE) product, which integrates measurements from three Doppler C-band weather radars

Radar-based QPE is inevitably affected by uncertainty due to e.g. the

Inspired by

calculating the correlation dimension log–log plot for increasing archive sizes (Fig.

detecting the “crossing” points, i.e. the smallest scaling distance between real analogues for increasing archive sizes (squares in Fig.

selecting the largest correlation dimension (corresponding to the largest archive) and extrapolating the linear fit to obtain the correlation integral

plotting the archive size vs. the

Estimation of the theoretical size of the archive needed to find good radar analogues at 4 km resolution in the US.

According to this analysis, for

An archive of radar rainfall fields can be structured as a temporal sequence of images into a 2D array of size

A common approach to studying non-linear dynamical systems is to look at the evolution of trajectories in the phase space of governing variables (e.g.

The attractor represents the subspace attracting the trajectories of atmospheric states, in our case radar precipitation fields, starting from any initial condition within the phase space. Chaotic systems, i.e. systems with sensitive dependence on initial conditions, never cross the same trajectory again and generate “strange” attractors, which have a non-integer intrinsic dimension (fractal dimension). The Lorenz system and the atmosphere are two examples of strange attractors. The strange attractor of this study is the ensemble of possible states and trajectories derived from weather radar images that are consistent with the precipitation climatology of a given region.

As time passes, divergence between two initially close states in phase space increases, which is generally referred to as

The established way of estimating the initial growth of errors, and therefore inferring the predictability of the system, is to compute Lyapunov exponents (e.g.

The analyses of the US attractor used only Eq. (

Fractal properties of the attractor can give useful insights into its intrinsic dimensionality. In this paper, we combined the time-delay embedding and Grassberger–Procaccia correlation dimension methods

The approach works by iteratively increasing the dimensionality of embedding space

There are many summary spatial and temporal statistics that can be extracted from precipitation fields and used as phase space variables, which we refer to here as

The

The

The

The

A radially averaged 1D power spectrum (RAPS) can be derived from the 2D Fourier power spectrum (see Appendix

Because rainfall rates often follow a log-normal distribution, it is more convenient to perform the Fourier transform on the reflectivity (

Several summary, spatial and temporal statistics were derived for the Swiss and US attractors, for instance:

WAR: wet area ratio

Area coverage: number of wet pixels over the radar composite domain. This is similar to WAR.

IMF: image mean flux

MM: marginal mean precipitation. Conditional mean precipitation (only wet pixels). Also referred to as conditional IMF. It can be computed in rain rate (mm h

Decorrelation time of precipitation fields, defined as the time when the temporal correlation falls below the value

Fourier analysis of the radar rainfall field at 17:00 UTC on 16 April 2016 in Switzerland.

Figure

Figure

US precipitation attractor. It is represented by 2D histograms (counts) using as phase space variables the marginal mean precipitation (MM), area coverage, decorrelation time, and eccentricity of composite radar precipitation fields. For illustrative purposes, panel

Swiss 4D precipitation attractor. The phase space variables are the marginal mean precipitation, the wet area ratio, and the two spectral slopes of the RAPS (

Figure

The four plots on the diagonal show the univariate histograms of the four variables and the corresponding summary statistics (mean and standard deviation).

The subplots in the upper triangular part of the matrix show the 2D histograms describing the density of points for all combinations of phase space variables (same as Fig.

The subplots in the lower triangular part of the matrix are a simplified representation of

These graphical illustrations represent a first useful insight into the attractor. For example, it is possible to distinguish between the stratiform and convective precipitation systems using combinations of phase space variables, in particular the eccentricity, spectral slopes and decorrelation time.

The domain-scale precipitation attractor provides additional insight into the origin of the spatial scaling break in the Fourier power spectrum. The scaling break was already noticed by previous studies, e.g.

Analysis of scaling break magnitude (

Figure

This brief analysis sheds new light on the origin of the scaling break in power spectra of precipitation fields, which is helpful for designing stochastic models

To estimate the fractal dimension of the attractor, we applied the time-delay embedding technique and correlation dimension method to each time series of phase space variables.

Correlation dimension estimation using the Grassberger–Procaccia algorithm and time-delay embedding on the variables fractional area coverage and marginal mean on the US attractor.

Figure

Instead of re-doing the same analysis with the Swiss attractor, in Sect.

Once the phase space is defined, we can study the intrinsic predictability of states starting from close initial conditions, the so-called

Average standard deviation of analogues in the US attractor as a function of lead time for phase spaces of increasing dimensionality. The

Figure

The reason for the slow error growth at 0–1 h is mostly unknown. It might be related to some radar data processing steps, in particular those that introduce smoothness in the precipitation field, which leads to overestimation of the predictability at the smallest scales. The rapid error growth at 1–6 h is attributed to the low predictability of precipitation growth and decay, especially in convective systems. The slower error growth at 6 h–20 d can be explained by the more predictable translation of synoptic-scale features across the continental US. Note that saturation already occurs after

These promising results show that there is unexpected intrinsic predictability of precipitation if the appropriate phase space variables are chosen. Given the substantial improvement of predictability in the range

Error growth of an ensemble of analogues starting from different initial conditions on the Swiss attractor. The upper end of the

Another interesting experiment is to analyse the local variability of predictability within the attractor. This could inform about the dependence of intrinsic predictability on initial location within the attractor

In order to simplify the task, here we only consider 1D trajectories, i.e. time series of individual phase space variables, hereafter extracted from the Swiss attractor. The small interval defining the initial conditions is selected by regularly spaced values between the 20th and 90th quantiles of each phase space variable. At each quantile, we select all the analogues that are within a small neighbourhood and that are at least 1 h from each other (to reduce dependence among analogues).

Figure

Most curves in the Swiss attractor miss the initial slow error growth observed in the US attractor (Fig.

These findings provide some useful information on the predictability that could be obtained by extrapolation nowcasts. In fact, the smaller size of the Swiss domain (as compared with the US) imposes a shorter limit of predictability, which could only be extended by enlarging the domain or by forecasting the evolution of domain-scale statistics.

Appendix

Domain-scale statistics are unable to describe the spatial distribution of precipitation, unless they are computed locally

The variables of the precipitation data matrix (Eq.

Due to the too large size of the US dataset, the radar fields were upscaled to a resolution of

Figure

Eigenvectors (loadings) extracted by PCA from the US radar archive.

Eigenvectors (loadings) extracted by PCA from the Swiss radar archive.

Figure

These shapes not only highlight the most common precipitation regimes but are also influenced by the rectangular shape of the domain and the orthogonality constraints of PCA. These dipole effects are known in the literature as

In an attempt to improve their interpretation, we implemented a varimax rotation of the principal components

Figure

PCA explains more variance with fewer components in the Swiss dataset. For instance, with 20 components the cumulative explained variances are 47.1 % and 35.5 % for the Swiss and US domains, respectively. This can be attributed to the smaller Swiss dataset but also to the more frequent orographic precipitation events related to the presence of the Alps, which determines more predictable spatial patterns on the upwind and downwind sides of the Alpine chain.

A common pattern for both Swiss and US attractors is that PCA decomposes the dataset into a set of eigenvectors that represent decreasing spatial scales, similarly to what is obtained with a Fourier-based cascade decomposition of precipitation fields

The sinusoidal patterns of eigenvector fields are an outcome of the Toeplitz-like nature of the covariance matrix of spatially correlated fields, whose eigenvectors represent sines and cosines of increasing frequencies. More precisely, for a stationary process the sinusoidal basis functions of the Fourier transform form a valid principal component basis, where the variance of each component represents the power spectrum

PCA derives the basis functions by decomposing an empirical covariance matrix. This may explain why in atmospheric science the principal components are referred to as

The similarity between PCA and Fourier decomposition creates interesting links to the cascade decomposition used in the Short-Term Ensemble Prediction System

Inspired by the relation to the cascade decomposition,

Figure

Trajectories of similar radar image sequences over the US in the space of standardized principal components. The dates of the 23 similar precipitation events are displayed on the right.

An interesting observation is that PC trajectories define quasi-regular trajectories, which result from the translation of precipitation systems from west to east. The most regular and illustrative PC shapes are found in the three sub-panels located as follows (row, column): (

This behaviour is not surprising: as explained in Sect.

The sorting of eigenvectors by spatial scale observed in Sect.

Explained variance and cumulative explained variance vs. the principal component number from the Swiss PCA.

The slow increase in cumulative explained variance does not allow us to define a clear cutoff level to truncate the principal components. These results do not leave a lot of optimism concerning the definition of a low-dimensional attractor for precipitation based on PCA. Instead, they point towards a stochastic approach for precipitation analysis and simulation.

Derivation of spatial wavelength from PC number.

One way of establishing an empirical relation to Fourier-based scaling analysis is to convert the ordinal PC numbers of Fig.

Compute the 2D Fourier spectra of the eigenvector fields (e.g. of Fig.

Derive the 1D RAPS from the 2D spectra.

Estimate the most representative wavelength

Plot the obtained wavelength against PC number in log–log scale (Fig.

Replace the PC number with the corresponding wavelength and plot it against the explained variance from Fig.

Figure

These findings point out that there is no universal relationship that maps the ordinal PC number to the spatial scale, as the latter depends on the covariance matrix of a given dataset. However, the method proposed above offers a simple and effective way of revealing the spatial scale represented by a given eigenvector.

Figure

Figure

The first minimum of the ACF was used as time delay

A major difficulty that we encountered in applying time-delay embedding is related to the short duration of precipitation events compared with the time delay

Finally, even though the fractal dimension estimates in this paper cannot be interpreted in absolute terms, they can be interpreted in relative terms. That is, lower PCs exhibit stronger chaotic behaviour than larger PCs, which have a more stochastic behaviour.

Correlation dimension analysis of principal component time series on the Swiss attractor.

Figure

Wav

Wav

PCA

PCA

PCA

Rotation (varimax): yes/no.

Random: random selection of analogues.

The forecast quality of analogues was measured by three continuous verification scores, i.e. mean absolute deviation (MAD), root-mean-square error (RMSE), Pearson's correlation, one categorical score (critical success index, CSI) and two probabilistic scores, i.e. the area under the ROC curve and the Brier score at the 1 dBZ threshold. The verification was done using 50 precipitation events in the US. For each of the 50 events, 25 analogues were selected based on the smallest Euclidean distance in PC space and forcing them to be at least 16 h from each other (for temporal independence). A 90 % threshold of the cumulative explained variance was used to define the dimensionality of the phase space.

Testing different PCA settings for retrieving analogues in the US. The dashed lines show the 25th and 75th percentile values of the score distribution.

According to MAD, RMSE and correlation, the best configuration is Wav

This analysis highlights that, once the attractor is defined, it is not that simple to retrieve analogue states. That is, practical implementation choices have an impact on the predictability estimations. Note that the low skill already at the start of the forecast (correlation

Similar to

This paper explored a framework to construct empirical low-dimensional precipitation attractors from multi-year archives of composite radar precipitation fields. The attractors were used to learn about the intrinsic predictability and various properties of precipitation fields. Data covering the Swiss Alps (2005–2010,

We tested two approaches to defining the attractor. The first approach uses as phase space variables selected domain-scale statistics of precipitation fields that are relevant for nowcasting applications, for example the precipitation fraction, mean precipitation and slopes of the Fourier power spectrum, which characterize the spatial autocorrelation. The second approach derives the phase space in a more objective way by principal component analysis, which also considers the location of precipitation.

After defining the phase space variables, we studied the fractal properties and error growth from analogues within both attractors. The pros and cons of the two types of attractors are summarized in Table

We could not find a unique objective way of defining the phase space of the attractor. That is, one is free to construct the attractor depending on the objective of the study or specific application.

Graphical representation of the attractor as the density of points in various combinations of phase space variables provides useful insight into data dependencies and precipitation regimes (e.g. stratiform vs. convective).

The magnitude of the scaling break in radially averaged power spectra of radar precipitation fields, previously observed by

Error growth from analogues retrieved by using domain-scale statistics starts slowly (0–1 h, reason mostly unknown), continues quickly (1–6 h, unpredictable convective precipitation growth and decay), and slows down again before predictability is lost to a large extent (6 h–20 d, more predictable synoptic scales).

The rate of error growth depends on the phase space used and the initial location within the attractor.

If the appropriate phase space variables are chosen, there is unexpectedly long intrinsic predictability of precipitation (several days), as shown with the US dataset.

Predictability of domain-scale statistics is longer in the US than CH, which is attributed mostly to the larger domain but also to the longer dataset.

By considering the spatial distribution of precipitation, PCA represents a useful framework for analysis, combination and simulation of precipitation fields.

Fourier analysis can be used to derive the spatial scales corresponding to eigenvector fields extracted by PCA.

The explained variance by PCA scales with both the ordinal PC number and corresponding spatial scale, which has a clear connection to Fourier-based decomposition of precipitation fields (e.g.

Fractal analysis of the principal component time series reveals that low PCs have a stronger chaotic contribution than high PCs, which have a stronger stochastic component.

Advantages and disadvantages of the two types of attractors.

The application of tools used in chaos theory, such as time-delay embedding and the correlation dimension method, is complicated by the precipitation intermittency, finite event duration, non-Gaussian distribution and multifractal properties. These difficulties are also reflected in the analysis of derived phase space variables (MM, WAR, PCs, etc.). In addition, the validity of theorems and assumptions from chaos theory is pushed beyond their limits because precipitation is the result not only of dynamical processes, but also of (stochastic) microphysical processes. It is important to mention that the current study did not have the ambition to demonstrate that the precipitation attractor is of (finite) low dimensionality (see the discussion in Appendix

Future perspectives comprise both improvements of the methodology and more practical applications. The methodology can be improved for example by exploring other phase space variables (e.g. orientation of anisotropy and precipitation translation speed), by using faster analogue retrieval methods

Concerning possible applications, it is not yet clear how to exploit the gathered knowledge to improve precipitation forecasting in practice. For instance, both NWP forecasting and stochastic nowcasting methods are known to underestimate the forecast uncertainty. That is, the ensembles are under-dispersive. One possibility would be to drive stochastic simulations with the large-scale features given by analogues. Another possibility could be to seamlessly blend forecast probabilities derived from extrapolation nowcasts, NWP models and analogues.

Finally, a completely different methodology, which has attracted the attention of the atmospheric science community for quite some time, relies on the training of machine learning algorithms to optimally extract the localized predictable patterns from the data

The discrete 2D power spectrum is defined as the squared norm of the complex Fourier transform:

The spatial autocorrelation function (ACF) is obtained via the Wiener–Khinchin theorem as the inverse Fourier transform of the power spectrum under the assumption of stationarity (e.g.

By assuming isotropy, from the 2D power spectrum we can derive a radially averaged 1D spectrum (RAPS):

Let

Set the value of pixels outside the radar domain to zero precipitation.

(Optionally) apply a logarithmic transformation to all precipitation values (use dBR or dBZ units; see Sect.

Centre the data matrix

Compute the covariance matrix to estimate the linear dependence of variables, i.e.

Diagonalize

(Optionally) rotate the eigenvectors to enhance interpretation

Project the original data matrix into the space spanned by eigenvectors, i.e.

An alternative way of performing PCA is by singular value decomposition (SVD) of the data matrix

Since SVD does not require the computation of the covariance matrix, it has larger numerical stability than EVD. However, SVD is slower than EVD if

For the Swiss archive, we used the SVD-based PCA decomposition available in the Python library sklearn

An important concept for studying non-linear dynamical systems is the

Takens' theorem is applied by lagging multiple times the time series of a state space variable:

The correlation dimension method, known as the Grassberger–Procaccia algorithm, estimates the fractal dimension of an attractor by counting the number of points that are contained in a

The Grassberger–Procaccia algorithm is known to underestimate the fractal dimension, which was the subject of controversial discussions questioning claims about the existence of low-dimensional attractors of atmospheric and hydrological processes (see e.g.

Example estimation of the correlation dimension by finding the maximum slope in a log–log plot of the correlation integral

Figures

Error growth of an ensemble of analogues starting from different initial conditions on the Swiss attractor. The

Error growth of an ensemble of analogues starting from different initial conditions on the Swiss attractor. Both axes are logarithmic. The spread is computed according to Eq. (

Error growth of an ensemble of analogues starting from different initial conditions on the Swiss attractor. Both axes are logarithmic. The spread is computed according to Eq. (

Figure

Weather radar coverage in the continental US. Downloaded from

The code was developed in Python at MeteoSwiss and in IDL at McGill University and is not publicly available.

The US radar composite dataset compiled at McGill University is unfortunately not available. The Swiss composite radar dataset is also not currently available for free, but it will be made available in the future as part of the Open Government Data law. More information can be found here:

LF performed the analyses using the Swiss radar dataset and wrote the paper. BPT and AA performed the analyses using the US radar dataset. DN supported the implementation and interpretation of the results. MG, IS and UG supported the study with numerous discussions about radar precipitation, nowcasting and chaos theory. IZ originated the idea of the precipitation attractor and supervised the study with unparalleled enthusiasm and creativity for as long as he could.

The contact author has declared that none of the authors has any competing interests.

Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

This study was supported by the Swiss National Science Foundation Ambizione project

This research has been supported by the Swiss National Science Foundation SNSF (grant no. PZ00P2_161316).

This paper was edited by Stéphane Vannitsem and reviewed by two anonymous referees.