The KENDA system employs the LETKF as suggested by . This formulation allows us to easily add new observations to the assimilation, while the core implementation can be kept once it is implemented. In the KENDA data assimilation system, the deterministic member represents the best estimate of the atmospheric state, while the ensemble members capture the range of possible states and uncertainties used to generate corrections to improve the deterministic member.

The formulation of – see also Chap. 5 – solves the Kalman filter equations in ensemble space defined by the ensemble members x(b,ℓ) for ℓ=1,…,L minus the ensemble mean 1x¯(b):=1L∑ℓ=1Lx(b,ℓ). We use the notation 2Xb:=(x(b,1)-x¯(b),…,x(b,L)-x¯(b)) for the matrix of ensemble differences from the mean and (for the case of linear H) 3Yb:=HXb for the ensemble differences in observation space, with yo the observation vector and y¯b the mean of observations simulated from the ensemble. The observation error covariance matrix is denoted by R. Now, we employ Eqs. (20) and (21) of , i.e., 4w‾a=P̃a(Yb)TR-1(yo-y‾b), to calculate the mean of the analysis ensemble and P̃a given by 5P̃a=[(L-1)I+(Yb)TR-1Yb]-1, where we use the letter L for the number of ensemble members, the notation w‾a for the linear coefficients of the analysis mean, and I for the identity matrix. P̃a denotes the L×L analysis covariance in the space of ensemble coefficients. Equation (2) in the model space leads to Eqs. (22) and (23) of : 6x‾a=x‾b+Xbw‾a,7Pa=XbP̃a(Xb)T, where x‾a is the analysis mean and Pa is the analysis covariance matrix. W is calculated by 8W=[(L-1)P̃a]1/2. As in Eq. (24) of , the analysis ensemble is calculated by 9Xa=XbW, where the power 1/2 denotes the symmetric square root of the symmetric matrix P̃a given by Eq. ().

It is obvious that, in the case where the ensemble of simulated reflectivities has a small or zero spread, the matrix Yb (see Eq. ) has small or zero entries, and in that case both Pa (see Eq. ) and the transform matrix W (see Eq. ) are small, such that the ensemble analysis increments given by Xa (see Eq. ) are small as well. The goal of targeted covariance inflation is to change Yb in such a way that the reflectivity observations lead to appropriate increments in the humidity and further variables. The basic challenge of different approaches to the TCI is how to construct the inflated matrix Yb such that the increments avoid spurious noise and generate meaningful convective processes in the model propagations following the analysis steps in a cycled data assimilation framework. We will develop the process-oriented and conditional approaches in Sect. .

The ICON modeling framework is the numerical weather prediction and climate modeling system collaboratively developed by various institutions and weather services where the Deutscher Wetterdienst (DWD) and the Max Planck Institute for Meteorology (MPI-M) are major contributors. At DWD, the ICON system runs operationally on a global scale, within the European subdomain known as ICON-EU and in the convection-permitting local-area-mode ICON-D2. The model domain of ICON-D2 covers all of Germany, Switzerland, Austria, and parts of the other neighboring countries; see Fig. . Therefore, the ICON-D2 model is very similar to that of the former operational COSMO-D2 model

COSMO refers to the Consortium for Small-Scale Modeling.

In the convective-scale ICON-D2 configuration of the ICON model, 3D radar observations obtained from the German radar network are employed . The German radar network consists of 17 dual-polarization C-band Doppler radar stations that comprehensively cover Germany (see Fig. ). The scanning procedure for 3D-volume scans at each radar station involves a complete 360° azimuthal sweep with a 1° resolution at 10 elevation angles ranging from 0.5 to 25°. Radially, the distance reaches up to 180 km for each station, with a resolution of 1 km.

To assimilate 3D radar observations, synthetic 3D radar data are derived from model variables utilizing the EMVORADO forward operator , where, considering only its single-polarization implementation in this study, Doppler velocities and reflectivities are computed. Note that simulated radar observations are produced in the observation space; i.e., for each observation, an associated model equivalent is computed. Furthermore, EMVORADO accounts for various intricate physical factors related to the simulation of radar measurements, e.g., beam bending, beam broadening, beam shielding, Doppler velocity with fall speed and reflectivity weighting, attenuated reflectivity, and a detectable signal. The EMVORADO operator also allows superobbing, i.e., the local spatial averaging of observations and corresponding observation equivalents as a standard technique for assimilating spatially high-resolution observations. For more comprehensive information and specifics that are beyond the required scope of this work, please refer to .

The LHN mechanism is a feature of KENDA that enables the assimilation of radar-derived precipitation rates, independent of the assimilation of volume radar data. Note that these precipitation rates are derived from radar data supplied by the OPERA network (including a precipitation scan of the German radar network). For further details on the LHN approach and its integration into KENDA, please refer to and .

The computation of artificially simulated reflectivities Z of the TCI approach is based on an application of a specifically constructed model M. Considering the general functional form of this model, we assume a linear relationship between the simulated reflectivity perturbation δZ (restricted to heights h above mean sea levels of 3000 to 4000 m) and the simulated specific humidity perturbation δqv at a certain ICON level L. Formally, this may also be written as follows: 10δZi(λ,ϕ,h)=M(δqvi(λ,ϕ,L))=αL⋅δqvi(λ,ϕ,L), with the ensemble member index i, longitude λ, and latitude ϕ. We would like to point out that, firstly, this model lives within the ensemble perturbation space as only ensemble perturbations δZ and δqv are used as variables. Secondly, this is a height-based approach; i.e., the algorithm only differentiates between the heights of the radar observations and does not explicitly take the actual radar elevation angle into account.

To determine the coefficients αL, we perform a linear regression for each available value for the parameter L, i.e., the specific ICON level used for the independent variable qv. The definite value for the parameter L is then selected by finding the maximum position of the corresponding correlation coefficient ρL. As shown in Fig. , there is a clear maximum for this correlation coefficient of ρ=0.8  at L=30 (with α=16000 dB(Z) kg kg⁻¹).

The dataset used for each linear regression is constructed as follows:

All simulated reflectivity data (from all elevation angles of the volume scan) and specific humidity data from all ensemble forecasts are collected. The forecasts provide data every 10 min and start hourly for the first 24 h of an ICON-D2 assimilation cycle with 40 ensemble members. This cycle is initialized at 2019-06-03T00:00:00, which coincides with the beginning of the period studied in Sect. .

A filter is then applied to this initial large dataset to only include data representative of early-stage convective events, i.e., data from spatial and temporal points near newly emerging convective cells. For each assimilation date d0 and lead time t0, this filter includes only those horizontal positions (x0,y0) in the final dataset that satisfy the following conditions:

At time t=t0-10 min, within a 20 km radius of (x0,y0), the ensemble mean of the lowest available radar data bin is below 1 dBZ.

By only including data associated with convective processes, we intend to capture the most relevant correlations associated with convection and, therefore, construct a more process-oriented approach which will eventually be more capable of pulling the system state towards an environment that is likely to initiate convection and help the model dynamically produce reflectivity.

Finally, it is important to note that the algorithm we just discussed for constructing and selecting a TCI model, used to compute artificial reflectivities, is only loosely related to the overall algorithm that applies this selected TCI model based on specific observation selection rules, as discussed in Sect. . However, as we will demonstrate, both algorithms follow the same general principles and ideas.

Another important advancement compared to earlier versions of the TCI approach is that multiple specific conditions must be fulfilled before the TCI is applied for a specific observation. We found that this restriction of an application of the TCI to only a small subset of all available observations and, as a consequence, keeping the overall impact on the system state at a minimum is essential for keeping the negative effects of the TCI under control.

Some of the following operations involve the calculation of a moving average acting solely on the two horizontal dimensions. This is implemented as a centered convolution employing a normalized rectangular function of width β (given in kilometers) in both horizontal dimensions λ and ϕ as a kernel. Denoting such a kernel as fβ(λ,ϕ), the processing of an arbitrary field X can be written formally as 11X̃β(λ,ϕ,h)=∫-∞∞∫-∞∞dλ′dϕ′fβ(λ-λ′,ϕ-ϕ′)X(λ′,ϕ′,h).

In the following, we employ the Boolean field B(λ,ϕ,h) to specify, for each spatial position, whether the TCI should be active (where its value is “true” or “1”) and simulated reflectivity values modified or whether the TCI should be inactive (“false” or “0”) and simulated values left unmodified. Furthermore, this field is defined as being the result of a logical conjunction of several auxiliary Boolean fields Bi(λ,ϕ,h): 12B(λ,ϕ,h)=∏iBi(λ,ϕ,h), where each of these Boolean fields Bi(λ,ϕ,h) is the result of an individual condition check. 13Spread check:B1(λ,ϕ,h)≡1if σi[Z(i)(λ,ϕ,h)]<0.1 dB(Z),0otherwise.14Deterministic check:B2(λ,ϕ,h)≡1if Z̃β=10det(λ,ϕ,h)<1 dBZ,0otherwise.15Ensemble mean check:B3(λ,ϕ,h)≡1if μi[Z̃β=10(i)(λ,ϕ,h)]<1 dBZ,0otherwise.16Observation check:B4(λ,ϕ,h)≡1if Zobs(λ,ϕ,h)]>15 dBZ,0otherwise.17Height check:B5(λ,ϕ,h)≡1if 3000 m≤h≤4000 m,0otherwise. Note that we employ σi[X(i)] and μi[X(i)] to denote the spread and mean, respectively, of a variable X.

With B1, we ensure that we only make changes for observations whose associated ensemble spread is too small. The fields B2, B3, and B4 ensure that the deterministic member and the ensemble mean have to vanish, while, simultaneously, there has to be a sizable observed reflectivity; i.e., there has to be a large discrepancy between observed and simulated values. Note that the calculation of B2 and B3 relies on simulated fields that have been preprocessed by means of a moving average (defined in Eq. ). This is done to take possible spatiotemporal displacements between observed and simulated reflectivity cells into account. Finally, B5 ensures that the TCI is only applied for radar observations whose heights fall within a certain height range. This is important as the previously constructed TCI model (see Sect. ) is based on observations falling within this specific height range.

We conducted sensitivity tests on the parameters in Eqs. ()–(), varying them within a defined range to assess their impact on DA performance and short-term NWP forecasts. The results showed that small changes had minimal impact, and the final parameters were chosen to balance effective TCI application while minimizing noise.

The TCI modifies the simulated reflectivity of all ensemble members employing the linear model M (see Eq. ) defined in Sect. , but only for a specific subset of all observations which is specified by means of the logical field B. Formally, the inflated reflectivities Z′(i) of the ith ensemble member are then computed via the following rule: 18Z′(i)(λ,ϕ,h)=μi[Z(i)(λ,ϕ,h)]+Mδqṽβ=10(i)(λ,ϕ,l=30)if B(λ,ϕ,h) is true,Z(i)(λ,ϕ,h)otherwise. λ, ϕ, and h loop over all discrete spatial points for which a reflectivity is measured. Note that in Eq. () the field for qv that enters the TCI algorithm is preprocessed by means of a moving average for taking spatiotemporal displacements into account.

Overall, Eqs. () and () demonstrate that reflectivity perturbations are derived deterministically from specific humidity perturbations using a linear model – without the involvement of any random perturbations – and are then applied conditionally.

Additionally, we modify the observation error for each observation for which the ensemble is inflated. Usually, we use a global observation error of 10 dB(Z) for all radar observations in our quasi-operational setup. However, if the TCI is applied for a specific observation Zobs(λ,ϕ,h), which means that B(λ,ϕ,h) is true, the observation error is reduced to 2 dB(Z). This results in much more pronounced increments, and the system is pulled significantly more strongly towards these observations.

Note that we performed the same sensitivity checks on the reflectivity observation error as in our previous TCI study , and the results were consistent. These findings confirm that the observation error significantly influences the size of the increments, underscoring its importance in the TCI process. The chosen value for the observation error of 2 dB(Z) strikes a balance, ensuring that no excessively large increments are introduced while maintaining the effectiveness of the TCI application.

To integrate the TCI approach into the KENDA system (see Sect. ), the input data that eventually enter the LETKF system are preprocessed. Usually, these input data are supplied in the form of feedback files (containing all superobbed

Superobbing refers to the process of “thinning out” radar data by spatial means.

To study the effects of the TCI, we performed data assimilation cycles for ICON-D2 over the period from 2019-06-03T00:00 to 2019-06-20T00:00 at an hourly data assimilation frequency. This specific time frame extends for several days for which individual case studies have been selected by the RealPEP (Near-Realtime Quantitative Precipitation Estimation and Prediction) research group. It includes many typical convective events. The general meteorological situation and its temporal development over the course of the chosen time period are shown in Fig.  by means of the spatial fraction of reflectivities above a certain threshold of time.

To study the intrinsic effects of the TCI approach, we performed two assimilation cycles

Note that the terms “cycle” and “experiment” are used as synonyms here.

Overall, the configuration of these two assimilation cycles is basically the operational configuration. This includes the assimilation of all conventional data, the assimilation of 3D radar data from the German radar network, the assimilation of radar data obtained from radar precipitation scans via the LHN mechanism, and the usage of an ensemble of 40 members. In contrast to the operational assimilation, we did not include the all-sky assimilation of satellite data in our experiments, since the operationalization of these was carried out during the project execution.

At each data assimilation step, the LETKF generates increments for several model variables, particularly for the temperature T and the specific humidity qv, which by incremental analysis update (IAU) are fed into the system propagation throughout a certain time window centered around the assimilation time. However, it is important to note that, for our operational setup, there are no increments for hydrometeors other than qv, e.g., qr, qi, and qc.

Let us now look more closely at the TCI effects by exemplarily studying the details of an assimilation at t0=2019-06-05T15:00:00 and its impact on a subsequent model run up to 1 h after t0.

Let us begin with an illustration of certain internal details and immediate effects of the TCI algorithm, in particular the construction of the total Boolean field B and the computation of the final inflated reflectivities (see Sect.  for more information). For this purpose, Fig.  depicts selected Bi fields as well as their corresponding input fields, the total Boolean field B, and the final inflated reflectivities obtained from the TCI. Note that here we only visualize the radar data of one exemplarily chosen radar station at a single radar elevation angle to improve the clarity of this illustration.

Two things become apparent here: firstly, there are only very few spatially connected regions for which B is true, and, secondly, the TCI is successfully able to increase the reflectivity ensemble spread within these regions. Directly related to Fig. , an aggregation (taking the mean) over all of the radar stations and all of the elevation angles of the Boolean field B and difference in the ensemble spread is depicted in Fig. , allowing an overview of the complete model domain. Similarly to before, it becomes evident that there are only very few spatially connected regions within the complete domain that are compatible with the imposed conditions for an application of the TCI. Looking at the depicted difference between the reflectivity ensemble spread with and without the TCI, it also becomes directly clear that the TCI is able to increase the reflectivity ensemble spread if B is true. However, the ensemble spread is always kept unmodified for observations for which B is false.

As the TCI increases the spread by modifying all ensemble members of only a few carefully selected observations for which the spread would be vanishing otherwise, we enable the LETKF to include these otherwise discarded observations. This leads to altered increments that are produced by the LETKF, and we expect these increments to modify our system in a way that makes the generation of reflectivity more likely. It is important to note that, firstly, reflectivity is not a prognostic variable but merely a diagnostic variable, and, secondly, we are not updating hydrometeor variables that are directly connected to the simulation of reflectivity (e.g., qr). Therefore, increments do not directly affect reflectivities, but the model has to respond dynamically to increments for other variables, e.g., temperature and specific humidity, and it may eventually, after a short period of time, dynamically generate reflectivity through the generation of, e.g., qr and qg.

To observe how the TCI is able to let new reflectivity cells emerge, let us consider Fig. . By considering the depicted reflectivity composite at a lead time of 1 h, it becomes clear that the TCI is very often able to produce new simulated reflectivity cells that are consistent with observed reflectivity cells and that are not produced (or at least are not as pronounced) without application of the TCI. Thus, we can already observe a positive impact of the TCI here.

It is important to note that Fig.  illustrates the general trend observed for other assimilation dates and lead times that are not shown. For regions in which the TCI is active, the simulation of the TCI cycle is very often able to produce new reflectivities that are not present in the reference cycle. However, for very large observed values, the corresponding simulated values of the TCI cycle are usually smaller than the observed ones – which is, however, plausible as the model had only 1 h here to dynamically respond to the additional increments introduced by the TCI and the production of reflectivity. Note that two other possible explanations for this general trend are that the TCI is only applied over a small height range of 3 to 4 km, limiting the region that can be convectively destabilized, and that the relatively sluggish model begins its convection later than in reality, preventing it from evolving as quickly.

Furthermore, we would like to note that the source of possible differences between the two simulated reflectivity composites may be two-fold: firstly, the TCI has an effect through the very last assimilation step, and, secondly, the TCI has also been applied hourly at many assimilation steps before the very last one at t0, such that there is also an accumulation of its effects and, therefore, a substantial divergence of the background states of the TCI and the reference cycle at t0.

The direct impact of the TCI on convective initiation (without any possible accumulation effects) was investigated in , where the TCI was applied at single time steps in the context of non-cycled experiments. These “cold-start scenarios” demonstrated that the TCI is effective even when applied only once. In the current study, we confirmed that the TCI produces similar results in cold-start scenarios, generating reflectivity even when applied at a single time step instead of through cycling.

Let us now further formalize and quantify the verification of reflectivities at a single time step as shown in Fig.  by employing a special version of the FSS designed by to deal with highly structured fields such as reflectivity that are particularly susceptible to double penalties .

The FSS is a popular spatial verification metric that is also used in this work for the spatial verification of reflectivities for longer-term experiments in Sect.  and, thus, it is particularly important for the overall evaluation of the TCI approach. We denote 2D fields for observations and associated predictions as Y(i,j) and X(i,j), respectively, where i and j are indices for the two horizontal dimensions. Considering an arbitrary 2D field A(i,j), we use the notation A^β,τ(i,j) to refer to the so-called fraction-of-occurrences field. The fraction-of-occurrences field is defined for each spatial point as the fraction of spatial points whose value is above the threshold τ with respect to all spatial points lying within a certain spatial neighborhood around this point – defined by a 2D box with box length β. The FSS with respect to these two input fields X and Y, the box length β, and the dBZ threshold τ may now be written as follows: 19FSSX,Y,β,τ=1-∑i,jX^β,τ(i,j)-Y^β,τ(i,j)2∑k,lX^β,τ(k,l)2+∑k,lY^β,τ(k,l)2. By comparing two different predictions X and X′ by means of the difference of their corresponding FSS values with each other, we obtain 20FSSX′,Y,β,τ-FSSX,Y,β,τ=∑i,jΔfssX′,X,Y,β,τ(i,j), where we inserted Eq. (), combined both sums in the resulting expression, and then implicitly defined the 2D field ΔfssX′,X,Y,β,τ(i,j) as the argument of this combined sum. Evidently, from the sign and magnitude of ΔfssX′,X,Y,β,τ(i,j), we may assess how much the prediction X′(i,j) improves (positive values) or worsens (negative values) the overall FSS with respect to the reference prediction X(i,j) and observation Y(i,j).

Following these considerations, Fig. d shows the aforementioned 2D field ΔfssX′,X,Y,β,τ(i,j) based on simulated reflectivities of the TCI cycle for X′, simulated reflectivities of the reference cycle for X, observed reflectivities for Y, β=20 km, and τ=15 dBZ. Observing that most of the values are positive, it becomes evident that the TCI is predominantly improving the FSS. This qualitative first impression is confirmed by computing the FSS related to the reference cycle reflectivity composite and the FSS related to the TCI cycle reflectivity composite, which amount to 0.796 and 0.826, respectively. Therefore, we obtain a relative improvement in the FSS of the TCI cycle of about 3.69 %.

Note that we are only evaluating a single point in time here.

Interestingly, Fig.  demonstrates that the TCI improves reflectivity not only in the near vicinity of regions for which the TCI is applied in the very last assimilation step (indicated by the red contours), but also for many other spatial regions, hinting at an accumulated impact of the TCI on the background state dating back to assimilation steps before the very last one.

Let us now proceed to a more statistical view of the TCI effects by studying different statistics and scores of longer-term NWP experiments covering a period of about 17 d. Note that the specific configuration of these experiments, including the setup of their assimilation cycles and free forecasts, was already discussed in Sect. .