The outsize impact of extreme weather events, and the need to understand the physical processes that cause them, have driven substantial research interest in the tails of climatological probability distributions. The fundamental challenge is scarcity of data: the historical record is too short to enable robust estimation of extremes rarer than a few times per century, even if the climate were stationary. Different modeling paradigms have developed to confront the issue. The most straightforward is direct numerical simulation (DNS), whereby a climate model is integrated extensively and the extreme events tallied, either as a single long run with stationary forcing e.g., or as an ensemble with non-stationary forcing e.g.,. This increases the sample size of extreme events, and reduces the relative error (mean/standard deviation) of an empirical estimate p^=#extremesN=#totalsamples, but at a slow rate of V[p^]E[p^]=p(1-p)/Np∼(Np)-1/2 for p≪1 . For example, estimating the probability of a once-per-century storm (p=0.01 yr⁻¹) to within 10 % relative error would take roughly N=10.01(0.1)-2=104 model years. Most of that simulation time is wasted, just waiting for the next event.

Rare event sampling (RES) takes a shortcut by repurposing that time to generate more extremes instead, perturbing simulations in a targeted way to favor extreme behavior – with the tradeoff of having to account for bias properly. The need to track probabilities makes rare event sampling distinct from optimization, i.e., finding the most extreme event possible (or plausible) given physical constraints. That problem has been attacked successfully with constrained optimization algorithms by and for extreme dissipation events in turbulence, and in AI-based weather forecasting by for extreme heat waves. RES can benefit from these techniques, but aims to represent the entire tail distribution of extremes with statistical fidelity and not just the maximum.

RES was first developed for nuclear safety assessment , and has since been generalized for diverse applications including structural reliability engineering , molecular dynamics , and more recently climate and weather e.g.,. RES stands in contrast to many other strategies which, in one way or another, replace the expensive physical model with a cheaper approximation. Extreme value theory gives principles for parametrically estimating distributions tails , but its asymptotic assumptions are not always justified by the finite datasets available, and it is best suited to model univariate distributions (e.g., average temperature over a region) rather than full spatiotemporal processes like storms, although spatial extreme value modeling is steadily progressing . Hybrid statistical/physical models aim to parameterize physical processes rather than the final output statistics, and include linear inverse models ; stochastic weather generators based on analogues or Markov state models ; empirical downscaling ; statistical (including machine-learned) emulation ; and generative modeling . Machine learning models in particular are proliferating at a dizzying pace, and they can indeed generate new samples at low cost, but their ability to represent physics outside their training data – perhaps the most essential requirement for extreme event modeling – is rightly regarded with suspicion.

In light of these options, modelers have several tools to help deal with the tradeoff between bias (incorrect physics or limited resolution) and variance (erratic statistical estimates due to limited sample size). The methods are not mutually exclusive, with many interesting synergies possible e.g., as conceptualized in, but RES in particular is our focus here as an under-utilized and under-developed strategy to reduce variance without incurring extra bias.

The generic RES procedure can be summarized as follows. We denote the full state vector by x(t)∈Rd, and the measure of severity by R*: some functional of a trajectory x that is user-defined, e.g., rainfall averaged over any time interval and spatial region of interest.

Generate an ensemble of initial conditions to serve as candidate extreme events. Call these “ancestors”.

This template must be specialized for the kind of target event. Diffusion Monte Carlo (DMC), as applied to season-long hot extremes with a variant called “GKTL” after its inventors; and tropical cyclones with a variant called “QDMC” that applies quantile mapping to intensity values;, performs the split/kill operation at a chronological sequence of time points, extending the timespan of surviving members while aborting discarded members before they can run to completion—thus, before their R* values can even be measured. This is appropriate when the propensity for a future extreme R* is well-approximated by some property R(x(t)) measurable at the present: for example, if R* is the mean temperature from June to August, R(x(t)) = (running average temperature from 1 June to t) is a good splitting criterion . If R* is peak wind speed over a tropical cyclone's lifetime, R(x(t))= (minimum sea-level pressure in the eye) is a good splitting criterion .

But suppose that no good predictor exists. In particular, assume that the severity function R* of a simulation is the maximum over the event's timespan of a user-defined observable R(x(t)), such as the accumulated rainfall over a small region between t-1 day and t, which we generically call the intensity function. Assume further that no better predictor for R* is known besides R itself at the present time. In this case, a better choice of RES algorithm might be adaptive multilevel splitting AMS;, or more general versions such as “anticipated AMS” and “trying-early” AMS (TEAMS), which we previously introduced in – itself a special case of subset simulation from engineering – in which every ensemble member runs to completion and produces an actual value of R*, not some proxy for it. Descendants are then spawned from the ancestor at some advance split time (AST) A before R* is achieved, to give them enough time to diversify and perhaps exceed their ancestor's severity, but not so much time to forget their ancestor's special initial conditions. Figure  illustrates this tradeoff when selecting AST in the context of a simple stochastic system, namely Langevin dynamics with a logarithmic potential which is specified in Appendix A, but the picture alone conveys the essential phenomenon of an optimal AST. The existence of a nontrivial (i.e., strictly positive) optimum is obvious when looking at isolated events, but its precise value is subtle to quantify when our purpose relates to climatological statistics, i.e., averages over many events.

There is no general procedure for selecting AST and other hyperparameters, which impedes the application of RES methods to arbitrary target events and models. We have shown empirically in the existence of an optimal AST – in the sense of accuracy of long return period estimates – that is roughly approximated by the time until 38 of error saturation. But this result might be specific to the Lorenz-96 system and a number of choices made in , in particular relating to

The target variable defining intensity (energy density, xk2, with site index k=0, though for Lorenz-96 all sites are statistically equivalent).

Practitioners working with models more complicated than Lorenz-96 face a vast menu of choices in all four domains, the first two falling under the purview of domain science and the last two falling under algorithm design. If the physical model or the choice of target variable changes, it stands to reason that the choice of metric should also change, and any single prescription of AST likethe38-saturationtime is unlikely to work for all cases. Indeed, in our recent application of TEAMS to extremes of temperature and daily precipitation in a general circulation model, we found that the 38 rule provided some guidance but underestimated the optimal AST for both temperature and precipitation . Error norms incorporating global information will be less relevant than local norms around the target region, which tend to saturate more slowly .

Our primary goal in this study is to establish a general principle for optimizing AST for intermittent extreme events in meteorologically relevant dynamical systems. To balance computational economy with physical realism, we select a system of intermediate complexity between Lorenz-96 and a moist GCM: a 2-layer quasigeostrophic (QG) flow with a passive tracer. Since its original formulation in , the 2-layer QG model has served as a useful paradigmatic minimal model for baroclinic instability and associated jets, waves, and vortices in the atmosphere and ocean. It has been augmented in many ways to study specific processes, for example by , who coupled in a moisture component and found the resulting precipitation and latent heating to strongly affect the balance between waves and vortices in the underlying flow. However, even the simpler addition of a passive tracer – one without feedback through latent heating – is enough to advance the algorithmic questions we pursue here. Passive tracer dynamics is physically interesting in its own right, as seen by many studies of intermittency and heavy-tailed tracer statistics in turbulent flows . In climate science too, extremes of pollution concentration and temperature can be captured partially through passive tracer advection .

Our choice of the 2-layer QG model as a test system is thus a major upgrade in physical relevance as well as algorithmic difficulty from Lorenz-96, which resembles QG dynamics only loosely via its Hopf bifurcation structure . This path up the model hierarchy has been trodden before by , who added passive tracers to Lorenz-96 and a QG model respectively and studied extreme fluctuations in the tracer's Fourier modes. Also, quantified extreme value statistics – including local and global statistics – of QG wind fields themselves. All these works have inspired and guided this one, but we focus distinctly on the link between short-time perturbation dynamics and long-term climate statistics.

The QG model has enough “space” to explore the effects of all four decision axes listed above on optimal AST. In principle, one can do this with an exhaustive suite of experiments: for every target region (location, size) and every version of stochastic input (e.g., perturbation magnitude and spatial scale) of interest, run TEAMS with a wide range of AST parameters, measure the skill of each AST in matching a reference ground truth distribution, and select the optimal AST. In practice, this exhaustive procedure is not feasible, in part because of the huge number of potential targets, but more fundamentally because TEAMS' performance is highly subject to randomness. Measuring the effect of any parameter change on the algorithm's performance requires many repetitions – several dozen at least – to average out the variability inherent in Monte Carlo. Moreover, other hyperparameters related to “population management” exist within TEAMS and other rare event algorithms: the number of initial ensemble members, how many of them to kill and clone at every iteration, and the termination criterion, to name a few. Randomness appears not only as physical forcing, but also in selecting which members to clone, thus interacting tightly with the population hyperparameters. One can think of this as confounding due to sampling bias, which further blurs the imprint of AST itself on performance.

So instead of using TEAMS for our investigation, we turn to a related method of ensemble boosting . The idea of ensemble boosting is simple: identify some extremes from an initial climatic timeseries, and re-simulate them with perturbed antecedent conditions to generate unrealized but physically plausible (and possibly more extreme) scenarios. By focusing on a limited set of ancestor events to boost, we avoid the additional randomness that occurs in TEAMS as the level is raised and additional ensemble members are stochastically added, which simplifies our investigation. In addition, has developed an approach to estimate probabilities based on the boosted ensembles, and we have also been developing such an estimator that is introduced below. With the addition of an ability to estimate probabilities, ensemble boosting may now be viewed as an RES algorithm.

We suspect that the optimal AST is closely related to a physically intrinsic quantity that is not particular to a given algorithm. Analogously to Lyapunov exponents, which encode the timescale for small perturbations to double, the optimal AST should encode the timescale for extreme values of some target variable to maximize in variability. This statement is heuristic, and a primary goal here is to propose some quantities that are very close to the optimal AST and that, like Lyapunov exponents, are intrinsic to the system and do not depend on arbitrary algorithmic choices. We propose and evaluate several candidates, including entropy and expected improvement: two functionals of ensemble distributions which are drawn from reinforcement learning.

We have three major contributions. First, we develop a new estimator for low probabilities of extreme fluctuations from boosted ensembles, similar to the estimator of but distinct in the aggregation step. Our approach includes an optional parametric fit of the response function to perturbations (applicable to both estimators), a simple quadratic regression model that imposes regularity on the resulting severity distribution. Second, we use the two estimators to measure the quality of a range of ASTs across a range of target events (tracer concentration at different target locations), finding evidence for an entropy-based optimality principle. Third, and most importantly from a practical perspective, we demonstrate that both estimators successfully approximate low probabilities when the ensembles are launched from a good AST, which the optimality principle can help to select efficiently. Our goal here is not to demonstrate a performant rare event algorithm – only to elucidate a necessary ingredient (AST) to be optimized in future algorithms – but even when comparing statistical errors at equal cost, we find (and report at the end of the analysis, in Fig. ) that our boosted ensembles are already competitive with an equal-cost DNS.

The rest of the paper is organized as follows. Section  details the procedure of generating samples and estimating tail statistics, at a model-agnostic level, and proposes several candidate indicators of measuring ensemble dispersion that may help select an optimal AST. Section  specifies the QG system, its numerical simulation, and its extreme value statistics. Section  specifies the perturbed-ensemble design at a model-specific level. Section  visualizes some examples of perturbed events, and how the AST selection criteria behave on these examples. Section  reports the performance of different AST choices, and visualizes the overall “optimization landscape”. Section  concludes with an outlook and proposed roadmap for subsequent research – theoretical, algorithmic, and applied.

There are many design choices in ensemble boosting : how to select extreme events to boost, how many boosts to generate, when to launch them, etc. This subsection details the choices used here.

We run a direct numerical simulation (“short DNS”) {x(t):0≤t≤Tshort}, long enough to generate some extremes but not enough to estimate probabilities smaller than 1/(ϵ2Tshort)=100/Tshort for a relative error tolerance of ϵ=0.1. The premise of RES, and ensemble boosting, is that the extremes it does generate might have been even worse, perhaps just a butterfly flap away from the more intense extremes one would see with a “long DNS” of duration Tlong≫Tshort. We generate such a long DNS as well to serve as a ground-truth for validation. Following the ensemble boosting methodology laid out in and , we first identify a threshold μ with exceedance probability q(μ) that is moderate enough to estimate precisely with the short DNS. In other words, μ is the [1-q(μ)]th quantile, or “q(μ)th complementary quantile”. Equivalently, q(μ) is the complementary cumulative density function (CCDF) of the random variable R, evaluated at μ. In line with the peaks-over-threshold procedure , we take cluster maxima of exceedances above μ as the “ancestral” extreme events. Concretely, a cluster maximum is a state from the DNS, x*=x(t*), such that 1R*=Rxt*=maxR(x(t)):t*-Amax≤t≤t*+B>μ. where Amax and B are buffer times longer than the mixing timescale of the dynamics (i.e., how long two perturbed simulations need to become independent), ensuring that two consecutive events (x(tn*),x(tn+1*)) are genuinely independent from each other. Amax is an upper bound on the ASTs used for boosting.

We collect all such peaks occurring in the short DNS, 2xn*=xtn*:n=1,…,Nshort, and for a sequence of increasing ASTs {Aj:j=1, …, J} bounded between 0 and Amax, launch an ensemble of descendants {xn,j,m*:m=1, …, Mn,j} by applying Mn,j different perturbations to the DNS at time tn*-Aj, and running each simulation to time tn*+B. Note that Mn,j could in principle vary between ancestors n and lead times j, which is not needed for our exhaustive sweeps in this paper, but certainly would be needed in an “online” rare event sampling procedure that iteratively homes in on a subset of the most extreme-ogenic ancestors {n} and ASTs {j} to draw more samples from.

A bit more notation helps clarify how the perturbing is done, abstractly at first and concretely in Sect.  when we specialize to the QG system. For each (n, j, m), we draw a random sample ωn,j,m from some sample space Ω. Denoting by ΦΔt:Rd×Ω→Rd the flow map that integrates the perturbed dynamics forward by a time interval Δt, the (n, j, m)th descendant's trajectory through state space Rd can be written 3xn,j,m(t)=x(t)fortn*-Amax≤t≤tn*-AjΦt-tn*-Ajxtn*-Aj,ωn,j,mfortn*-Aj<t≤tn*+B. In words, the descendant shares its ancestor's past up until the time of perturbation tn*-Aj, after which it diverges.

There are two main forms of commonly used perturbation. An impulsive perturbation is a kick applied at a single time (which is used in ensemble boosting), in which case Ω=Rk or Ck, typically with k≪d, and a sample ω is transformed to spate space via a function G:Rk→Rd (e.g., a low-rank matrix multiplication). Then, the perturbed dynamics can be written ΦΔt(x,ω)=ΦΔt(x+G(ω)), where ΦΔt with only one argument is the unperturbed dynamics. We also use the convention that G(0)=0, i.e., ω=0 corresponds to no perturbation.

The other common case is where x(t) is a stochastic process, e.g., an Itô diffusion forced by white noise, as we used in as well as the schematic in Fig. . In that case, ω is a white noise process sampled at discrete times, whose dimensionality scales with the number of timesteps. In the QG experiments, we adhere to impulsive perturbations for three reasons: it introduces fewer arbitrary parameters, it is less disruptive to the system's intrinsic dynamics, and it keeps the dimensionality of the random space low. If, as we conjecture, even low-dimensional butterfly flaps are sufficient to excite the more extreme fluctuations, it would make deterministic search methods – which should always be preferred over Monte Carlo – more viable.

Following the perturbation, the descendant drifts away from the parent and achieves its own severity R* (peak of its intensity function R) at some time tn,j,m* possibly different from its ancestor's peak time tn*: 4Rn,j,m*=Rxn,j,mtn,j,m*=Rn,j*ωn,j,m where the latter notation emphasizes dependence on ω, while recognizing that each (n, j) induces a different severity function R* because perturbations may be felt differently depending on the initial condition.

If the perturbation is small, the descendant's peak time tn,j,m* will be close to the ancestor's peak time tn*. However, if the intensity function R(x(t)) tends to oscillate, e.g., with each passing Rossby wave crest, a large-enough perturbation might cause the next wave crest after tn* to outgrow the original peak, misappropriating the imposed perturbation to fuel a different event than the original target. Tersely, t*=argmaxtR(x(t)) might be a discontinuous function of ω, and R*(ω) a non-differentiable function of ω, which is a nuisance for our goal to optimize over ω and, more importantly, complicates the causal chain between perturbation and response. We explicitly prohibit this behavior by restricting the range of tn,j,m* as follows.

Set an “argmax drift” parameter δt* based on physical timescales, e.g., half an oscillation period. Initially set tn,m,j*=argmax{R(xn,j,m(t)):tn*-δt*≤t≤tn*+δt*}.

Assume now there is a probability measure PΩ on Ω with associated density function pΩ(ω), which might for example place higher weight on smaller kicks. The Ω superscript will generally relate to statistics over this conditional probability measure, to distinguish it from long-term climatological statistics. A major aim of this paper is to show how they relate to each other. Each ensemble of descendants at each lead time gives rise to its own conditional severity distribution: 5Qn,jΩ(r)=PΩRn,j*>r=∫ΩIRn,j*(ω)>rpΩ(ω)dω, which can be estimated from the samples {Rn,j,m*:m=1, …, Mn,j}. Here conditional means starting with a perturbation of the nth ancestor's particular initial condition at time tn*-Aj and running forward until time tn*+B. By contrast, we refer to the climatological severity distribution as that resulting from a long DNS.

Integrals of the form (Eq. ) arise in many diverse risk analysis tasks, such as reliability engineering, where Ω often represents wind, waves, or tremors buffeting a built structure , and is therefore high-dimensional. The default strategy for high-dimensional sampling is vanilla Monte Carlo, whose infamously slow convergence has motivated more efficient workarounds. A particular class of “variational” and “first- and second-order reliability” methods approximate the sampling by constrained optimization, relying on the large-deviation principle that increasingly rare events have a shrinking space of possible pathways, concentrating around a single point of Ω. We could certainly make use of those methods here, but there is a crucial distinction: in our setting, the perturbation space is an arbitrary design choice aiming at an indirect goal (climate estimation), rather than some externally imposed distribution (e.g., a Gaussian process model for ocean bathymetry in and ). Therefore, nothing stops us here from deliberately choosing low-dimensional perturbations instead of high-dimensional ones as in and . This enables numerical quadrature instead of Monte Carlo or elaborate large-deviation approaches, and saves on cost by allowing sample re-use across different input distributions.

It is possible that higher-dimensional spaces are more effective for exciting extreme fluctuations, which would make the above-cited methodologies very useful for our purpose in future research. They can also be useful when conditional risk estimation (for near-term weather forecasting) is the end goal, as well as the previously-mentioned optimization methods demonstrated in , , and . But our first goal is to determine whether our chosen low-dimensional kicks can suffice for climatological estimation.

Based on the samples drawn from Ω, we fit a regression model R^n,j*(ω;θ) with parameters θ, in our case coefficients for linear and quadratic polynomials. In general R^n,j could be a more elaborate parametric model, e.g., a Gaussian process or neural network with learned weights θ, as often used in modern uncertainty quantification . Then the integral over Ω can be estimated, either analytically (if p and R^* take simple enough forms) or numerically by densely filling Ω with a grid of points, evaluating R^* and p at each point, and taking the inner product of I{R^*>r} with p for any r. The result is an estimate Q^n,jΩ(r) for the conditional CCDF, Q^n,jΩ(r), obtained by replacing the Rn,j* with R^n,j* in Eq. (). The final step is to estimate the tail of the conditional CCDF, 6Qn,jΩ(r;μ)=PΩRn,j*>r|Rn,j*>μ=Qn,jΩ(r)Qn,jΩ(μ), which we could do just by putting hats (⋅)^ on the Qs on the right-hand side. However, this risks dividing by zero, because the fitted function Q^n,j may imply zero probability of exceeding the threshold, particularly at long ASTs when descendants have enough time to decorrelate totally with their ancestor. This loss of ancestral “wisdom” is a more fundamental problem than the numerical issue of zero denominator, and we address it by implementing a continuous version of the “accept-reject” step of the TEAMS procedure in . Wherever the descendant severity R^n,j*(ω) falls below μ, we replace it with the ancestor severity, denoted Rn* (with no second subscript): 7Q^n,jΩ(r;μ):=∫ΩIR^n,j*(ω)>rifR^n,j*(ω)>μI{Rn*>r}otherwisepΩ(ω)dω8=∫R^n,j*(ω)>μIR^n,j*(ω)>rpΩ(ω)dω+∫R^n,j*(ω)≤μIRn*>rpΩ(ω)dω9=∫ΩIR^n,j*(ω)>rpΩ(ω)dω+IRn*>r∫R^n,j*(ω)≤μpΩ(ω)dω10=Q^n,jΩ(r)+IRn*>r1-Q^n,jΩ(μ) (Q^n,jΩ(r)=0 when Q^n,jΩ(μ)=0 since Qn,jΩ is decreasing, hence the two terms in the last expression correspond to the two cases).

This estimator can be extended to other expectations of interest conditional on the target variable being extreme. Denote by Φ[Xn,j]=:Φn,j a generic function of the trajectory Xn,j launched at AST Aj ahead of ancestor n, such as time-averaged wind speed or air temperature. It is actually a random variable (a function of ω, Φn,j(ω)) and its mean can be estimated by replacing each I{R*(ω)>r} with Φ(ω) to obtain 11EΦn,j|Rn,j*>μ≈Φ^n,j=∫ΩΦn,j(ω)IR^n,j*(ω)>μpΩ(ω)dω+Φn,j(0)1-Q^n,jΩ(μ). The first term collects statistics of the part of the W-disc that contributes to the tail {R*>μ}, and the second term just moves the remaining (“rejected”) probability mass back onto the ancestor at ω=0. We do not explore the properties of this estimator for different Φs, but note it could be important to an applied study with RES.

Another heuristic way to justify the accept-reject expression for Q^n,j(r;μ) in Eq. () is to stipulate that we care about approximating only the extreme part of the boosting distribution, i.e., those ωs near enough to 0 that R*(ω)>μ, excluding the descendants bound to fall below μ. We re-allocate the probability mass in the “non-extreme” region of the disc (where R*(ω)≤μ) to the very center of the disc (the ancestor, where R*>μ by construction). This rearrangement ensures that Q^Ω(μ) is close to 1, justifying a Taylor series expansion in 1-Q^Ω(μ) 12Qn,jΩ(r;μ)=Qn,jΩ(r)Qn,jΩ(μ)13=Qn,jΩ(r)1-1-Qn,jΩ(μ)14≈Qn,jΩ(r)+1-Qn,jΩ(μ)Qn,jΩ(r)15≈Q^n,jΩ(r)+1-Q^n,jΩ(μ)IRn*>r16=:Q^n,jΩ(r;μ) The crux of our hypothesis is that these conditional distributions from boosting can be aggregated across ancestors to approximate the climatological distribution QΘ(r,μ)=P(R*>r|R*>μ), where Θ is used to denote the ground truth that would be obtained from a long DNS. We specifically propose to aggregate the conditional CCDFs as a uniform mixture over ancestors, selecting one representative AST Ajn from each ancestor n to best represent its alternate realities according to some selection rule (different rules will be evaluated thoroughly for the QG system in Sect. ). We write the mixture as 17Q^M(r;μ)=1Nshort∑n=1NshortQ^n,jnΩ(r;μ), and call it the “MoCTail” estimator of QΘ(r,μ), for “Mixture of Conditional Tails”.

The recent works and formulate a different estimator, which makes for an interesting comparison. Rather than summing Nshort tail CCDFs, each approximating a ratio of the form (Eq. ), they construct a single ratio by summing Nshort numerators and Nshort denominators. Translated into our own notation, this becomes 18Q^P(r;μ)=∑n=1NshortQ^n,jnΩ(r)∑n=1NshortQ^n,jnΩ(μ). We call this the “PoPTail” estimator of QΘ(r,μ), for “Pool of Perturbed Tails”. do not model R*(ω) parametrically, but instead use a standard Monte Carlo estimate Q^n,jΩ(r)= (fraction of descendants exceeding r), which is probably necessary for their high-dimensional perturbations. However, we can convert the PoPTail estimator to our parametric version just by thinking in terms of CCDFs, hence the formulation in Eq. (). The more important difference is that PoPTail avoids the potential degeneracy Q^Ω(μ)=0 by “pooling” non-extreme descendants together with extreme ones in the denominator.

One could argue for either estimator based on the validity of its underlying assumptions which are challenging to rigorously verify. Here we adopt a more openly empirical perspective in testing the skill of both.

An important advantage of both estimators is extensibility with respect to the dataset: if the variance is too high, one can always either generate new ancestors by extending the short DNS, or extend the range of ASTs sampled, or enlarge the ensemble at any ASTs deemed promising, without discarding the laborious samples already generated. This is unfortunately not the case with an algorithm like AMS, TEAMS, GKTL, or QDMC: because of the random rules by which ancestors are selected and new members generated, a completed run cannot be enlarged while retaining its estimation properties unless we are willing to do an entirely new additional run and combine estimates from multiple runs as was done in , and . This results in waste during the fine-tuning process of calibrating TEAMS. For example, one might decide in retrospect that a TEAMS run was too aggressive in killing non-extreme simulations and raising the threshold and we cannot easily extend the run with a new set of hyperparameters. With boosting, we can simply go back, perturb those less-extreme simulations, and incorporate them into the dataset, without needing to re-generate everything. To make boosting competitive at sampling the highest levels of severity, we suspect it will be necessary to augment our current scheme with an iterative level-raising schedule, like TEAMS, but with less restriction on the sampling procedure.

We evaluate the MoCTail and PoPTail estimators Q^M and Q^P by comparing to the ground truth QΘ as estimated from a long DNS. DNS is in fact a trivial special case of ensemble boosting with M=0 (no descendants), reducing each summand of Eq. () and the numerator of Eq. () to I{Rn*>r} and the denominator of Eq. () to Nshort. Both estimators reduce to the same vanilla empirical CCDF in this case, and this is what we use to estimate QΘ.

We use χ2-divergence to measure the disparity of Q^M and Q^P from QΘ. This is estimated from a discrete histogram with a sequence of thresholds μ=r0<r1<r2<…<rK-1<rK=∞, and define the probability mass function ΔQkΘ=QkΘ-Qk+1Θ as the probability contained in the kth bin (note that QKΘ=0 and so ΔQK-1Θ=QK-1). As described further in Sect. , we select the rks as quantiles with consecutively halving exceedance probabilities, i.e., QkΘ=125+k for 0≤k<K=11. These quantiles change with latitude, as the tail is different for each. Note the same set of rk's based on the climatological distribution is used also for evaluating estimated distributions. The χ2-divergence of either estimator Q^∈{Q^M,Q^P} is then defined as 19χ2(ΔQ^‖ΔQΘ)=∑k=0K-1ΔQkΘ-ΔQ^k2ΔQkΘ.

We will compute both the MoCTail and PoPTail estimates on the same dataset, and find them numerically quite similar, both in terms of skill and in terms of individual bin estimates. It would be interesting to develop test cases where they differ more systematically, to clarify which (if either) is generally superior.

Computational efficiency is another important consideration besides accuracy, as the entire goal of rare event algorithms is to improve efficiency or accuracy (or both) relative to DNS. For a boosting-like rare event algorithm to be useful, its error should decrease faster by perturbing existing ancestors (increasing M) than by extending DNS by generating new ancestors (increasing N and not M), at least in some range of N that samples the attractor broadly but not exhaustively. However, this paper does not present a complete rare event algorithm per se, in the sense we do not yet stake our claim on a speedup. Rather, we ask a pre-requisite question: does increasing M decrease the error at all? Clearly boosting can increase the maximum severity, but that could happen in ways that do not respect the tail CCDF's shape, e.g., if perturbations tend to maximize the event's severity while bypassing moderate severities that carry significant statistical weight.

We will thus make two comparisons between boosting and DNS: accuracy at fixed N, and accuracy at fixed cost (where DNS runs an additional length equal to the cost of simulating descendants, allocating its full budget to “exploration” rather than “exploitation”). Specifically, we approximate the cost of the boosting approach for a given AST A as 20Averageboostingcostperancestor=MA+δt*+(meanreturnperiod), where δt*, the “argmax drift” parameter, accounts for the extra time needed to run after the ancestor's peak to account for changes in peak timing. “Mean return period” is the average time between consecutive independent peaks over the threshold μ, which will be longer than 1/(1-q(μ)) because of de-clustering. The dependence on A is a complication, as each AST tried would merit a different-length DNS for cost comparison, and we do not want to penalize boosting too severely by summing over all ASTs because in practice we would not bother simulating the obviously sub-optimal ASTs. Rather, we optimistically estimate the cost if A is already known. On the other hand, our chosen M(=21) is likely more samples than necessary to fit a satisfactory parametric model, as we have deliberately sampled the perturbation space more generously than we would if chasing a speedup. We simplify the comparison by fixing A to 12Amax in Eq. (), which is close to or slightly greater than the optimal values that we found empirically.

We will show (Fig. b) that boosting is unambiguously more accurate than DNS when fixing the number of ancestors N, and similarly accurate with marginal improvements when fixing cost, though with variation across latitudes and AST criteria. Thus, we do achieve some speedup, even though it is not (yet) our main objective. Any fixed-cost performance gains we achieve here (not our main objective) should be viewed as a lower bound for future algorithms, which will benefit from the conceptual insights into AST that we glean presently.

To emphasize the conditional nature of the AST – its possible dependence on the ancestor n due to initial condition-dependent predictability – we refer to Ajn as the “conditional advance split time” (CAST), and its optimal value (by χ2 or other criteria) as the “conditionally optimal advance split time” (COAST). Our goal is to define the COAST, calculate it given extensive sampling from boosted ensembles, and finally to suggest useful criteria to estimate it when sample size is limited, as in a real rare event algorithm deployment.

With a data-generating plan and an estimator in place, we return to our central question of interest: how to select the CASTs {Ajn}? There are three natural kinds of criteria.

Choose a single uniform AST Ajn=AU for all ancestors (“U” for “uniform”). In this case, the CAST is not really “conditional” at all. In , we found the COAST for TEAMS by systematic grid search through candidate ASTs, and found post-hoc an empirical relationship for the COAST: AU≈t3/8‾, where tϵ(x0) is the time until an ensemble dispersing from initial condition x0 (each member forced by a different noise realization) reaches a fraction ϵ of its asymptotic root-mean-squared-error (RMSE), and tϵ‾ is the average of tϵ(x0) over different initial conditions x0. In , we sampled x0 from the stationary distribution; here, for computational expediency, we will repurpose the boosting ensembles for estimating tϵ‾, i.e., sampling x0 from pre-peak antecedent conditions.

We implement a version of the QG model combining elements of several classic studies. Our numerical method and friction form follow , but on a smaller domain with weaker friction magnitude as in to contain only 1–2 more energetic zonal jets. We furthermore add bottom topography in the lower layer as in to fix preferred latitudes for jets while still allowing them to temporarily split, merge, and meander. Thus climate statistics, and hence the COAST itself, can vary with latitude. Further, we augment the system with a passive tracer to represent a key component of precipitation dynamics, following the spirit of and who used turbulent advection-diffusion as a paradigm for intermittency.

The model equations are as follows, in non-dimensionalized form using the deformation radius λ as the length scale and a velocity scale U. To make plain the role of the background shear, we define a non-dimensional wind U as the ratio between the imposed upper-level zonal wind and U. All non-dimensional parameter values are listed in Table . The horizontal coordinates (x, y) each run from 0 to L. The integer-valued vertical coordinate z is an index for the layer (1 for the top and 2 for the bottom, appearing as a subscript). ψ represents the streamfunction minus a background of -Uyδz,1. h is the bottom topography which is specified to vary sinusoidally with wavenumber 2 in latitude. q represents potential vorticity minus a background of βy+hδz,2, due to planetary vorticity gradient and topography. c represents the passive tracer field. 29∂t+∂xψz∂y+Uδz,1-∂yψz∂xqz+hδz,2+βy=-κδz,2∇2ψz-ν∇6ψz30∂t+∂xψz∂y+Uδz,1-∂yψz∂xcz=031for(x,y,z)∈[0,L)2×{1,2}32where33qz=∇2ψz+(-1)zψ1-ψ2234h(y)=h0sin⁡2⋅2πyL For ψ, we impose doubly periodic boundary conditions and timestep with a pseudo-spectral method with 64 Fourier modes in each dimension and standard 23-dealiasing (hence, an effective maximum wavenumber of 20). We time-step linear terms with the trapezoid rule (Crank–Nicolson) and nonlinear and topographic terms with a predictor-corrector (Heun's) method. Meanwhile, boundary conditions on c are periodic in x and Dirichlet in y, with values (0, 1) at y=(0,L). Together with a first-order upwind monotone finite-volume scheme, this setup guarantees that 0≤c≤1 everywhere, making clear that its probability distribution has compact support. Note there is no explicit dissipation for c, but the low-order discretization creates some effective diffusivity.

The number of degrees of freedom, or state space dimension, is 35d=(2layers)×412Fouriermodesforψ+642gridcellsforc=11554, and we will sometimes refer to the full state vector as {ψ,c}(x,y,z,t)=x(t)∈Rd – not to be confused with the spatial coordinate x. For simplicity, we refer to one time unit as a day, which is ∼110 of an eddy turnover timescale (see Fig. ). The common timestep for ψ and c is 0.025 d, and the output frequency is once per day. The spatiotemporal resolution is coarse by modern standards, but we are not seeking to calculate any real-world physical quantity: we are seeking a general rule that will help make the COAST clear for a wide class of models.

We run a “short DNS” of length Tshort=4×103 d ≈11 years (after a 500 d spinup) to supply the pool of initially un-perturbed (“ancestral”) events. Then, to provide “ground truth” statistics, we run a control simulation, or “long DNS”, of duration Tlong=16×103d≈44 years, which is O(1600) eddy turnover times and O(160) jet meandering times (see Fig.  caption for timescale definitions). However, in estimating climatological statistics from the long DNS, we take advantage of statistical zonal symmetry by concatenating the timeseries of all 64 longitudes, increasing the effective sample size by a factor of ∼L/(some typical correlation length). Conceptually, the short and long DNS are analogous to “training” and “validation” datasets in standard machine learning procedures, in the sense that we want to infer properties of the validation set using only information extracted from the training set (for example, by perturbing and re-simulating events seen in training). As we show below, simply counting events from the short DNS gives probability estimates that deterioriate at high levels of severity, which we aim to rectify with boosting.

Figure  shows representative snapshots of three dynamical fields in the upper layer from the long DNS: tracer concentration c, zonal velocity u=U-∂yψ, and meridional velocity v=∂xψ. Figure  shows Hovmöller diagrams of zonal-mean anomalies of c and u (not v, since zonal-mean meridional velocity is zero), as well as their climatological means and standard deviations plotted alongside the topography. These are statistics of the grid-cell values, not zonal means, but depend only on latitude because so does topography. Two eastward jets are prominent in the snapshots Fig. b and in the zonal mean profile Fig. b.iii, with preferred latitudes of ∼14L and ∼34L. The Hovmöller diagram gives a sense of characteristic timescales: jets tend to remain roughly stationary for stretches of ∼100 d at a time before shifting, as seen by the group of closed contours of ψ and associated dipole of u centered at time t=3400. and persisting ±50 d to either side. Within these stretches of quasi-stationarity, there are shorter undulations of duration ∼10, which we identify as the eddy turnover timescale.

The tracer statistics (Fig. a.iii and iv) have some easily explainable large-scale patterns and some subtler small-scale patterns. The tracer time-mean 〈c〉(y) increases linearly overall as yL, in keeping with its Dirichlet boundary conditions. However, in the central region of the domain (inside the weak westward jet) the tracer mean varies more rapidly with latitude and has a larger standard deviation (see also dashed curves in Fig. b and c).

In the eastward jets, the tracer mean varies more slowly with latitude and has a smaller standard deviation. Comparison with the Hovmöller diagram (Fig. a.i) suggests that the central region owes its high variance to short-lived anomalous pulses, both positive and negative, which are more intense than in surrounding regions. We won't try to explain these patterns from first principles, but simply state that the setup accomplishes our intention to provide a variety of statistical behaviors as a suite of test cases for our approach.

We define the intensity function of interest R(x) as the upper-level concentration, c1 (henceforth, simply c), averaged over a small square box [x0-ℓ,x0+ℓ]×[y0-ℓ,y0+ℓ] of half-width ℓ=264. This function is designed to capture the real-world considerations and algorithmic difficulties that originally motivated the AST: it describes localized conditions, similar to concentrated pollution, high heat, or heavy rainfall over a region on Earth, and it is mediated by traveling baroclinic waves, and as a result it displays intermittency, with extreme spikes that come and go quickly. The choice of upper- instead of lower-level concentration is simply to weaken the impact of arbitrary aspects of the model setup like the surface topography. Real-world applications would of course refine this choice in many ways, but our choice is suitable for the QG level of model idealization.

To investigate the effects of location-dependent flow regimes, we vary y0 across 23 evenly spaced latitudes y0∈1064,1264,…,5464L, restricted to the central region to avoid boundary effects. The central longitude x0 is fixed to L/2, but by zonal homogeneity any longitude would be statistically equivalent. We also repeated the analysis with double the box length, and found results to be qualitatively similar, so we only show results for the smaller box size. The effect of spatial scale is worth considering in its own right with a wider range, which we postpone to future work.

Figure  displays some summary statistics of R(x(t)) as functions of the target latitude y0: alongside (a) the topography for reference, we show (b) the meridionally de-trended time-mean 〈R〉(y0)-y0L and (c) the standard deviation 〈R2〉(y0)-〈R〉2(y0). Note the restricted latitude range. In Fig. a and b, dashed lines show the corresponding mean and standard deviation of c itself, as in Fig. c and d, of which R is a regional average: note that R has the same mean as c but a smaller standard deviation, and larger box sizes would reduce it even further.

While the low-order moments capture ordinary behavior of intensities R, the intensity peaks – i.e., severities R*, defined in Sect.  – are better viewed from an extreme value theory perspective, and summarized with the peaks-over-threshold procedure . We set a threshold μ as the 125th complementary quantile of R, also denoted μ125, i.e., the level whose exceedance probability is q(μ)=125. Severities R* are extracted as cluster maxima above μ, with buffer times Amax=40 d and B=20 d. All cluster maxima from the long DNS are used as input data points to infer the parameters (scale σ, shape ξ) of a generalized Pareto distribution (GPD), using the maximum-likelihood routine of the Extremes.jl package : 36PR*>r|R*>μ≈Gμ(r;σ,ξ)=1+ξr-μσ+-1/ξξ≠0exp⁡-r-μσ+ξ=0 where (⋅)+=max(⋅,0). Figure d–f displays the threshold (detrended by y0L), scale parameter σ, and shape parameter ξ. Several characteristics are noteworthy.

The detrended threshold μ-y0L has a maximum-over-minimum profile similar to the the detrended mean intensity 〈R〉-y0L, but shifted southward. The maximum of μ-y0L is close to the mid-channel maximum in the standard deviation of R, perhaps because extremes depend more on variability than on average behavior.

We implemented the boosting and estimation procedures for every latitude separately, but for illustration focus the in-depth analysis on y0=2664L (the small boxes in Fig. ), an interesting location where the (detrended) mean and threshold μ125 are both low, the GPD scale σ is large, and the GPD shape slightly more negative than in surrounding regions. Figure  displays the underlying probability distributions at y0=2664L to show the nature of the tails of the distributions and also to help clarify the relationship between intensities, severities, and GPD parameters. The full PDF of intensity, in Fig. a, has a positive skew and sub-Gaussian tail. Black and red solid curves are estimates obtained from the long and short DNS, respectively, and 90% confidence intervals are obtained by longitudinal translation. Specifically, the shaded intervals are the 5th–95th percentile ranges of intensities at the same y0, but with x0 shifted from its base location of 12L by 064, 164L, 264L, …, 6364L. The dashed black curve is the mean of all 64 curves, our best available estimate of ground truth. The discrepancy between short and long DNS is most pronounced in the upper tail, which in Fig. b is magnified and integrated from the top, giving the CCDF. Gray lines mark the threshold, μ=0.52, and its CCDF value 132≈0.03. Above this level, the short DNS becomes rapidly more uncertain (error bar widens), and severely underestimates probabilities smaller than ∼0.005.

Both short and long DNS estimates diverge markedly from the GPD fit shown in gray in Fig. b. This is where the distinction between intensity and severity comes into play: the GPD is fitted to peaks over the threshold μ – i.e., severities – whose distribution differs (most notably in the upward direction) from that of all exceedances over μ, which would include the clusters surrounding the peaks. Figure c confirms that the GPD fit is much more appropriate for severities R* than for intensities R, and thereby clarifies the distinction. If the threshold were raised, the clusters would shrink, the sequence of peaks would form a Poisson process, and the CCDFs of R and R* would converge. For computational economy and because non-asymptotic extremes are of interest for climate risk, we keep the threshold at μ125 and formally define our goal with boosting as correcting the distribution of severities – not intensities. Hence, our measure of success will be whether the short-DNS severity CCDF in Fig. c, when augmented by boosting, will become closer to the long-DNS severity CCDF.

We perturb the QG model with impulsive forcing, which we now specify as a concrete version of the generic form in Sect. . The stochastic input ω lives in the complex plane C(=Ω, the “input space”), and the state-space perturbation G(ω) consists of a single Fourier mode to be added to ψ. We stress that our focus here is on optimizing AST, not the perturbation space Ω, so the choice of mode is arbitrary so long as Ω remains low-dimensional. The optimal AST would probably change if Ω changes, e.g., if we perturbed a different mode or multiple modes at once; but the rule for choosing it based on entropy may well generalize, which will have to be tested in follow-up research.

Bearing these caveats in mind, we select a mode that is likely to grow fast, according to linear stability analysis, which is more easily explained as a procedure than as a closed formula:

Decompose ψ into a Fourier basis ψz(x,y)=∑k,ℓψ^z(k,ℓ)ei(kx+ℓy), and write the linearized dynamics (about the baroclinically unstable background state with vertical zonal wind shear and ψ=0, and neglecting topography) into the abstract form37C(k,ℓ)ddtψ^1(k,ℓ)ψ^2(k,ℓ)=D(k,ℓ)ψ^1(k,ℓ)ψ^2(k,ℓ)where C∈C2×2 represents the conversion from streamfunction to potential vorticity, and D∈C2×2 represents the advection and linear dissipation terms (excluding topography).

The steps above completely specify G(ω), a linear map from C to functions of (x, y, z), which can be easily computed offline before running any ensembles. One could argue for two obvious refinements of this choice: (1) specializing the linearization to the actual initial state, not just the background state, by linearizing the quadratic form J(q,ψ) and including that in the calculation of D(k,ℓ); and (2) accounting for finite time horizons by using the leading singular vector of the linear propagator, i.e., the initial infinitesimal error whose magnitude amplifies the most over a given time horizon and which could be estimated by a bred vector approach .

For this study, we stick to the simpler choice of the most unstable modes of the background shear, choosing to focus attention on the less-studied optimization of the advance split time given a fixed perturbation shape. There are several reasons that singular vectors may not be suitable for our goals. First, it is easier to compare different initial conditions, different advance split times, and even different topographies (which we do not do here) when they are all subject to precisely the same perturbation. Second, as our results will demonstrate, the COAST tends to lie beyond the time range where linearized error dynamics are appropriate, which is natural because we aim for finite-amplitude boosts in extreme events. Third, singular vectors are typically designed to optimize global errors, which might not be as relevant for local extremes. Fourth, such highly specialized perturbation shapes might not be accessible in a generic GCM. Nonetheless, sensitivity analysis with respect to perturbation shape leads the agenda for follow-up work.

Having fixed a subspace Ω=C for perturbations ω, we need to specify an input distribution pΩ(ω) over that space. We design the PDF for ω as a radially symmetric, smooth, “bump function” which has compact support in order to prevent perturbations so large as to induce oscillatory transients. The PDF is parameterized by two scales: W which is the maximum permissible magnitude of ω, and s which sets the typical perturbation strength: 39p(ω;s,W)∝exp⁡-|ω|22s21-|ω|2W2-1for|ω|<W,and0for|ω|≥W. When s≪W, p is approximately a bivariate Gaussian density with diagonal covariance s2I. When s≳W, p is approximately uniform over the W-disc {ω:|ω|≤W}, with rapid (but mathematically smooth) transition to 0 on the boundary. We fix W=0.3, limiting the maximum possible perturbation amplitude to |δψ|≤2W=0.6 (see text after Eq. ), which is small compared to the characteristic streamfunction amplitude of |ψ|∼10. We include s as a parameter to vary because there is no established principle to set the magnitude of impulses for the purpose of rare event sampling. In contrast, numerical weather prediction has an established (if heuristic) practice of tuning noise amplitude to match ensemble spread with model error e.g.,. Optimizing for climatological accuracy is a different, murkier goal calling for less prejudice with regard to perturbation magnitude. We therefore vary s widely from 0.06 to 0.9 in increments of 0.06 for 15 total values. s is the impulsive-forcing analogue to the continuous-forcing amplitude that we called F4 in , which strongly influenced the perturbation growth rate and therefore the optimal advance split time.

Figure a and b depicts p(ω;s,W) in two ways: (a) two-dimensional level sets of the unnormalized density (Eq. ) logarithmically spaced from e-4 to e-0.01, each value of s occupying one of 15 sectors of the circle; and (b) one-dimensional transects across p(ω;s,W) fixing Re{ω}=0. To save the labor of drawing Monte Carlo samples from p(ω;s,W) separately and simulating the perturbed children for each value of s, we compute the MoCTail and PoPTail estimators using numerical quadrature over the W-disc using a single set of samples drawn by quasi-Monte Carlo (QMC), and displayed as black dots in Fig. a. QMC is a general strategy which places samples deterministically across the input space in a way that mimics properties of randomness, but with lower discrepancy (fewer clumps and patches), thereby aiming to reduce variance in estimated statistics . We specifically use the LatticeRuleSampler from the QuasiMonteCarlo.jl Julia library to distribute points {(Um,Vm)}m=1M quasi-uniformly on the unit square [0, 1]², and transform them to the W-disc with the formula 40ωm=WUmexp⁡2πiVm. Since Um is a “quasi-random sample” of the uniformly distributed random variable U∼U([0,1]), we have 41Pr1≤|ω|≤r2=Pr12≤W2U≤r22=Pr12W2≤U≤r22W2=r22-r12W2 which is the fraction of the W-disc between the radii r1 and r2. The phase 2πV is clearly U([0,2π]). If U and V were independent random variables, we would immediately conclude ω is uniformly distributed over the W-disc; in QMC they are not independent, but the conclusion still holds true . In all experiments to follow, M=21, corresponding to the 21 points plotted in Fig. a. While other sampling rules are possible, the LatticeRuleSampler enjoys a distinct advantage of being extensible: sampling 12 points at first and later deciding to add 9 more gives the same result as sampling 21 in one batch.

Following the procedure laid out in Sect. , we apply each perturbation {ωm}m=1M to a collection of ancestor events {x(tn*)}n=1N at a range of ASTs {tn*-Aj}j=1J. We set the number of ancestors, N to whichever is smaller: the total number of cluster maxima (see Sect. ) in the short DNS, or 32. Considering all latitudes, the minimum N was 14, the median was 22, and the maximum 32 was found at four latitudes including y0=2664L which we consider in more depth. In the equal-cost comparisons to be shown later, we restrict N to smaller values. The ASTs sampled are {Aj}j=1J=20={2,4,…,40}, with a two-day spacing chosen as roughly half the period of small fluctuations in R(x(t)) (see Fig. ).