We first give an intuitive description of the clustering method used before
we explain it in much more detail in Sect. 3.1. That section can be skipped
by readers who are more interested in the clustering results.
Many studies have provided evidence that the atmospheric circulation can be
efficiently described by a few persistent cluster states
.
Conventional clustering methods such as k-means partition phase space using
heuristic algorithms, for example using empirical orthogonal functions
(EOFs), into an a priori arbitrarily determined k set of cluster
centroids whose points within each cluster are close but where each centroid
is in some sense far apart from each other . Similarly,
self organizing maps SOMs; are typically based on
minimizing the geometric (Euclidean) distance between the observational data
and some specified set of recurrent patterns but without consideration of the
persistency of those states and mostly without considering the dynamics and
differences in dynamics within these states .
Furthermore, classical clustering methods do not consider differences in the
dynamics of the cluster states . Recently,
applied SOMs to reanalyzed daily zonal-mean zonal wind data
for the austral summer period employing four SOM patterns with the DJF (December–January–February) global
mean temperature taken as an indicator of the response to GHG (greenhouse gas) forcing. They
attribute the main response to ozone by correlating the third SOM DJF
zonal-mean zonal wind pattern with the November Antarctic ozone index. One of
the central problems in applying these methods to historical geophysical data
is related to the robustness, meaning that increasing the k away from
k=1 increases the total number of parameters – thereby increasing the risk
of overfitting. Concepts from information theory like the Akaike information
criterion (AIC) are usually deployed to find the optimal
number of clusters k that allows avoiding overfitting.
Our approach considers the
hemispheric response of not only the zonal annular mode but also systematic
changes in wave 3 and blocking. We also use reanalyzed data but consider all
possible combinations of the observed radiative forcings and relevant indices
of internal variability. Moreover, we make no a priori assumptions on
the number of states and employ an approach that considers persistency,
changes in the dynamics and that is able to ascertain causation between the
time series and the external forcings.
Specifically, we use the non-stationary clustering method FEM-BV-VARX (finite
element method of time series analysis with bounded variation and vector autoregressive factor
of model
parameters; ) to
systematically attribute circulation trends to observed external forcings. In
this study we assume that the large-scale circulation can be effectively
decomposed into a small set of distinct patterns or regime states. On each
day the atmosphere is only in one of these patterns, where it might stay for
some time before it switches to one of the other regime states. If we order
the regime states from 1 to n, we get a daily index denoting in which state the atmosphere is in on that particular day;
this index is referred to in the literature as the Viterbi path
. We then construct a statistical model which
simultaneously estimates the geographical structure of these patterns and the
evolution of the switching between the patterns. We do this by minimizing the
distance between the observed atmospheric circulation and the regime states
(see next subsection for a more detailed description of this procedure).
Furthermore, we also allow external factors, like CO2, ozone, or ENSO, to
influence the evolution of the regime states. This is again done by
minimizing the distance. This also enables us to evaluate different forcing
combinations. By using different forcing combinations and looking for the
statistical model with the minimum distance, we can systematically find the
forcings which are most likely responsible for the observed evolution of the
regime states.
Overview of the FEM-BV methodology
The FEM-BV-VARX approach is a general variational framework that is reduced to
the well-known methods of linear regression, autoregressive models, k means
and hidden Markov approaches when more restrictive assumptions are made on
the nature of the underlying data-generating process
. Furthermore, as VARX is
a tool for inferring the Granger causality (causation
between time series variables in terms of predictability and not
correlation), FEM-BV-VARX is a more general approach which allows going
beyond the standard stationarity assumption of the usual methods currently
used for inferring cause–response relationships, e.g., in ecology
, economics and climate science
. The non-stationary FEM-BV-VARX framework
contains the standard stationary VARX and the concept of standard
Granger causality as a particular special case and allows for a systematic
comparison of causality relations inferred with and without the stationarity
assumption for a given set of observational data. See for
more details explaining the relation of the FEM-BV framework to standard
stationary approaches of data-driven causality inference.
The approach we are following here is that we perform FEM-BV-VARX fits to all
possible combinations of the external forcings. Then we apply a standard
information theoretic criterion: the AIC
, which is a measure of
the relative goodness of fit of a model to data. That means that AIC can be
used to assess the goodness of fit relative to the number of fitted
parameters or external factors used, allowing us to find the model that is least
overfitting and best fitting the analyzed data . This
standard procedure can tell us which forcing combination is best able to
explain the secular circulation trends given by the FEM-BV-VARX model.
The presence of unresolved external covariates (which are not
statistically independent or identically distributed) may result in the
non-stationarity and non-homogeneity of the resulting data-driven statistical
models and may be manifested in the presence of secular trends and/or in
regime-transition behavior. By covariate we not only mean external forcings
but also unresolved physical processes and scales (e.g., due to EOF
truncation). This may then introduce problems when applying the standard
stationary approaches common to machine learning and statistics
. In the context of this paper, this issue plays a very
important role when analyzing atmospheric data since many of the
potentially relevant covariates might not be available explicitly in the set
of covariates that we have chosen for testing. Therefore, when deploying
statistical time series analysis methods, they should be capable of dealing
with non-stationarity and non-homogeneity issues that emerge in the models as
a result of these systematically missing (and potentially important) external
influences.
Combining the concepts and ideas from pure and applied mathematics (such as
the FEM from numeric of partial differential equations,
regularization in infinitely dimensional spaces from the theory of ill-posed
problems, stochastic calculus and theory of stochastic processes, information
criteria from information theory, and embedding theorems from the theory of
dynamical systems), Horenko and colleagues developed a family of
non-stationary, non-homogeneous and non-parametric time series analysis
methods. This family of time series analysis techniques, which is reviewed
concisely by , allows for systematic time-dependent model
identification when assumptions of temporal stationarity or spatial
homogeneity of some underlying statistics are not justifiable. The main idea
is based on regularized variational minimization of a scalar-valued
functional describing the error
gx(t),u(t),θt of some model for a given
observation x(t) subject to available external impacts/covariates u(t)
and characterized by the time-dependent set of model parameters θ(t)
L(Θ,Γ)=∑k=1K∑t=1NTγk(t)gx(t),u(t),Θk→minΓ,Θ,
subject to the constraints on Γ(t)=γ1(t),…,γK(t):
∑k=1Kγk(t)=1∀t=1,…,NT,γk(t)≥0∀t=1,…,NT,∀k=1,…,K,|γk(⋅)|BV=∑t=2NT|γk(t+τ)-γk(t)|≤NC,∀k=1,…,K,
where θ(t)=∑k=1Kγk(t)Θk, thereby allowing the
problem of statistical data analysis beyond the usual stationarity assumption
to be reformulated and algorithmically solved as a clustering problem with
K clusters, distance kernel gx(t),u(t),θt
and a regularization parameter NC, controlling the number of transition
between different clusters in time (special case NC≡0 and K≡1 corresponds to the homogeneous and stationary statistical problem).
Imposed regularization confines the bounded variation (BV) of the
regime-switching process γk(t) in time, thereby making the temporal
change of inferred model parameters θ(t) more or less persistent.
Changing the constraining variable NC, one can test the whole range of
possible statistical models, going from NC≡0 (stationary/well-posed)
to NC≡NT (completely non-stationary/very ill-posed problems). The
above variational problem is non-convex since the parameters Θk are
not known a priori and have to be inferred simultaneously with the
γk(t).
Many classical methods of data analysis and machine learning (e.g.,
multilinear statistical regression, k-means
clustering, Gaussian mixture
models (GMMs) and hidden Markov models (HMMs) )
can be derived as special cases of this FEM-BV methodology. For example, HMMs are
obtained additionally assuming that Γ is an output of a homogeneous
Markov chain and setting NC≡NT. For more detailed information
please see Sect. 2h “Relation to classical methods of unsupervised
learning” in . It is an important feature of the deployed
methodology, since it allows us to test different standards or more advanced
methods in the context of the same theoretical and algorithmic
FEM-BV framework.
In the FEM-BV methodology, finite element methods are employed in the
numerical representation of indicator functions γk(t) for the time
domain of applicability of different models from a common model class. Model
class is defined by the choice of the particular analytical form of the error
function gx(t),u(t),θt,j; the explicit
VARX-form deployed in this paper is given below. As shown above, these
indicators are regularized using a bounded variation criterion, hence the
acronym FEM-BV. The choice of the model class depends on the type of data
and a specific form of error function g considered. Here we have
implemented vector autoregressive models with external influences (VARX),
which are defined as xt=μ(t)+∑i=1MEMAi(t)xt-iτ+B(t)ϕu(t)+ϵt, parameters being
θ(t)=μ(t),A1(t),…,AMEM(t),B(t), and
the model error functional defined as gx(t),u(t),Θk=‖|xt-μ(k)+∑i=1MEMAi(k)xt-iτ+B(k)ϕu(t)‖|22 for more details please see.
The algebraic structure of the above problem allows us to deploy efficient
numerical algorithms based on the iterative application of linear and/or
quadratic programming problems (optimizing for fixed Θ) followed by
stationary/convex inference of Θ for a fixed Γ. This procedure
is repeated iteratively, until the change in L(Θ,Γ) is
less than some small a priori-determined threshold, resulting in monotonic
minimization of L. Here we have used the adaptive finite element method
from the numerics of partial differential equations (PDEs)
for numerical minimization of the
FEM-BV problems (Eqs. , ).
The number of different spatiotemporal K regimes/clusters – the model
parameters to be chosen within these regimes, such as memory depth and number
of EOFs, and the indicator functions γk(⋅) signaling activation
of the respective models – are all determined simultaneously in a global
optimization procedure. This yields a judicious compromise between low
residuals, reproducing the data of a training set on the one hand, and the
demand for the smallest-possible overall number of free parameters of the
complete model on the other. The optimization is based on a new
non-parametric modified AIC and may thus be
interpreted as a constructive implementation of “Occam's razor” for data
analysis problems. Thereby, the resulting FEM-BV-framework is essentially
free of parameters that should be defined and tuned by the user. The only
parameter that needs to be set externally is the overall number of
optimization repetitions with different randomly chosen initial values of
Γ or Θ for parameter optimization (in the following referred to
as the number of annealing steps). Increasing this number reduces
the probability of getting trapped in one of the local minima of L
(for NC>0), simultaneously linearly increasing the amount of computations.
Therefore, the number of annealing steps should be chosen carefully,
depending on the available computational resources and the size of the data
to be analyzed.
Application of FEM-BV-VARX to atmospheric reanalysis and observed
forcing data
We have chosen to examine a time series of 500 hPa geopotential height
anomalies (seasonal cycle subtracted) projected on the 20 leading EOFs and with u(t) taken as
32 combinations of forcings (to be described in more detail below). Deploying
the FEM-BV-VARX method, comprehensive sensitivity tests were carried out for
the cluster parameters involving memory depth MEM and the number of
annealing steps. Although the resulting optimal choice of forcing
combinations has converged after 16 annealing steps, the model affiliation
sequence already remained unchanged after 4 annealing steps. The results
presented in the manuscript are for 64 annealing steps. The optimal memory
depth τ was 2 days; however, the degree of memory tested ranged from 0 to
5 days. Daily forcing agents was spline interpolated with no lag apart from
OMD. Every possible combination of forcing agents was considered including
the observed OMD lagged by 0, 30, 60 and 90 days as well as a variant with a lag average of 365 days span (here we artificially introduce persistency into
the OMD time series). The optimal external radiative forcing agent was found
ultimately to correspond to the Cape Grim CO2 time series.
In order to choose the optimal model M we first used the AIC:
AIC(M)=-2log(L(M))+2M.
Application of the AIC is equivalent to assuming that the scalar-valued
squared model errors are χ2-distributed and that the vector-valued
FEM-BV-VARX model errors are Gaussian, i.e., dependent on the residuals having
a log-normal distribution. We tested this assumption using a non-parametric
information-theoretic algorithm from and found that for
all of the model errors ϵt the most optimal parametric family was
indeed the log-normal distribution.
Additionally, the log-normal distribution was fitted to the model errors, the
respective log-likelihoods computed and used to calculate the AIC for the
non-stationary models. The most informative non-stationary model that emerged
using the AIC criteria (with posterior probability almost equal to one in
each case) was the model with Cape Grim CO2 and memory of 3 days.
Comparable results were found using Akaike information criteria corrected
(AICc) and the Bayesian information criteria (BIC). The two latter criteria
also take the size of the statistics into account and are derived under very
different mathematical assumptions than the AIC. This further confirmed our
results, demonstrating that they are not induced by the implicit assumptions
necessary for the information criteria applicability.
In our study we have included only a selected number of several external
forcings. One might argue now that we have neglected some additional forcing
(e.g., sea ice extent or the strength of the Antarctic Circumpolar Current)
which might be responsible for the observed secular trend. However, if we
include something like sea ice extent into the set of forcings and
get a result showing that sea ice extent is more statistically
significant, then this would not contradict this study simply because the
variable γ (describing the switching process and describing the bias
coming from all the unresolved covariates in our study) would be different
from the one that we obtained in this study. From our study we can guarantee
that in a given set of explicit covariates we found the one covariate that is
most important (in the Granger-causality sense) and – taking into account
the presence of the unresolved covariates – this covariate is CO2
. All of the other eventually important
covariates are in the regime-switching process that we have also
identified but were not explicitly represented in our results.