To clearly address the issue posed in the introduction, this study employs the simple pycnocline prediction model developed by Zhang (2011a, b). This conceptual coupled model is based on Lorenz's three-variable chaotic model (Lorenz, 1963) that is coupled a slab ocean variable (Zhang et al., 2012) interacting with a pycnocline predictive model (Gnanadesikan, 1999). For the problem that this concerns, this conceptual coupled model shares the fundamental features with a coupled general circulation model (CGCM; see Zhang, 2011a; Han et al., 2013). The model development can be traced in Zhang (2011a, b) and Zhang et al. (2012) in detail. Here, we only comment on major points that are relevant to this study. The model includes five equations: x˙1=-a1x1+a1x2x˙2=-x1x3+(1+c1w)a2x1-x2x˙3=x1x2-bx3Omw˙=c2x2+c3η+c4wη-Odw+Sm+Sscos⁡(2πt/Spd)Γη˙=c5w+c6wη-Odη. The first three equations (Lorenz's three-variable chaotic model) represent the dynamics of “atmosphere”. The last two equations, respectively, represent the dynamics of the slab upper ocean and the pycnocline depth variation of deep ocean. There are five variables in the model. x1, x2 and x3 are the fast-varying variables of the atmosphere with the parameters a1, a2 and b set as 9.95, 28 and 8/3, which sustain the chaotic nature of the atmosphere. w and η are the low-frequency variables of the ocean. Equation (1) tells that the ocean in this system is driven by two kinds of forcings: the chaotic x2 from the Lorenz equations and the periodic cosine function term serving as the external forcing of the system. The coupling parameter c2, which interacts with the chaotic forcing x2, is set as 1. Od is the damping coefficient. In this simple model, the damping coefficient is set to be identical for the upper ocean and deep ocean as 1. Values of other parameters such as c1, c3, c4, Sm, Ss, Spd, Om, Γ, c5, c6 are set as 10-1, 10-2, 10-2, 10, 1, 10, 10, 102, 1, 10-3 (the justification can be found in the literature cited before). The upper ocean is slower than the atmosphere due to the great heat capacity of water. The deep ocean is slower than the upper ocean due to lack of mixing. The parameter Om (Γ) that represents the heat capacity of upper (deep) ocean, combined with the damping coefficient Od, defines the fluid characteristic timescale. For example, the ratio Od / Om of 10-1 (Od= 1, Om= 10) defines the characteristic timescale of w being 10 times of that of x2. It is important to mention that, with these parameter settings, the chaotic atmospheric forcing is stronger than the periodical forcing in the “ocean” equation. The ocean feeds back to the atmosphere in the low-frequency band; therefore, in this coupled model, the uncertainty caused by chaotic atmosphere spreads to whole range of resolved periods for both the atmosphere and the ocean. From Eq. (1), it can be seen that the parameter a2 has a direct influence on the variation of the state variable x2, and the parameter c2 has a direct influence on the variation of the state variable w. The estimation of these two parameters will be used later to interpret the relation between the accuracy of SE and success of PE. Although very simple, this low-order (limited-size) conceptual model mimics very fundamental natures of interactions of three typical timescales in the real world: synoptic (chaotic) atmosphere, seasonal–interannual upper tropical oceans and decadal/multidecadal deep ocean (Zhang, 2011b). The boundary condition is a predefined seasonally varying solar radiation: S(t)=Sm+Sscos⁡(2πt/Spd). The state variable w mimics the surface temperature of the ocean and the x2 mimics the surface wind of the atmosphere. Here, we may mimic some parameterization of CGCM using the relation of parameters and model variables (c2 and x2 analogous to the drag coefficient cd and wind for the stress on ocean, for instance).

The filtering method used in this study is the ensemble adjustment Kalman filter (EAKF; Anderson, 2001). The EAKF algorithm shares all theoretical derivation of ensemble Kalman filter (EnKF; e.g., Evensen, 1994; Houtekamer and Mitchell, 1998) that combines an observational probability distribution function (PDF) with model PDF but under an adjustment idea. After the first version (Anderson, 2001), the EAKF algorithm had improved its implementation as a sequential local least squares filter (Anderson, 2003). The EAKF is a member of ensemble square root filters (Tippett et al., 2003), taking the advantage of ensemble Kalman filter without perturbing the observation (Whitaker and Hamill, 2002). While the detailed and exhausted mathematical derivations can be referred to the aforementioned literature and others (e.g., Zhang and Anderson, 2003), here we mainly comment on the computational implementation with a two-step procedure (Anderson, 2003; Zhang et al., 2007) that is relevant to this study. The first step uses two Gaussian convolutions to derive the observational increment at the observational location as Δyio=1(σp)2y¯p+1(σo)2yo1(σp)2+1(σo)2︸adjusted ensemblemean+Δyip1+(σpσo)2︸adjusted ensemblespread-yip,(i=1∼N,Nis the ensemble size), where y represents the observable state variable and σ is its error standard deviation. A superscript p always denotes the prior quantity estimated by the model, and o denotes observational quantity. An over-bar denotes the ensemble mean.

The second step regresses the observational increment onto the related model states or parameters by the model ensemble-evaluated covariance as Δpiu=cov(Δp,Δy)SD(Δy)2Δyio,(i=1∼N,Nis the ensemble size). The linear regression in Eq. (3) is built with the help of the 20-member ensemble. Δpiu is the adjusted state (parameter) increment given the observational increment Δyio. cov(Δp, Δy) is the error covariance computed between the ensembles of the model variable at the model grid and at the observational location (for SE) or between the ensembles of the state variable and perturbed parameter being estimated (for PE). SD(Δy) is the standard deviation of the ensemble of state variables at the observational location. For example, when using x2 to estimate c2, on each estimating step, the ensemble of x2 and the ensemble of c2 are used to calculate the ratio of cov / SD2 and adjust c2 toward a better value that minimizes the errors of model states from the observations. While such a sequential implementation provides much computational convenience for data assimilation, the EAKF maintains the nonlinearity of background flows as much as possible (Zhang and Anderson, 2003; Zhang et al., 2007). It is worth mentioning that just as usual EnKFs or variational methods without a model error compensation term, the EAKF has the disadvantage of dealing with model errors.

Some other relevant aspects of the method are also commented here. Just as in Zhang and Anderson (2003), based on the trade-off between cost and assimilation quality, after a series of sensitivity tests on ensemble sizes of 10 and 20–100, no significant difference in the quality of standard assimilation is found when the ensemble size is greater than 20. Thus, a practical ensemble size of 20 is chosen as a basic experiment setting. We will examine the sensitivity of major conclusions of the addressed problem in this study to the ensemble size in related places later. Although the intervals of the atmosphere and ocean observations are different in the real world, for convenience of comparison, we set a uniform update interval for SE (in the atmosphere and ocean) and PE as five time steps as the basic setting in this study (we will also discuss the influence of update intervals in related places later). The inflation method must be included in the EAKF PE. Considering that the inflated parameter ensemble will influence state variables, no inflation is applied to the model state ensemble. The PE inflation scheme follows Zhang (2011b): when the SD (spread) of the parameter ensemble is below some limit (40 % of the initial spread), a factor is applied to inflate the parameter ensemble spread to this value. During this process, the ensemble structure of parameter remains unchanged. In addition, to avoid the uncertainty and complexity of evaluating cross covariance between media that have too different characteristic timescales (Han et al., 2013), in the SE of this study, we only allow x2 observation impact on all x variables, and w (η) observation impact on w (η) itself, while the PE could use different medium observations.

Twin experiments are set to test the relation between coupled SE and PE. The model with the standard parameter values described in Sect. 2.1 is running 103 time units (TUs) after a spinup of 103 TUs (2 × 103 TUs in total). Here, a TU is a dimensionless time unit as defined in Lorenz (1963), roughly referring to the timescale of atmosphere going through from an attractive lob to the other (1 TU equals 100 steps of the model integrations with a Δt of 0.01). The output of last 103 TUs is then used as the “truth” to produce “observations”. The observations are sampled as the truth values superimposed by a white noise with an observational interval (five time steps in this case). To simplify, we sample the atmosphere observations by x2 and ocean observations by w as the basic experiment setting. The standard deviation of observational errors (from the in situ instruments, for example) is 2 for x2 and 0.2 for w in our cases. Note that what we describe here is a kind of observing system simulation experiment (OSSE; e.g., Tong and Xue, 2005; Jung et al., 2010). The assimilation model control is an ensemble of integrations for each test case with the perturbed parameters on an erroneously set parameter value (will be described later). The initial conditions of the ensemble for assimilation are taken from the end of a 103-TU spinup (the different members in the ensemble are all the resulting consequences from the parameter perturbation).

The first set of PE experiments is done to study the parameters in “air–sea” interaction. To do that, we use two parameters – a2 in the atmosphere equation and c2 in the ocean equation to perform PE experiments. We first conduct two PE cases with full SE – both x and w are constrained by their observations. Then, we conduct eight PE cases with partial SE – only some medium is constrained by its observations as listed in Table 1. Through thoroughly analyzing these PE cases with partial SE, which have different SE accuracy, we are able to detect the influence of the SE accuracy in different media on coupled model PE. After the first set PE experiments, we also conduct a second set of PE experiments to examine the influence of state estimation accuracy on the “deep ocean” parameter c6 using η observations (the observational error is set as 0.1).

In all PE cases, the initial value of the parameter to be estimated is deliberately set as biased from the truth (referring to the standard parameter values described in Sect. 2.1). To maintain the chaotic nature of the Lorenz equation, parameter values are required to be within a certain range. This is a constraint for the biased amount of the initial values of a parameter. Based on some sensitivity studies, the chaotic performance is more vulnerable to the change of the atmospheric parameter a2 than to the change of the oceanic parameters. Therefore, we set the ensemble initial values of a2 as a Gaussian distribution N (30, 1; 30 as the mean and 1 as the standard deviation), and the spread is enough for the model ensemble uncertainty. The ensemble initial values of c2 are set as N (0.8, 0.5; restricted to be positive definite). If PE is successful, then the ensemble mean value of a2 (c2) should converge to 28 (1). In all PE experiments, the PE is activated after 80 TUs of SE constrain the model states close to the observations so as to enhance the parameter–state covariance for the coupled PE (Zhang et al., 2012). The delayed timescale of PE from SE will be discussed later.

In Table 1, X-to-Y means using observations of X to estimate the parameter Y (x2-to-a2 means using observations of x2 to estimate parameter a2, for instance). Table 1 shows that all four PE cases with atmospheric SE succeed while all four PE cases with oceanic SE fail, no matter what medium observations are used to estimate which medium parameter. The root mean square error (RMSE) of the model states and parameters during the last 100 TUs are shown in Table 2. The x2 RMSEs in the failed cases are higher than in the successful cases, while the w RMSEs in two failed cases F(5) and F(6) are even smaller than the ones in successful cases. Also, in both cases of S(2) and F(6), a2 is estimated by w observations but only when the x states are constrained by x2 observations, the PE is successful or otherwise the PE has failed, although the state w is constrained by the w observations. These suggest that the uncertainty of x2 is mainly responsible for the failure of the PE. An example of failed PE in which the observations of w are used to estimate a2 is shown in Fig. 1b. We can see the ensemble of a2 in Fig. 1b cannot converge to its true value of 28. We will thoroughly analyze such failed cases next.

The stability of PE is different among partial SE settings, as shown in Figs. 2 and 3, as the time series of the ensemble mean of the estimated parameters. Figs. 2b, c and 3b, c show the four successful cases with only atmospheric SE. Compared to full SE (using observations of x2 and w, shown in Figs. 2a and 3a), the partial SE cases show much bigger fluctuation in estimated parameter values at the beginning of spinup period (Figs. 2b, c and 3b, c). From Figs. 2 and 3, it can also be seen that generally the accuracy of PE with partial SE is lower although overall the estimated parameter values converge to the truth. This can be comprehended by the lower signal-to-noise ratio of state–parameter covariance provided by the SE process, which will be discussed in more detail at the end of this section.

The convergence rate of PE is also obviously different with different SE settings. The case of w-to-a2 converges much more slowly than the other cases in a2 estimation. This phenomenon can be explained by the different timescales of different media. Figure 4 shows the variation of the state variable during SE. The observational constraint makes the mean value and the whole ensemble follow the truth (see Fig. 4a for x2 and Fig. 4e for w). It can be seen that in cases assimilating x2, due to no direct constraint on w and η, their spread shrinks slowly. Instead, they are forced by the constrained x2 but with slower adjustment of ocean processes. As mentioned in Sect. 2.3, the SE starts before the PE to make sure the state needed is constrained enough. Slow shrinking of w and η spreads shall be considered in determining a longer delayed time for the PE related to w and η.

The inflation method is also important in PE (Yang and DelSole, 2009; DelSole and Yang, 2010; Zhang, 2011a, b; Zhang et al., 2012). The partial and full SE cases use the same inflation scheme (Zhang, 2011a, b; Zhang et al., 2012). Shadows in Figs. 1–3 show the range of the parameter ensemble. The zigzag shape of the shadows represents the inflation during PE. In these figures, the width of the shadows shrinks quickly once PE is activated, while some of the mean values move toward the truth slowly (for example, Figs. 2c and 3b). Also from the zigzag shapes, we can see some inflation effects before the parameter converges to the truth. All of these imply that the designed PE is stable and its convergence rate is not very sensitive to the inflation scheme.

In addition, larger ensemble sizes are used to test the sensitivity of the conclusion above. The results show that bigger ensemble size has a positive impact on SE and PE quality but the drawn conclusion from the experiments above does not change its essence. Also, the ensemble size far exceeds the problem size in this simple model study. In this regard, further examination may be necessary in CGCM cases. We also performed the experiments under different SE update interval settings. Test results show that for the issue we are addressing, the conclusion is not sensitive to the update interval if it is within a reasonable range (x2-to-c2 and w-to-c2 fail on any update interval with SE of w and succeed with SE of x2 in a SE interval range of no larger than 0.3 TUs, for instance).

In case-3 and case-4 of Table 1, we successfully estimate the oceanic parameter c2, suggesting we can use different medium measurements to help calibrate the parameter within a coupled model. In case-3, the atmospheric observations are used for both SE and PE, while in case-4, the atmospheric observations are used for SE but the oceanic observations are used for PE. Case-3 uses only the atmospheric observations to determine an oceanic parameter and does a better job than when the oceanic observations are used in case-4.

The phenomenon above, in estimation of c2, can be comprehended by the air–sea interaction process. What about a pure oceanic parameter (a parameter used for deep ocean, for instance)? It is interesting to see the influence of atmospheric SE accuracy on PE for a deep ocean parameter. To do that, a series of η-to-c6 PE experiments with different SE settings is carried out. The deep ocean observation is generally sparse in the real world. However, within our twin experiment framework described in Sect. 2.3, the observations of η used for our PE can be produced as sufficiently as other variables. All PE experiments on c6 are conducted with η observations (observations of x2 and w are only used in different SE but not used in the PE). The result is shown in Fig. 5. Given the long timescale of η, the η PE experiments are extended to 104 TUs. The PE cases include four SE settings. They are case-1: all state variables, case-2: x1,2,3 only, case-3: w and η, and case-4: η only. Both case-1 and case-2 succeed greatly, but the convergence rate of case-1 is faster than case-2 and the accuracy of case-1 is a little higher than case-2. In case-3, the convergence rate is fast but the estimated values remain in a bias from the truth. Case-4 apparently fails, never stably converging to any value. It is clear that the η-to-c6 PE succeeds only when the atmospheric state is constrained by observations.

It is interesting that once the atmospheric states (the Lorenz equation in this simple model) are constrained by the observations, both the atmospheric parameter (a2) and oceanic parameters (c2 and c6) can be successfully estimated even in the case using the atmospheric observations (x2) to estimate the oceanic parameter (c2) or using the ocean observations (w) to estimate the atmospheric parameter (a2). This seems different from our previous intuition that in situ ocean data are always considered as the first important piece of information for determining the oceanic coefficients. Our results here strongly suggest that, in the future, when real coupled model PE experiments are used for determining the best coefficient values, no matter the atmospheric or oceanic parameters, sufficient and accurate atmospheric measurements will be crucially important. Next, we will conduct more sophisticate analyses to extend our understanding on this point.

In our twin experiment setting, there are three types of model uncertainties: strong nonlinearity in the atmosphere (chaotic in this case), weaker nonlinearity in the ocean and biased parameter values. The SE process before PE aims to control the first and second types of uncertainties by putting observational constraints on model states. Figure 6 shows the wavelet analyses for the atmospheric variable x2 and the oceanic variable w in the truth run. They represent the uncertainties of type 1 (panel a) and type 2 (panel b). With the expanded exhibition of the wavelet on different periods, Fig. 6 clearly tells significantly different features of x2 and w. The energy of x2 is in the high-frequency band and the energy of w is in the low-frequency band. x2 varies fast and represents the most uncertain mode, transferrable to low-frequency w through the air–sea interaction. Later in Sect. 3.2, we will show that the feedback of ocean can magnify the role of atmospheric chaotic forcings. The chaotic nature can spread out and result in uncertainties in all frequency bands in the system. Under such a circumstance, the method of picking a particular frequency (e.g., Barth et al., 2015) or using averaged covariance (Lu et al., 2015) to implement PE cannot essentially resolve the issue although it may relax the problem. Instead, reducing x2 uncertainty (enhancing the estimation accuracy of the atmospheric states) is more relevant to the solution of the problem.

Without direct observations of parameter values, PE completely relies on the covariance between the parameter and model states for projecting the observational information of states onto the parameter. While the PE projection is carried out by a linear regression equation based on the state–parameter covariance (EnKF/EAKF, for instance), only a linear or quasi-linear relationship between parameters and states in the ensemble is recognized. All failure of PE without direct atmospheric SE could be attributed to the chaotic disturbances in the atmosphere (Lorenz equations in this case) that create difficulties for the system to build up a quasi-linear relationship between the state variable and the parameter.

To investigate the parameter–state relationship in the model background (prior PE), we conduct a series of parameter perturbation runs corresponding to eight partial SE experiments (without PE to fix the parameter spread – the PE process sets the parameter ensemble as an additional system freedom and makes the relationship of the parameter and model state more complicate). In that way, the parameter perturbations can be fully transferred to the model states so that we can study the state–parameter relationship in a straightforward manner. The results are shown in Figs. 7 and 8, where the horizontal axis is the ensemble anomaly (vs. ensemble mean) of the state variable and the vertical axis is the ensemble anomaly of the parameter, and the background black dots represent the model runs starting from different initial conditions. Since the parameter ensemble does not change (once perturbed at the initial time) during the model integration, the lines constructed by black dots in a perturbation run are parallel to the x axis perfectly. However, the set of dots at the same integration time step from different initial conditions can be used to sample the relationship between the perturbed parameter and the model state. For example, two sets of such ensembles, which have the biggest positive and negative correlation coefficients between the parameters and the model states, are colored (20 red dots and 20 blue dots) in each case. From Fig. 7, we can see that with SE for the atmosphere, the overall quasi-linear relationship between the model state anomalies (observational increments) and the parameter adjustments is constructed by the model. Under this circumstance, a meaningful projection from the observational increment on the parameter is gained to form a signal-dominant adjustment for the parameter ensemble. As shown in Fig. 8, without the atmosphere SE, the linear relationship between the parameter being estimated and the model states is not correctly built up, and thus the parameter estimation fails.

The relationship between the states and the parameters can be analyzed quantitatively. Zhang et al. (2012) defined an ad hoc index to measure the signal-to-noise ratio (called rs2n) of a model ensemble. Following the idea, we diagnose the signal-to-noise ratio of the ensemble-based error covariance between the states and parameters here. The new rs2n is defined as R × S, where R is the averaged correlation coefficient between the parameter perturbations and the ensemble states in a selected time window, and S is the ratio of root mean square linear fitting errors of the parameter–state points in the full SE and in a partial SE (Sf / Sp). The best (worst) representation of the signal-to-noise ratio is then characterized by a rs2n value of 1(0). Table 3 gives the rs2n values for the SE-only experiments of Figs. 7 and 8. Correlation coefficients of F(5) and F(8) are 0.19 and 0.24, respectively. Though the dependences of x2 on a2 in F(5) and w on c2 in F(8) are fairly direct, the low R values suggest these relations can be easily interrupted by the atmospheric uncertainty. The values of rs2n are much higher in the successful cases than in the failed cases. These results clearly show that reduction of the atmospheric uncertainty can greatly increase the signal-to-noise ratio of the parameter–state covariance in the system through enhancing the bonding between the state variable and the estimated parameter.

From the results above, we learned that the PE of c2 or c6 strongly relies on the SE of x. In a coupled system characterized as Eq. (1), the influence of atmosphere can thoroughly propagate to all variables of other media, although the influence may reduce for the deep ocean. However, some previous studies (e.g., Annan et al., 2005; Barth et al., 2015; Gharamti et al., 2014; Leeuwenburgh, 2008; Massonnet et al., 2014) show their success in estimating parameters in ocean only using oceanic observations without constraints on atmospheric states. To understand what character of the model makes this difference, we make full use of this simple model with convenience to investigate the influence of model characteristics on coupled parameter estimation. For mimicking the real climate signals, the variability of the oceanic state variables w and η in Eq. (1) are driven by two kinds of forcings: the chaotic forcing from the atmosphere (Lorenz equations) and the periodic forcing associated with the external radiative forcing (simulated by a cosine function with the amplitude coefficient of Ss in this simple model). The oceanic states in the real world consist of both periodic and chaotic variations. The periodic characteristic of a state is naturally with high predictability and is generally easier to be detected after an averaging or filtering process. In this simple model, w (η) is directly under the influence of the parameter c2 (c6) – perturbations of c2 (c6) first directly affecting w (η) and then influencing the whole model by the interactions between w (η) and other variables. To understand the influence of chaotic/periodic variability of the ocean on oceanic parameter estimation, we modify the model in Appendix A to set a one-way coupling model; i.e., only w is forced by x2 but x2 remains independent from w. In that way, we do not need to worry about the instability of Lorenz equations due to the dramatic influence from large w values. Then, we define a chaotic-to-periodic ratio (CPR) in the signals of w (η) to study the PE performance under the different chaotic/periodic variability regimes of a model system. Details of the CPR definition are given in Appendix B. The CPR of w (η) can be easily manipulated by changing the coefficient of Ss. We first compare the results of w in one-way coupling (Fig. 9a) and two-way coupling (Fig. 6b) models with the identical Ss value of 1. The CPR in the full period (from 0.3 to 165 TUs, the longest period that is selected to avoid boundary effects) of w in Fig. 9a is 1.0963. It is interesting that the one-way coupling (without the feedback of w to x2 in the model) can transfer more energy to low-frequency band. Then, we perform eight PE experiments, four for w-to-c2 and four for η-to-c6. We examine four Ss values of 100, 250, 500 and 1000, representing a reduced CPR sequence of w (η). Their CPR values are, respectively, 1.0485 (0.6084), 1.0386 (0.6083), 1.0333 (0.6081) and 1.0282 (0.6080). Note that the CPR value will change once a PE process is activated. We compare these one-way coupling model results and show two examples (Ss= 1 and Ss= 250, Fig. 9a, b for w; Fig. 10a, b for η). We found that the increasing amplitude of periodic forcing can enhance the periodic signals for w and η. Clearly, when the η CPR decreases, the periodic portion dominates and the η-to-c6 PE becomes more and more robust (see Fig. 11a–d). However, in the other four w-to-c2 cases, for any w CPR, the w-to-c2 PE fails (Fig. 12a). This is due to strong dependence of cov(w, c2) (the covariance between w and c2) on x2 (see Eq. 1) that is still chaotic without observational constraint. Though w is very periodic, the chaotic variability of x2 sheds on the variability of w (the needed variability of w for PE should come from c2 but now comes from the chaotic x2) and makes the PE process misjudge the difference between the simulated w and its observation, thus not producing a correct PE projection.

To further test the role of periodic signals in ocean states for oceanic PE, we conduct oceanic PE on a particular frequency band using the method described in Appendix C. Some results are shown in Fig. 12 which shows that using the covariance of η in a particular frequency and c6 to project the corresponding η observational information can make a η-to-c6 PE case with Ss= 250 as successful as the result of Ss= 1000 with full frequencies (compare Fig. 12b to 11d). The method is designed to limit the PE process working on the 10-TU period of η information, which dramatically reduces the CPR of wη (the CPR of η now is 0.1424, and the CPR of w is 1.0525 at the beginning of the PE) and thus helps c6 estimation, but given strong dependence of cov(w, c2) on x2, and that the CPR of x2 is big on every frequency band, this particular frequency PE method does not help for estimation of c2 (Fig. 12a).