Controllability, not chaos, key criterion for ocean state estimation

Gebbie, Geoffrey; Hsieh, Tsung-Lin

doi:https://doi.org/10.5194/npg-24-351-2017

Articles | Volume 24, issue 3

https://doi.org/10.5194/npg-24-351-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/npg-24-351-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 24, issue 3

Research article

|

19 Jul 2017

Research article |

| 19 Jul 2017

Controllability, not chaos, key criterion for ocean state estimation

Geoffrey Gebbie and Tsung-Lin Hsieh

Download

Final revised paper (published on 19 Jul 2017)
Preprint (discussion started on 30 Sep 2016)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Controllability, not chaos, key criterion for ocean state estimation', Anonymous Referee #1, 26 Oct 2016
- AC1: 'Response to Reviewer 1', Geoffrey Gebbie, 06 Jan 2017
RC2: 'Reviewer comments on "Controllability, not chaos, key criterion for ocean state estimation" by Geoffrey Debbie and Tsung-Lin Hsieh', Anonymous Referee #2, 03 Nov 2016
- AC2: 'Response to Reviewer 2', Geoffrey Gebbie, 06 Jan 2017
RC3: 'reviewer comments', Anonymous Referee #3, 16 Nov 2016
- AC3: 'Response to Reviewer 3', Geoffrey Gebbie, 06 Jan 2017

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Geoffrey Gebbie on behalf of the Authors (06 Jan 2017) Author's response Manuscript

ED: Reconsider after major revisions (further review by Editor and Referees) (07 Feb 2017) by Zoltan Toth

Dear Authors,
Thank you for the careful revision of your manuscript and your response to comments from the three expert Reviewers! Since all Reviewers recommended Major revisions, I am asking them if they can comment on the revised manuscript. Beyond potential additional comments from the Reviewers, I also offer some suggestions below, mostly editorial. I hope that with some additional revisions your manuscript will become ready for publication in NPG. Thank you for submitting your research results for publication to NPG
Zoltan Toth

p2 l8 - "and it does not pose"
p9, l35: "The nonlinear timescale here IS defined"
p12, l30 - "COMPLETE observability"
p13 l15 - "In THIS section"

Response to Rev. 1:
"The reviewer’s point about the underdetermined nature of the problem is consistent with our discus- sion in Sec. 3.3, where we acknowledge that the solution is not unique, but we focus on finding any acceptable fit. We view the problem as having two clear steps. It is a first step to find any solution that acceptably fits the data. Only then can we proceed to investigate the uniqueness of the solution. In real-world situations, the first step may be the only one that is practical."
- While I agree the Rev's point is consistent with your discussion in Section 3.3, perhaps you could use more direct language in Section 3.3 (or elsewhere, similar to your response to the Reviewer's comment?)

Rev. 3 comments
"The controllability in this article is very vaguely defined."
- Perhaps you could clarify, as you define "controllability" on p.3, that in your study, you control the system via adjustments to the atmospheric boundary forcing?

comments on p3 & p14 - perhaps in passing by, the dependent relationship between omega and theta could be pointed out in the manuscript

Hide

ED: Referee Nomination & Report Request started (09 Feb 2017) by Zoltan Toth

RR by Anonymous Referee #3 (12 Mar 2017)

RR by Anonymous Referee #1 (28 Mar 2017)

Suggestions for revision or reasons for rejection

In the revised version the authors have answered many of my initial questions, but some questions remain and the revisions have raised new questions. As I now understand it, the major point of the paper is that earlier results that seemed to show that 4DVAR didn’t work so well in chaotic systems is that, in the cases cited, the control space didn’t have enough degrees of freedom. With insufficient degrees of freedom in the control space, the problem becomes that of finding a minimum in the surface depicted in figure 2. From this point of view, the punch line to the manuscript is figure 7.

The conclusion that initial conditions are not sufficient to control a chaotic system in many cases of interest is well known, but it’s useful to see it stated succinctly in one place. The philosophical problem I have with the approach taken in this manuscript is that it seems to imply that a control space of greater dimension is always preferable to one of lesser dimension. This is definitely not the case. For the linear problem, the dimension of the control space should not exceed the number of observations, and conditioning issues will usually restrict the dimension further. This is not necessarily so for nonlinear problems, but it is a useful guide.

As I said before, it is good to see the chi-square test used systematically in this way, but the authors should note that it is J (equation 6) rather than J_d (equation 4) that is the useful chi-square random variable, the randomness arising only from the observation noise. If only J_d is considered, there will be no indication if the prior control estimates and their statistics were reasonable; see the texts by Bennett or Kalnay.

p5, line 1: “Sx and Sf are weights that restrict the size of the perturbations to 5rad and 10rad s−2, respectively.” I wouldn’t say “restrict.” Greater perturbations are permitted, but penalized more severely.

p6, line 20: Please give the definition of the controllability matrix, and state the theorem that specifies the condition for controllability. If I don’t remember, neither do 90% of the readers of NPG. It’s not reasonable to send your readers scurrying to the literature to find those things at this point.

p7 The “method of total inversion” needs some further explanation. The terminology is not standard in the community that constitutes the readership of NPG. Without further explanation, it’s hard to know what to make of (12). Consider the expression in the first square brackets. The matrix E is 1x2 (p4, line 24) and the matrix Q appears to be 3x3. It’s not even clear that (12) is a valid vector expression. Also, “…we update the state transition and controllability matrices iteratively.” Since that is the case, you probably want the last expression to reflect the evolution state from i\Delta t_y to (i+1)\Delta t_y (pardon my TeX, but I don’t see how to put symbols into this writeup)

Starting with the improved first guess and then iterating until a satisfactory chi-square statistic is achieved is a good strategy. I have seen very, very few inversions of realistic models of the ocean or atmosphere that satisfy the chi-square condition. It’s really the chi-square statistics that allow us to conclude that the black curve in the bottom panel of figure 3 is to be preferred to the gray curve. The real world is rarely so accommodating. The last sentence of section 3.2, “The method of Lagrange multipliers is therefore superior to the first-guess estimate because all components of the solution, both the state and forcing, can be physically interpreted.” is only meaningful to the extent that the chi-square statistics say it is.

p9: “…small-scale features that are added to the surface forcing of ocean models in order to fit observations (e.g., Stammer et al., 2002b)” The real problem with the reconstructed forcing in Stammer et al. (2002b) was not small scale features but large scale features because of inaccurate model dynamics.

p10: As is strongly implied in the following discussion, the statement that “…there appears to be no lower limit on the number of observations necessary in order to produce an acceptable state estimate with this model” is only valid if there are sufficient degrees of freedom in the control space. The Lagrange multiplier method would not be nearly so successful if the control space were restricted to the initial condition, as was the case in some of the earlier studies that cast doubt on the efficacy of 4DVAR in chaotic systems. This point is nicely made in figures 2 and 7 of the present manuscript.

p10: “…the null hypothesis that the model is consistent with observations.” It is true that, for the area above the heavy line in figure 6 a satisfactory fit to data can be obtained, but a too-low chi square statistic is evidence of overfitting, and the reconstructed inputs cannot be trusted.

Also, figure 6 depicts a contour plot of J_d. It does not, therefore, distinguish among different choices of forcing fields.

last sentence: “In the ocean state estimation problem, all air-sea fluxes are uncertain and temporally variable so there is no shortage of controls that can be defined.” That is true, but amount of independent air-sea flux data is limited. There may be no shortage of controls that can be defined, but there is a very real shortage of controls that can be constrained by observation.

p12: Relation to the Kalman filter/smoother. Line 21: “Both methods solve the same least squares problem …” This is not strictly true. The Kalman smoother solves the same least squares problem as 4DVAR, but the Kalman filter does not. The Kalman filter and 4DVAR are only guaranteed to agree at the final time.

Figure 9 gives a nice illustration of the effect of nonlinearity.

Hide

ED: Publish subject to minor revisions (further review by Editor) (17 Apr 2017) by Zoltan Toth

Dear Authors,
Clearly, you invested significant efforts in carrying out your research, preparing, and then revising your manuscript. Still, Rev. 1 recommends rejection, Rev. 2 further revisions of your manuscript. The primary reason may be that as is, your manuscript is inaccessible to most readers of NPG.

Rev. 2 acknowledges the care you put into the revision of your manuscript, yet (s)he comments that the additional details raise further questions and may have made the manuscript even harder to follow. Nevertheless, after the revision, Rev. 2 is able to identify your main message; however, Rev. 1 is still uncertain about the new findings and value of your work, lacking any evidence with more complex systems.

Both Reviewers comment on the lack lack of clarity in your manuscript. Rev. 2 also offers a number of additional technical questions and comments.

I find that your work may have a potential contribution to the field. However, before your manuscript can be considered for publication in NPG, its presentation must be improved considerably and the remaining technical issues clarified.

In particular, you must make the manuscript more accessible to the general readership of NPG. Please note Rev. 2's comments related to p.6, p.7, and p.10 specifically, and also as general guidelines while you attempt to make your work more accessible. Be sure to use standard terminology. Must you deviate from it or introduce new terms, do it in a way that allows uninitiated readers to still reasonably follow your presentation. And please highlight your main findings in clear language in the Abstract and Conclusions section (refer to Rev. 2's introductory par.)

As for Rev. 2's other, technical comments, please respond to each of them. Please pay special attention to Rev. 2's comments on whether greater control space dimension necessarily is always preferable, and the use of J vs. J_d in your Chi-square tests.

I hope you will take the extra effort to bring your manuscript up to the standards of NPG and invest in a second revision of your manuscript. If you do so, along with your revised manuscript, please submit a point by point response to each comment by the reviewers. Upon receipt, I will inspect your revised submission and may also request an additional review by one of the original Referees.

Thank you for considering NPG for the publication of your research results and looking forward to receiving your revised manuscript.
Zoltan Toth, Editor

Hide

AR by Geoffrey Gebbie on behalf of the Authors (29 Apr 2017) Author's response Manuscript

ED: Publish as is (05 Jun 2017) by Zoltan Toth

AR by Geoffrey Gebbie on behalf of the Authors (09 Jun 2017) Manuscript

Short summary

The best reconstructions of the past ocean state involve the statistical combination of numerical models and observations; however, the computationally efficient method that produces physically interpretable fields is thought to not be applicable to chaotic dynamical systems, such as ocean models with eddies. Here we use a model of the chaotic, forced pendulum to show that the most popular existing method is successful so long as there are enough uncertain boundary conditions through time.