Bolton, T. and Zanna, L.: Applications of
deep learning to ocean data inference and subgrid parameterization, J. Adv. Model. Earth Sy.,
11, 376–399, 2019. a
Brunton, S. and Kutz, J.: Data-driven Science
and Engineering: Machine Learning, Dynamical Systems, and Control, Cambridge University Press,
2019. a
Chantry,
M., Thornes, T., Palmer, T., and Düben, P.: Scale-selective precision for weather and climate
forecasting, Mon. Weather Rev., 147, 645–655, 2019. a
Chattopadhya, A.:
RC_ESN_spatio_temporal, available at:
https://github.com/ashesh6810/RCESN_spatio_temporal, GitHub,
last access: 29 June 2020. a
Chattopadhyay, A., Nabizadeh, E., and Hassanzadeh, P.:
Analog forecasting of extreme-causing weather patterns using deep learning, J. Adv.
Model. Earth Sy., 12,
https://doi.org/10.1029/2019MS001958,
2020b.
a
Chattopadhyay, A., Subel, A., and Hassanzadeh, P.:
Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96
systems and transfer-learning, arXiv [preprint],
arXiv:2002.11167, 25 February 2020c.
a
Chen, K., Zhou, Y., and Dai, F.: A
LSTM-based method for stock returns prediction: A case study of China stock market, in: 2015
IEEE International Conference on Big Data, 29 October–1 November, Santa Clara, CA, USA, 2823–2824, IEEE, 2015. a
Cho, K., V.M, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk,
H., and Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical
machine translation, arXiv [preprint],
arXiv:1406.1078, 2014.
a
Collins, W., Rasch, P., Boville, B., Hack, J., McCaa, J.,
Williamson, D., Briegleb, B., Bitz, C., Lin, S., and Zhang, M.: The formulation and atmospheric
simulation of the Community Atmosphere Model version 3 (CAM3), J. Climate, 19, 2144–2161, 2006. a
Collins, W. J., Bellouin, N.,
Doutriaux-Boucher, M., Gedney, N., Halloran, P., Hinton, T., Hughes, J., Jones, C. D., Joshi, M.,
Liddicoat, S., Martin, G., O'Connor, F., Rae, J., Senior, C., Sitch, S., Totterdell, I.,
Wiltshire, A., and Woodward, S.: Development and evaluation of an Earth-System model – HadGEM2,
Geosci. Model Dev., 4, 1051–1075,
https://doi.org/10.5194/gmd-4-1051-2011, 2011.
a
Düben, P. and Palmer, T.: Benchmark
tests for numerical weather forecasts on inexact hardware, Mon. Weather Rev., 142, 3809–3829,
2014.
a,
b
Düben, P., Joven, J., Lingamneni, A., McNamara, H., De Micheli, G.,
Palem, K., and Palmer, T.: On the use of inexact, pruned hardware in atmospheric modelling,
Philos. T. Roy. Soc. A-Math., 372, 1–16,
https://doi.org/10.1098/rsta.2013.0276, 2014.
a
Düben, P., Yenugula, S., Augustine, J., Palem, K.,
Schlachter, J., Enz, C., and Palmer, T.: Opportunities for energy efficient computing: A study of
inexact general purpose processors for high-performance and big-data applications, in: 2015
Design, Automation and Test in Europe Conference and Exhibition (DATE), 9–12 March, Grenoble, France, 764–769,
IEEE, 2015. a
Dueben, P. D. and Bauer, P.: Challenges and
design choices for global weather and climate models based on machine learning, Geosci. Model
Dev., 11, 3999–4009,
https://doi.org/10.5194/gmd-11-3999-2018, 2018.
a,
b,
c,
d,
e,
f,
g,
h,
i,
j,
k
Epanechnikov, V.: Non-parametric estimation of a
multivariate probability density, Theor. Prob. Appl., 14, 153–158, 1969. a
Fan, H., Jiang, J., Zhang,
C., Wang, X., and Lai, Y.-C.: Long-term prediction of chaotic systems with machine learning,
Phys. Rev. Res., 2, 012080,
https://doi.org/10.1103/PhysRevResearch.2.012080, 2020.
a
Flato, G.: Earth system models: an overview, WIRES
Clim. Change, 2, 783–800, 2011. a
Gagne, I., John, D., Christensen, H. M., Subramanian, A. C., and Monahan, A. H.: Machine Learning
for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz'96 Model,
J. Adv. Model. Earth Sy., 12,
https://doi.org/10.1029/2019MS001896, 2020.
a
Garcia, R., Smith, A., Kinnison, D., Cámara, Á. l., and
Murphy, D.: Modification of the gravity wave parameterization in the Whole Atmosphere Community
Climate Model: Motivation and results, J. Atmos. Sci., 74, 275–291, 2017. a
Gauthier, D.: Reservoir computing: Harnessing a
universal dynamical system, SIAM News, 51, 12, 2018.
a,
b
Gentine, P., Pritchard, M., Rasp, S., Reinaudi, G., and Yacalis, G.: Could machine learning break
the convection parameterization deadlock?, Geophys. Res. Lett., 45, 5742–5751, 2018. a
Goodfellow,
I., Bengio, Y., and Courville, A.: Deep learning, MIT Press, 2016.
a,
b,
c,
d
Graves, A., Mohamed,
A., and Hinton, G.: Speech recognition with deep recurrent neural networks, in: 2013 IEEE
international Conference on Acoustics, Speech and Signal Processing, 26–31 May, Vancouver, Canada,
6645–6649, IEEE, 2013. a
Hatfield, S., Subramanian, A., Palmer, T., and Düben, P.: Improving weather forecast skill
through reduced-precision data assimilation, Mon. Weather Rev., 146, 49–62, 2018. a
Hourdin, F., M., T., G., A., Golaz, J., Balaji, V., Duan, Q., Folini,
D., Ji, D., Klocke, D., Qian, Y., et al.: The art and science of climate model tuning,
B. Am. Meteorol. Soc., 98, 589–602, 2017. a
Jaeger, H. and Haas, H.: Harnessing
nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science,
304, 78–80, 2004.
a,
b,
c
Jeevanjee, N., Hassanzadeh, P., Hill, S., and Sheshadri,
A.: A perspective on climate model hierarchies, J. Adv. Model. Earth Sy., 9, 1760–1771, 2017.
a,
b
Khairoutdinov, M. and Randall,
D.: A cloud resolving model as a cloud parameterization in the NCAR Community Climate System
Model: Preliminary results, Geophys. Res. Lett., 28, 3617–3620, 2001. a
Khodkar, M. and Hassanzadeh, P.:
Data-driven reduced modelling of turbulent Rayleigh–Bénard convection using DMD-enhanced
fluctuation–dissipation theorem, J. Fluid Mech., 852,
https://doi.org/10.1017/jfm.2018.586, 2018.
a
Khodkar,
M., Hassanzadeh, P., Nabi, S., and Grover, P.: Reduced-order modeling of fully turbulent
buoyancy-driven flows using the Green's function method, Phys. Rev. Fluids, 4,
013801,
https://doi.org/10.1103/PhysRevFluids.4.013801, 2019.
a
Kim, H., Eykholt, R., and
Salas, J.: Nonlinear dynamics, delay times, and embedding windows, Phys. D: Nonlin. Phenom., 127,
48–60, 1999. a
Kingma, D. and Ba, J.: Adam: A method for stochastic
optimization, arXiv [preprint],
arXiv:1412.6980, 22 December 2014.
a
Kooperman, G., Pritchard, M., O'Brien, T., and Timmermans,
B.: Rainfall From Resolved Rather Than Parameterized Processes Better Represents the Present-Day
and Climate Change Response of Moderate Rates in the Community Atmosphere Model,
J. Adv. Model. Earth Sy., 10, 971–988, 2018. a
Kutz, J.: Deep learning in fluid dynamics, J. Fluid Mech., 814,
1–4, 2017. a
Leyffer, S., Wild, S., Fagan, M., Snir, M., Palem, K., Yoshii, K.,
and Finkel, H.: Doing Moore with Less–Leapfrogging Moore's Law with Inexactness for
Supercomputing, arXiv [preprint],
arXiv:1610.02606, 9 October 2016.
a,
b
Lim, S. H.,
Giorgini, L. T., Moon, W., and Wettlaufer, J.: Predicting Rare Events in Multiscale Dynamical
Systems using Machine Learning, arXiv [preprint],
arXiv:1908.03771, 10 August 2019.
a,
b
Lorenz, E.: Predictability: A problem partly
solved, in: Predcitibility of Weather and Climate, vol. 1, 40–58, ECMWF, 1996. a
Lu, Z.,
Pathak, J., Hunt, B., Girvan, M., Brockett, R., and Ott, E.: Reservoir observers: Model-free
inference of unmeasured variables in chaotic systems, Chaos, 27, 041102,
https://doi.org/10.1063/1.4979665, 2017.
a,
b,
c
Lu, Z., Hunt, B., and Ott, E.:
Attractor reconstruction by machine learning, Chaos, 28, 061104,
https://doi.org/10.1063/1.5039508, 2018.
a
Ma, Q., Shen, L.,
Chen, E., Tian, S., Wang, J., and Cottrell, G.: WALKING WALKing walking: Action Recognition from
Action Echoes, in: Internationa Joint Conference on Artificial Intelligence, 19–25 August, Melbourne, Australia, 2457–2463, 2017. a
McDermott, P. and Wikle, C.: An
ensemble quadratic echo state network for non-linear spatio-temporal forecasting, Stat, 6,
315–330, 2017.
a,
b
McDermott, P. and Wikle, C.: Deep echo
state networks with uncertainty quantification for spatio-temporal forecasting, Environmetrics,
30, e2553,
https://doi.org/10.1002/env.2553, 2019.
a,
b
Meng, Q., Chen, W.,
Wang, Y., Ma, Z.-M., and Liu, T.-Y.: Convergence analysis of distributed stochastic gradient
descent with shuffling, Neurocomputing, 337, 46–57, 2019. a
Mezić, I.: Spectral properties of dynamical
systems, model reduction and decompositions, Nonlin. Dynam., 41, 309–325, 2005. a
Mohan, A.,
Daniel, D., Chertkov, M., and Livescu, D.: Compressed Convolutional LSTM: An Efficient Deep
Learning framework to Model High Fidelity 3D Turbulence, arXiv [preprint],
arXiv:1903.00033, 28 February 2019.
a,
b
Moosavi, A., Attia,
A., and Sandu, A.: A machine learning approach to adaptive covariance localization, arXiv [preprint],
arXiv:1801.00548, 2 January 2018.
a
O'Gorman, P. and Dwyer, J.: Using Machine Learning
to Parameterize Moist Convection: Potential for Modeling of Climate, Climate Change, and Extreme
Events, J. Adv. Model. Earth Sy., 10, 2548–2563, 2018. a
Pascanu, R.,
Mikolov, T., and Bengio, Y.: On the difficulty of training recurrent neural networks, in:
International Conference on Machine Learning, 16–21 June, Atlanta, USA, 1310–1318, 2013.
a,
b
Pathak, J., Lu,
Z., Hunt, B., Girvan, M., and Ott, E.: Using machine learning to replicate chaotic attractors and
calculate Lyapunov exponents from data, Chaos, 27, 121102,
https://doi.org/10.1063/1.5010300, 2017.
a,
b,
c,
d,
e
Pathak, J., Hunt, B., Girvan, M., Lu, Z., and Ott, E.: Model-free prediction of large
spatiotemporally chaotic systems from data: A reservoir computing approach, Phys. Rev. Lett., 27,
https://doi.org/10.1063/1.5010300, 2018a.
a,
b,
c,
d,
e,
f,
g,
h,
i,
j
Pathak, J., Wikner, A., Fussell, R., Chandra, S., Hunt, B., Girvan, M.,
and Ott, E.: Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a
knowledge-based model, Chaos, 28, 041101,
https://doi.org/10.1063/1.5028373, 2018b.
a
Raissi, M.,
Perdikaris, P., and Karniadakis, G.: Physics-informed neural networks: A deep learning framework
for solving forward and inverse problems involving nonlinear partial differential equations,
J. Comput. Phys., 378, 686–707, 2019. a
Reichstein, M., Camps-Vallis, G., Stevens, B., Jung, M., Denzler, J., and Prabhat, N. C.: Deep learning and process understanding for data-driven Earth
system science, Nature, 566, 195–204,
https://doi.org/10.1038/s41586-019-0912-1,
2019.
a,
b
Rudy, S., Kutz, J., and Brunton,
S.: Deep learning of dynamics and signal-noise decomposition with time-stepping constraints, arXiv
[preprint],
arXiv:1808.02578, 7 August 2018.
a,
b
Scher, S. and Messori, G.: Generalization properties of feed-forward neural networks trained on Lorenz systems, Nonlin. Processes Geophys., 26, 381–399,
https://doi.org/10.5194/npg-26-381-2019, 2019.
a
Schneider, T., Lan, S., Stuart, A., and Teixeira, J.: Earth
system modeling 2.0: A blueprint for models that learn from observations and targeted
high-resolution simulations, Geophys. Res. Lett., 44, 12396–12417,
https://doi.org/10.1002/2017GL076101, 2017a.
a,
b
Schneider, T., Teixeira, J., Bretherton, C.,
Brient, F., Pressel, K., Schär, C., and Siebesma, A.: Climate goals and computing the future
of clouds, Nat. Clim. Change, 7, 3–5,
https://doi.org/10.1038/nclimate3190,
2017b.
a
Siegelmann, H. T. and Sontag,
E. D.: On the computational power of neural nets, in: Proceedings of the fifth annual workshop on
Computational learning theory, July, Pittsburgh, Pennsylvania, USA,
440–449, 1992. a
Sutskever, I.,
Vinyals, O., and Le, Q.: Sequence to sequence learning with neural networks, in: Advances in
Neural Information Processing Systems, 8–13 December, Montreal, Canada, 3104–3112, 2014. a
Thornes, T.,
Düben, P., and Palmer, T.: On the use of scale-dependent precision in Earth System
modelling, Q. J. Roy. Meteorol. Soc., 143, 897–908, 2017.
a,
b,
c,
d,
e,
f
Toms, B. A., Kashinath, K.,
Yang, D., et al.: Deep Learning for Scientific Inference from Geophysical Data: The
Madden–Julian oscillation as a Test Case, arXiv [preprint],
arXiv:1902.04621, 12 February 2019.
a
Tu, J., Rowley,
C., Luchtenburg, D., Brunton, S., and Kutz, J.: On dynamic mode decomposition: Theory and
applications, J. Comput. Dynam., 1, 391–421, 2014. a
Vlachas, P., Byeon, W., Wan, Z., Sapsis, T., and Koumoutsakos, P.: Data-driven forecasting of
high-dimensional chaotic systems with long short-term memory networks, P. Roy. Soc. A-Math., 474,
20170844,
https://doi.org/10.1098/rspa.2017.0844, 2018.
a,
b,
c,
d,
e
Watson, P.: Applying machine learning to improve
simulations of a chaotic dynamical system using empirical error correction, J. Adv. Model. Earth
Sy., 11, 1402–1417,
https://doi.org/10.1029/2018MS001597, 2019.
a
Wu, J. et al.: Enforcing Statistical Constraints in
Generative Adversarial Networks for Modeling Chaotic Dynamical Systems, arXiv [preprint],
arXiv:1905.06841, 13 May 2019.
a,
b
Xingjian, S., Chen, Z., Wang, H., Yeung, D., Wong, W., and Woo,
W.: Convolutional LSTM network: A machine learning approach for precipitation nowcasting, in:
Advances in Neural Information Processing Systems, 7–12 December, Montreal, Canada, 802–810, 2015. a
Yu, R., Zheng, S., Anandkumar,
A., and Yue, Y.: Long-term forecasting using tensor-train RNNs, arXiv [preprint],
arXiv:1711.00073,
31 October 2017.
a
Zhu, Y.,
Zabaras, N., Koutsourelakis, P., and Perdikaris, P.: Physics-Constrained Deep Learning for
High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data, arXiv
[preprint],
arXiv:1901.06314, 18 January 2019.
a
Zimmermann, R. and Parlitz, U.:
Observing spatio-temporal dynamics of excitable media using reservoir computing, Chaos, 28,
043118,
https://doi.org/10.1063/1.5022276, 2018.
a
