Document Zbl 07867355

Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations. (English) Zbl 07867355

Chaos 31, No. 10, Article ID 101103, 8 p. (2021).

MSC:

68Txx	Artificial intelligence
62Mxx	Inference from stochastic processes
37Mxx	Approximation methods and numerical treatment of dynamical systems

Software:

EnKF

Cite Review PDF

Full Text: DOI arXiv

References:

[1]	Han, J.; Jentzen, A.; E, W., Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci. U.S.A., 115, 8505-8510, 2018 · Zbl 1416.35137 · doi:10.1073/pnas.1718942115
[2]	Han, J.; Ma, C.; Ma, Z.; E, W., Uniformly accurate machine learning-based hydrodynamic models for kinetic equations, Proc. Natl. Acad. Sci. U.S.A., 116, 21983-21991, 2019 · Zbl 1431.76100 · doi:10.1073/pnas.1909854116
[3]	Raissi, M.; Karniadakis, G. E., Hidden physics models: Machine learning of nonlinear partial differential equations, J. Comput. Phys., 357, 125-141, 2018 · Zbl 1381.68248 · doi:10.1016/j.jcp.2017.11.039
[4]	Raissi, M.; Perdikaris, P.; Karniadakis, G., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707, 2019 · Zbl 1415.68175 · doi:10.1016/j.jcp.2018.10.045
[5]	Schneider, T.; Lan, S.; Stuart, A.; Teixeira, J., Earth system modeling 2.0: A blueprint for models that learn from observations and targeted high-resolution simulations, Geophys. Res. Lett., 44, 12, 396-417, 2017 · doi:10.1002/2017GL076101
[6]	Dueben, P. D.; Bauer, P., Challenges and design choices for global weather and climate models based on machine learning, Geosci. Model Dev., 11, 3999-4009, 2018 · doi:10.5194/gmd-11-3999-2018
[7]	Rasp, S.; Pritchard, M. S.; Gentine, P., Deep learning to represent subgrid processes in climate models, Proc. Natl. Acad. Sci. U.S.A., 115, 9684-9689, 2018 · doi:10.1073/pnas.1810286115
[8]	Gagne, D. J.; Christensen, H. M.; Subramanian, A. C.; Monahan, A. H., Machine learning for stochastic parameterization: Generative adversarial networks in the Lorenz ’96 model, J. Adv. Model. Earth Syst., 12, e2019MS001896, 2020 · doi:10.1029/2019MS001896
[9]	Bolton, T.; Zanna, L., Applications of deep learning to ocean data inference and subgrid parameterization, J. Adv. Model. Earth Syst., 11, 376-399, 2019 · doi:10.1029/2018MS001472
[10]	Brajard, J.; Carrassi, A.; Bocquet, M.; Bertino, L., Combining data assimilation and machine learning to infer unresolved scale parametrization, Philos. Trans. R. Soc. A, 379, 20200086, 2021 · doi:10.1098/rsta.2020.0086
[11]	Nadiga, B., Reservoir computing as a tool for climate predictability studies, J. Adv. Model. Earth Syst., 13, e2020MS002290, 2020 · doi:10.1029/2020MS002290
[12]	Cleary, E.; Garbuno-Inigo, A.; Lan, S.; Schneider, T.; Stuart, A. M., Calibrate, emulate, sample, J. Comput. Phys., 424, 109716, 2021 · Zbl 07508438 · doi:10.1016/j.jcp.2020.109716
[13]	Rahimi, A. and Recht, B., “Random features for large-scale kernel machines,” in Advances in Neural Information Processing Systems, edited by J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis (Curran Associates, Inc., 2008), Vol. 20, pp. 1177-1184.
[14]	Rahimi, A. and Recht, B., “Uniform approximation of functions with random bases,” in 2008 46th Annual Allerton Conference on Communication, Control, and Computing (IEEE, 2008), pp. 555-561.
[15]	Bach, F., Breaking the curse of dimensionality with convex neural networks, J. Mach. Learn. Res., 18, 19, 1-53, 2017 · Zbl 1433.68390
[16]	Bach, F., On the equivalence between kernel quadrature rules and random feature expansions, J. Mach. Learn. Res., 18, 21, 1-38, 2017 · Zbl 1435.65045
[17]	Sun, Y.; Gilbert, A.; Tewari, A.
[18]	Park, J.; Sandberg, I., Universal approximation using radial-basis-function networks, Neural Comput., 3, 246-257, 1991 · doi:10.1162/neco.1991.3.2.246
[19]	Cybenko, G., Approximation by superposition of a sigmoidal function, Math. Control Signals Syst., 2, 303-314, 1989 · Zbl 0679.94019 · doi:10.1007/BF02551274
[20]	Barron, A., Universal approximation bounds for superposition of a sigmoidal function, IEEE Trans. Inf. Theory, 39, 930-945, 1993 · Zbl 0818.68126 · doi:10.1109/18.256500
[21]	Maass, W.; Natschläger, T.; Markram, H., Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Comput., 14, 2531-2560, 2002 · Zbl 1057.68618 · doi:10.1162/089976602760407955
[22]	Jaeger, H., “A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the ‘echo state network’ approach,” German National Research Institute for Computer Science (GMD) Report No. 159, 2002.
[23]	Jaeger, H.; Haas, H., Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science, 304, 78-80, 2004 · doi:10.1126/science.1091277
[24]	Pathak, J.; Hunt, B.; Girvan, M.; Lu, Z.; Ott, E., Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach, Phys. Rev. Lett., 120, 024102, 2018 · doi:10.1103/PhysRevLett.120.024102
[25]	Jüngling, T., Lymburn, T., Stemler, T., Correa, D., Walker, D., and Small, M., “Reconstruction of complex dynamical systems from time series using reservoir computing,” in 2019 IEEE International Symposium on Circuits and Systems (ISCAS) (IEE, 2019), pp. 1-5.
[26]	Algar, S. D.; Lymburn, T.; Stemler, T.; Small, M.; Jüngling, T., Learned emergence in selfish collective motion, Chaos, 29, 123101, 2019 · doi:10.1063/1.5120776
[27]	Bollt, E., On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to VAR and DMD, Chaos, 31, 013108, 2021 · Zbl 1512.68265 · doi:10.1063/5.0024890
[28]	Gauthier, D.; Bollt, E.; Griffith, A.; Barbosa, W.
[29]	Platt, J. A.; Wong, A.; Clark, R.; Penny, S. G.; Abarbanel, H. D. I.
[30]	Gottwald, G. A.; Reich, S., Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation, Physica D, 423, 132911, 2021 · Zbl 1508.68288 · doi:10.1016/j.physd.2021.132911
[31]	Evensen, G., Data Assimilation: The Ensemble Kalman Filter, 2006, Springer: Springer, New York · Zbl 1157.86001
[32]	Bocquet, M.; Brajard, J.; Carrassi, A.; Bertino, L., Data assimilation as a learning tool to infer ordinary differential equation representations of dynamical models, Nonlinear Process. Geophys., 26, 143-162, 2019 · doi:10.5194/npg-26-143-2019
[33]	Brajard, J.; Carrassi, A.; Bocquet, M.; Bertino, L., Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the Lorenz 96 model, J. Comput. Sci., 44, 101171, 2020 · doi:10.1016/j.jocs.2020.101171
[34]	Bocquet, M.; Brajard, J.; Carrassi, A.; Bertino, L., Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization, Found. Data Sci., 2, 55-80, 2020 · doi:10.3934/fods.2020004
[35]	Bocquet, M., Farchi, A., and Malartic, Q., “Online learning of both state and dynamics using ensemble Kalman filters,” Found. Data Sci. (to be published). · Zbl 1478.93687
[36]	Abarbanel, H.; Rozdeba, P.; Shirman, S., Machine learning: Deepest learning as statistical data assimilation problems, Neural Comput., 30, 2025-2055, 2018 · Zbl 1472.68127 · doi:10.1162/neco_a_01094
[37]	Tomizawa, F.; Sawada, Y., Geosci. Model Dev. Discuss., 14 · doi:10.5194/gmd-14-5623-2021
[38]	Wikner, A.; Pathak, J.; Hunt, B. R.; Szunyogh, I.; Girvan, M. I.; Ott, E., Using data assimilation to train a hybrid forecast system that combines machine-learning and knowledge-based components, Chaos, 31, 053114, 2021 · Zbl 1462.37090 · doi:10.1063/5.0048050
[39]	Levine, M. E.; Stuart, A. M.
[40]	Takens, F., “Detecting strange attractors in turbulence,” in Dynamical Systems and Turbulence, Warwick 1980, edited by D. Rand and L.-S. Young (Springer, Berlin, 1981), pp. 366-381. · Zbl 0513.58032
[41]	Hamilton, F.; Berry, T.; Sauer, T., Ensemble Kalman filtering without a model, Phys. Rev. X, 6, 011021, 2016 · doi:10.1103/PhysRevX.6.011021
[42]	Lu, Z.; Pathak, J.; Hunt, B.; Girvan, M.; Brockett, R.; Ott, E., Reservoir observers: Model-free inference of unmeasured variables in chaotic systems, Chaos, 27, 041102, 2017 · doi:10.1063/1.4979665
[43]	Hart, A.; Hook, J.; Dawes, J., Embedding and approximation theorems for echo state networks, Neural Netw., 128, 234-247, 2020 · Zbl 1468.68098 · doi:10.1016/j.neunet.2020.05.013
[44]	Nakai, K.; Saiki, Y., Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable, Discrete Contin. Dyn. Syst., 14, 1079-1092, 2021 · Zbl 1467.76046 · doi:10.3934/dcdss.2020352
[45]	Pathak, J.; Wikner, A.; Fussell, R.; Chandra, S.; Hunt, B. R.; Girvan, M.; Ott, E., Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a knowledge-based model, Chaos, 28, 041101, 2018 · doi:10.1063/1.5028373
[46]	Wikner, A.; Pathak, J.; Hunt, B.; Girvan, M.; Arcomano, T.; Szunyogh, I.; Pomerance, A.; Ott, E., Combining machine learning with knowledge-based modeling for scalable forecasting and subgrid-scale closure of large, complex, spatiotemporal systems, Chaos, 30, 053111, 2020 · doi:10.1063/5.0005541
[47]	Sauer, T.; Yorke, J. A.; Casdagli, M., Embedology, J. Stat. Phys., 65, 579-616, 1991 · Zbl 0943.37506 · doi:10.1007/BF01053745
[48]	Sauer, T.; Yorke, J. A., How many delay coordinates do you need?, Int. J. Bifurcation Chaos, 03, 737-744, 2011 · Zbl 0871.58066 · doi:10.1142/S0218127493000647
[49]	Kantz, H.; Schreiber, T., Nonlinear Time Series Analysis, xvi+369, 2004, Cambridge University Press: Cambridge University Press, Cambridge · Zbl 1050.62093
[50]	Schreiber, T., Interdisciplinary application of nonlinear time series methods, Phys. Rep., 308, 1-64, 1999 · doi:10.1016/S0370-1573(98)00035-0
[51]	Small, M., Applied Nonlinear Time Series Analysis, World Scientific Series on Nonlinear Science. Series A: Monographs and Treatises Vol. 52 (World Scientific Publishing Co. Pte Ltd., Hackensack, NJ, 2005), pp. xiv+245. · Zbl 1186.62109
[52]	Casdagli, M.; Eubank, S.; Farmer, J.; Gibson, J., State space reconstruction in the presence of noise, Physica D, 51, 52-98, 1991 · Zbl 0736.62075 · doi:10.1016/0167-2789(91)90222-U
[53]	Schreiber, T.; Kantz, H., Noise in chaotic data: Diagnosis and treatment, Chaos, 5, 133-142, 1995 · doi:10.1063/1.166095
[54]	Kennel, M. B.; Brown, R.; Abarbanel, H. D. I., Determining embedding dimension for phase-space reconstruction using a geometrical construction, Phys. Rev. A, 45, 3403-3411, 1992 · doi:10.1103/PhysRevA.45.3403
[55]	Fraser, A. M.; Swinney, H. L., Independent coordinates for strange attractors from mutual information, Phys. Rev. A, 33, 1134-1140, 1986 · Zbl 1184.37027 · doi:10.1103/PhysRevA.33.1134
[57]	C. Ma, W. E.; Wojtowytsch, S.; Wu, L.
[58]	Rotskoff, G. M.; Vanden-Eijnden, E.
[59]	Burgers, G.; van Leeuwen, P. J.; Evensen, G., Analysis scheme in the ensemble Kalman filter, Mon. Weather Rev., 126, 1719-1724, 1998 · doi:10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2
[60]	Anderson, J. L.; Anderson, S. L., A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts, Mon. Weather Rev., 127, 2741-2758, 1999 · doi:10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2
[61]	Lorenz, E. N., Deterministic nonperiodic flow, J. Atmos. Sci., 20, 130-141, 1963 · Zbl 1417.37129 · doi:10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2
[62]	Ehrendorfer, M., A review of issues in ensemble-based Kalman filtering, Meteorol. Z., 16, 795-818, 2007 · doi:10.1127/0941-2948/2007/0256
[63]	Nadiga, B. T.; Casper, W. R.; Jones, P. W., Ensemble-based global ocean data assimilation, Ocean Modell., 72, 210-230, 2013 · doi:10.1016/j.ocemod.2013.09.002
[64]	Ng, G.-H. C.; McLaughlin, D.; Entekhabi, D.; Ahanin, A., The role of model dynamics in ensemble Kalman filter performance for chaotic systems, Tellus A, 63A, 958-977, 2011 · doi:10.1111/j.1600-0870.2011.00539.x
[65]	Nadiga, B.; Jiang, C.; Livescu, D., Leveraging Bayesian analysis to improve accuracy of approximate models, J. Comput. Phys., 394, 280-297, 2019 · Zbl 1453.62417 · doi:10.1016/j.jcp.2019.05.015
[66]	Gilani, F.; Giannakis, D.; Harlim, J., Kernel-based prediction of non-Markovian time series, Physica D: Nonlinear Phenomena, 418, 132829, 2021 · Zbl 1490.62289 · doi:10.1016/j.physd.2020.132829
[67]	Harlim, J.; Jiang, S. W.; Liang, S.; Yang, H., Machine learning for prediction with missing dynamics, J. Computational Physics, 428, 109922, 2021 · Zbl 07511409 · doi:10.1016/j.jcp.2020.109922

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.