Learning dynamical systems from data: a simple cross-validation perspective. IV: Case with partial observations. (English) Zbl 1541.37090
Summary: A simple and interpretable way to learn a dynamical system from data is to interpolate its governing equations with a kernel. In particular, this strategy is highly efficient (both in terms of accuracy and complexity) when the kernel is data-adapted using Kernel Flows (KF) [H. Owhadi and G. R. Yoo, J. Comput. Phys. 389, 22–47 (2019; Zbl 1452.65028)], (which uses gradient-based optimization to learn a kernel based on the premise that a kernel is good if there is no significant loss in accuracy if half of the data is used for interpolation). In this work, we extend previous work on learning dynamical systems using Kernel Flows
[B. Hamzi and H. Owhadi, Physica D 421, Article ID 132817, 10 p. (2021; Zbl 1509.68217); M. Darcy et al., Physica D 444, Article ID 133583, 18 p. (2023; Zbl 07642851); J. Lee et al., Physica D 443, Article ID 133546, 12 p. (2023; Zbl 1510.37122); B. Hamzi and H. Owhadi, Physica D 421, Article ID 132817, 10 p. (2021; Zbl 1509.68217)] to the case of learning vector-valued dynamical systems from time-series observations that are partial/incomplete in the state space. The method combines Kernel Flows with Computational Graph Completion.
For Part III see [J. Lee et al., Physica D 443, Article ID 133546, 12 p. (2023; Zbl 1510.37122)].
For Part III see [J. Lee et al., Physica D 443, Article ID 133546, 12 p. (2023; Zbl 1510.37122)].
MSC:
| 37M10 | Time series analysis of dynamical systems |
| 62M10 | Time series, auto-correlation, regression, etc. in statistics (GARCH) |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68T20 | Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.) |
Keywords:
learning dynamical systems; kernel flows; partial observations; computational graph completionReferences:
| [1] | Kantz, Holger; Schreiber, Thomas, Nonlinear Time Series Analysis (1997), Cambridge University Press: Cambridge University Press USA · Zbl 0873.62085 |
| [2] | Casdagli, Martin, Nonlinear prediction of chaotic time series, Physica D, 35, 3, 335-356 (1989) · Zbl 0671.62099 |
| [3] | Hudson, J. L.; Kube, M.; Adomaitis, R. A.; Kevrekidis, I. G.; Lapedes, A. S.; Farber, R. M., Nonlinear signal processing and system identification: applications to time series from electrochemical reactions, Chem. Eng. Sci., 45, 8, 2075-2081 (1990) |
| [4] | Rico-Martinez, Ramiro; Krischer, K.; Kevrekidis, IG; Kube, MC; Hudson, JL, Discrete-vs. continuous-time nonlinear signal processing of Cu electrodissolution data, Chem. Eng. Commun., 118, 1, 25-48 (1992) |
| [5] | Grandstrand, Ove, Nonlinear system identification using neural networks: dynamics and instabilities, (Bulsari, A. B., Neural Networks for Chemical Engineers (1995), Elsevier: Elsevier Elsevier), 409-442 |
| [6] | Gonzalez-García, R.; Rico-Martinez, R.; Kevrekidis, I. G., Identification of distributed parameter systems: A neural net based approach, Comput. Chem. Eng., 22, S965-S968 (1998), European Symposium on Computer Aided Process Engineering-8 |
| [7] | Chattopadhyay, Ashesh; Hassanzadeh, Pedram; Palem, Krishna V.; Subramanian, Devika, Data-driven prediction of a multi-scale lorenz 96 chaotic system using a hierarchy of deep learning methods: Reservoir computing, ANN, and RNN-LSTM (2019), CoRR, abs/1906.08829http://arxiv.org/abs/1906.08829 |
| [8] | Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 113, 15, 3932-3937 (2016) · Zbl 1355.94013 |
| [9] | Pathak, Jaideep; Lu, Zhixin; Hunt, Brian R.; Girvan, Michelle; Ott, Edward, Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data, Chaos, 27, 12, Article 121102 pp. (2017) · Zbl 1390.37138 |
| [10] | Nielsen, A., Practical Time Series Analysis: Prediction with Statistics and Machine Learning (2019), O’Reilly Media |
| [11] | Abarbanel, H., (Analysis of Observed Chaotic Data. Analysis of Observed Chaotic Data, Institute for Nonlinear Science (2012), Springer New York) |
| [12] | Chen, Yifan; Hosseini, Bamdad; Owhadi, Houman; Stuart, Andrew M., Solving and learning nonlinear PDEs with gaussian processes (2021), arXiv preprint arXiv:2103.12959 · Zbl 07516428 |
| [13] | Owhadi, Houman, Computational graph completion (2021), https://arxiv.org/abs/2110.10323 · Zbl 1497.68464 |
| [14] | B. Haasdonk, B. Hamzi, G. Santin, D. Wittwar, Greedy Kernel Methods for Center Manifold Approximation, in: Proc. of ICOSAHOM 2018, International Conference on Spectral and High Order Methods, Nr. 1, 2018,. |
| [15] | Haasdonk, B.; Hamzi, B.; Santin, G.; Wittwar, D., Kernel methods for center manifold approximation and a weak data-based version of the Center Manifold Theorems, Physica D (2021) · Zbl 1523.34046 |
| [16] | Giesl, P.; Hamzi, B.; Rasmussen, M.; Webster, K., Approximation of Lyapunov functions from noisy data, J. Comput. Dyn. (2019), https://arxiv.org/abs/1601.01568 |
| [17] | Bittracher, Andreas; Klus, Stefan; Hamzi, Boumediene; Schütte, Christof, Dimensionality reduction of complex metastable systems via kernel embeddings of transition manifolds, Journal of Nonlinear Science, 31 (2021) · Zbl 1466.37068 |
| [18] | Hamzi, Boumediene; Colonius, Fritz, Kernel methods for the approximation of discrete-time linear autonomous and control systems, SN Appl. Sci., 1, 7, 1-12 (2019) |
| [19] | Klus, Stefan; Nuske, Feliks; Hamzi, Boumediene, Kernel-based approximation of the koopman generator and Schrodinger operator, Entropy, 22 (2020), https://www.mdpi.com/1099-4300/22/7/722 |
| [20] | Klus, Stefan; Nüske, Feliks; Peitz, Sebastian; Niemann, Jan-Hendrik; Clementi, Cecilia; Schütte, Christof, Data-driven approximation of the koopman generator: Model reduction, system identification, and control, Physica D, 406, Article 132416 pp. (2020) · Zbl 1485.93097 |
| [21] | Alexander, Romeo; Giannakis, Dimitrios, Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques, Physica D, 409, Article 132520 pp. (2020) · Zbl 1496.37085 |
| [22] | Bittracher, Andreas; Klus, Stefan; Hamzi, Boumediene; Koltai, Peter; Schutte, Christof, Dimensionality reduction of complex metastable systems via kernel embeddings of transition manifold (2019), https://arxiv.org/abs/1904.08622 · Zbl 1466.37068 |
| [23] | Jake Bouvrie, Boumediene Hamzi, Empirical Estimators for Stochastically Forced Nonlinear Systems: Observability, Controllability and the Invariant Measure, in: Proc. of the 2012 American Control Conference, 2012, pp. 294-301,. · Zbl 1368.93248 |
| [24] | Bouvrie, Jake; Hamzi, Boumediene, Kernel methods for the approximation of nonlinear systems, SIAM J. Control Optim. (2017), https://arxiv.org/abs/1108.2903 · Zbl 1368.93248 |
| [25] | Bouvrie, Jake; Hamzi, Boumediene, Kernel methods for the approximation of some key quantities of nonlinear systems, J. Comput. Dyn., 1 (2017), http://arxiv.org/abs/1204.0563 · Zbl 1394.37115 |
| [26] | Hamzi, Boumediene; Kuehn, Christian; Mohamed, Sameh, A note on kernel methods for multiscale systems with critical transitions, Math. Methods Appl. Sci., 42, 3, 907-917 (2019) · Zbl 1410.34170 |
| [27] | Akian, J.-L.; Bonnet, Luc; Owhadi, Houman; Savin, Éric, Learning “best” kernels from data in Gaussian process regression. with application to aerodynamics, J. Comput. Phys., 470, Article 111595 pp. (2022) · Zbl 07599627 |
| [28] | Hamzi, Boumediene; Owhadi, Houman; Paillet, Léo, A note on microlocal kernel design for some slow-fast stochastic differential equations with critical transitions and application to EEG signals, Physica A, 616, Article 128583 pp. (2023) · Zbl 07679922 |
| [29] | Jake Bouvrie, Boumediene Hamzi, Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space, in: 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2010, pp. 294-301. · Zbl 1368.93248 |
| [30] | Lu, Yang; Hamzi, Boumediene; Owhadi, Houman; Xiuwen, Sun; Naiming, Xie, Learning dynamical systems from data: A simple cross-validation perspective, part v: Sparse kernel flows for 132 chaotic dynamical systems (2023) |
| [31] | Hamzi, Boumediene; Owhadi, Houman, Learning dynamical systems from data: A simple cross-validation perspective, part I: Parametric kernel flows, Physica D, 421, Article 132817 pp. (2021) · Zbl 1509.68217 |
| [32] | Santin, Gabriele; Haasdonk, Bernard, Kernel methods for surrogate modeling (2019), https://arxiv.org/abs/1907.105566 · Zbl 07445319 |
| [33] | Owhadi, Houman, Do ideas have shape? Plato’s theory of forms as the continuous limit of artificial neural networks (2020), arXiv:2008.03920 · Zbl 07642857 |
| [34] | Smirnov, Alexandre; Hamzi, Boumediene; Owhadi, Houman, Mean-field limits of trained weights in deep learning: A dynamical systems perspective, Dolomites Res. Notes Approx., 15, 3 (2022) · Zbl 1540.68229 |
| [35] | Owhadi, H.; Yoo, G. R., Kernel flows: From learning kernels from data into the abyss, J. Comput. Phys., 389, 22-47 (2019) · Zbl 1452.65028 |
| [36] | Lee, Jonghyeon; De Brouwer, Edward; Hamzi, Boumediene; Owhadi, Houman, Learning dynamical systems from data: A simple cross-validation perspective, Part III: Irregularly-sampled time series, Physica D, 443, Article 133546 pp. (2023) · Zbl 1510.37122 |
| [37] | Hamzi, Boumediene; Maulik, Romit; Owhadi, Houman, Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels, Proc. R. Soc. A, 477, 2252 (2021) · Zbl 1509.68217 |
| [38] | Yang, Lu; Hamzi, Boumediene; Kevrekides, Yannis; Owhadi, Houman; Sun, Xiuwen; Xie, Naiming, Learning dynamical systems from data: A simple cross-validation perspective, part VI: Hausdorff-metric based kernel flows to learn attractors and invariants sets (2023) |
| [39] | Darcy, Matthieu; Hamzi, Boumediene; Livieri, Giulia; Owhadi, Houman; Tavallali, Peyman, One-shot learning of stochastic differential equations with data adapted kernels, Physica D, 444, Article 133583 pp. (2023) · Zbl 07642851 |
| [40] | Prasanth, Sai; Haddad, Ziad S.; Susiluoto, Jouni; Braverman, Amy J.; Hamzi, Houman Owhadiand Boumediene; Hristova-Veleva, Svetla M.; Turk, Joseph, Kernel flows to infer the structure of convective storms from satellite passive microwave observations, (preprint (2021)) |
| [41] | Darcy, M.; Hamzi, B.; Susiluoto, J.; Braverman, A.; Owhadi, H., Learning dynamical systems from data: a simple cross-validation perspective, Part II: nonparametric kernel flows (2021), https://www.researchgate.net/publication/356818178_Learning_dynamical_systems_from_data_a_simple_cross-validation_perspective_part_II_nonparametric_kernel_flows |
| [42] | Hamzi, Boumediene; Hutter, Marcus; Owhadi, Houman, A note on learning kernels from data from an algorithmic information theoretic point of view (2023) |
| [43] | Bakarji, Joseph; Champion, Kathleen P.; Kutz, J. Nathan; Brunton, Steven L., Discovering governing equations from partial measurements with deep delay autoencoders (2022), CoRR, abs/2201.05136https://arxiv.org/abs/2201.05136 |
| [44] | Buisson-Fenet, Mona; Morgenthaler, Valery; Trimpe, Sebastian; Di Meglio, Florent, Learning dynamics from partial observations with structured neural ODEs (2022), Version: 2022 |
| [45] | Bhouri, Mohamed Aziz; Perdikaris, Paris, Gaussian processes meet neuralODEs: A Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data (2021), https://arxiv.org/abs/2103.03385 |
| [46] | Harlim, John; Jiang, Shixiao W.; Liang, Senwei; Yang, Haizhao, Machine learning for prediction with missing dynamics, J. Comput. Phys., 428, Article 109922 pp. (2021) · Zbl 07511409 |
| [47] | Saurabh Malani, Tom Bertalan, Tianqi Cui, Michael Betenbaugh, Jose L. Avalos, Ioannis G. Kevrekidis, Some of the variables, some of the times, with some things known: Identification with partial information. |
| [48] | Owhadi, Houman; Sovel, Clint, Operator-Adapted Wavelets, Fast Solvers and Numerical Homogenization: from a Game Theoretic Approach To Numerical Approximation and Algorithm Design (2019), Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press · Zbl 1477.65004 |
| [49] | Sauer, T. D., Attractor reconstruction, Scholarpedia, 1, 10, 1727 (2006), revision #91017 |
| [50] | Cucker, Felipe; Smale, Steve, On the mathematical foundations of learning, Bull. Amer. Math. Soc., 39, 1-49 (2002) · Zbl 0983.68162 |
| [51] | Aronszajn, N., Theory of reproducing kernels, Trans. Am. Math. Soc., 68, 3, 337-404 (1950) · Zbl 0037.20701 |
| [52] | min Wu, Zong; Schaback, Robert, Local error estimates for radial basis function interpolation of scattered data, IMA J. Numer. Anal., 13, 13-27 (1992) · Zbl 0762.41006 |
| [53] | Owhadi, Houman, Bayesian numerical homogenization, Multiscale Model. Simul., 13, 3, 812-828 (2015) · Zbl 1322.35002 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
