Derivative-based SINDy (DSINDy): addressing the challenge of discovering governing equations from noisy data. (English) Zbl 1539.62229
Summary: Recent advances in the field of data-driven dynamics allow for the discovery of ODE systems using state measurements. One approach, known as Sparse Identification of Nonlinear Dynamics (SINDy), assumes the dynamics are sparse within a predetermined basis in the states and finds the expansion coefficients through linear regression with sparsity constraints. This approach requires an accurate estimation of the state time derivatives, which is not necessarily possible in the high-noise regime without additional constraints. We present an approach called Derivative-based SINDy (DSINDy) that combines two novel methods to improve ODE recovery at high-noise levels. First, we denoise the state variables by applying a projection operator that leverages the assumed basis for the system dynamics. Second, we use a second order cone program (SOCP) to find the derivative and governing equations simultaneously. We derive theoretical results for the projection-based denoising step, which allow us to estimate the values of hyperparameters used in the SOCP formulation. This underlying theory helps limit the number of required user-specified parameters. We present results demonstrating that our approach leads to improved system recovery for the Van der Pol oscillator, the Duffing oscillator, the Rössler attractor, and the Lorenz 96 model.
MSC:
| 62J07 | Ridge regression; shrinkage estimators (Lasso) |
| 37M10 | Time series analysis of dynamical systems |
| 65D10 | Numerical smoothing, curve fitting |
| 90C25 | Convex programming |
References:
| [1] | Ghadami, A.; Epureanu, B. I., Data-driven prediction in dynamical systems: recent developments, Phil. Trans. R. Soc. A, 380, 2229 (2022) |
| [2] | Neumann, P.; Cao, L.; Russo, D.; Vassiliadis, V. S.; Lapkin, A. A., A new formulation for symbolic regression to identify physico-chemical laws from experimental data, Chem. Eng. J., 387, Article 123412 pp. (2020) |
| [3] | Mangan, N. M.; Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Inferring biological networks by sparse identification of nonlinear dynamics, IEEE Trans. Mol. Biol. Multi-Scale Commun., 2, 1, 52-63 (2016) |
| [4] | Dam, M.; Brøns, M.; Juul Rasmussen, J.; Naulin, V.; Hesthaven, J. S., Sparse identification of a predator-prey system from simulation data of a convection model, Phys. Plasmas, 24, 2, Article 022310 pp. (2017) |
| [5] | Raissi, M.; Yazdani, A.; Karniadakis, G. E., Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations, Science, 367, 6481, 1026-1030 (2020) · Zbl 1478.76057 |
| [6] | Bongard, J.; Lipson, H., Automated reverse engineering of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 104, 24, 9943-9948 (2007) · Zbl 1155.37044 |
| [7] | Quade, M.; Abel, M.; Shafi, K.; Niven, R. K.; Noack, B. R., Prediction of dynamical systems by symbolic regression, Phys. Rev. E, 94, 1, 12214 (2016) |
| [8] | Raissi, M., Deep hidden physics models: Deep learning of nonlinear partial differential equations, J. Mach. Learn. Res., 19, 1-24 (2018) · Zbl 1439.68021 |
| [9] | Schmidt, M.; Lipson, H., Distilling free-form natural laws from experimental data, Science, 324, 5923, 81-85 (2009) |
| [10] | Wang, W.-X.; Yang, R.; Lai, Y.-C.; Kovanis, V.; Grebogi, C., Predicting catastrophes in nonlinear dynamical systems by compressive sensing, Phys. Rev. Lett., 106, 15, Article 154101 pp. (2011) |
| [11] | Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 113, 15, 3932-3937 (2016) · Zbl 1355.94013 |
| [12] | Cortiella, A.; Park, K. C.; Doostan, A., A priori denoising strategies for sparse identification of nonlinear dynamical systems: A comparative study, J. Comput. Inf. Sci. Eng., 23, 1, 1-34 (2022) |
| [13] | Delahunt, C. B.; Kutz, J. N., A toolkit for data-driven discovery of governing equations in high-noise regimes, IEEE Access, 10, 31210-31234 (2022) |
| [14] | Schaeffer, H.; McCalla, S. G., Sparse model selection via integral terms, Phys. Rev. E, 96, 2, 023302, 7 (2017) |
| [15] | Chen, Z.; Liu, Y.; Sun, H., Physics-informed learning of governing equations from scarce data, Nature Commun., 12, 1, 6136 (2021) |
| [16] | Tran, G.; Ward, R., Exact recovery of chaotic systems from highly corrupted data, Multiscale Model. Simul., 15, 3, 1108-1129 (2017) |
| [17] | F. Sun, Y. Liu, H. Sun, Physics-informed spline learning for nonlinear dynamics discovery, in: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, (ISSN: 1045-0823) 2021, pp. 2054-2061. |
| [18] | Hokanson, J. M.; Iaccarino, G.; Doostan, A., Simultaneous identification and denoising of dynamical systems, SISC (2023), in press · Zbl 1520.34017 |
| [19] | Kaheman, K.; Brunton, S. L.; Kutz, J. N., Automatic differentiation to simultaneously identify nonlinear dynamics and extract noise probability distributions from data, Mach. Learn.: Sci. Technol., 3, 1, Article 015031 pp. (2022) |
| [20] | Wang, Z.; Huan, X.; Garikipati, K., Variational system identification of the partial differential equations governing the physics of pattern-formation: Inference under varying fidelity and noise, Comput. Methods Appl. Mech. Engrg., 356, 44-74 (2019) · Zbl 1441.65071 |
| [21] | Gurevich, D. R.; Reinbold, P. A.K.; Grigoriev, R. O., Robust and optimal sparse regression for nonlinear PDE models, Chaos, 29, 10, Article 103113 pp. (2019) · Zbl 1433.35387 |
| [22] | Reinbold, P. A.; Gurevich, D. R.; Grigoriev, R. O., Using noisy or incomplete data to discover models of spatiotemporal dynamics, Phys. Rev. E, 101, 1 (2020) |
| [23] | Messenger, D. A.; Bortz, D. M., Weak SINDy: Galerkin-based data-driven model selection, Multiscale Model. Simul., 19, 3, 1474-1497 (2021) · Zbl 1512.65163 |
| [24] | Messenger, D. A.; Bortz, D. M., Weak SINDy for partial differential equations, J. Comput. Phys., 443, Article 110525 pp. (2021) · Zbl 07515424 |
| [25] | Cortiella, A.; Park, K. C.; Doostan, A., Sparse identification of nonlinear dynamical systems via reweighted \(\ell 1\)-regularized least squares, Comput. Methods Appl. Mech. Engrg., 376, Article 113620 pp. (2021) · Zbl 1506.37104 |
| [26] | Stewart, G. W., On the perturbation of pseudo-inverses, projections and linear least squares problems, SIAM Rev., 19, 4, 634-662 (1977) · Zbl 0379.65021 |
| [27] | Schulz, E.; Speekenbrink, M.; Krause, A., A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions, J. Math. Psych., 85, 1-16 (2018) · Zbl 1416.62648 |
| [28] | Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.; Brucher, M.; Perrot, M.; Duchesnay, É., Scikit-learn: Machine learning in Python, J. Mach. Learn. Res., 12, 2825-2830 (2011) · Zbl 1280.68189 |
| [29] | Anderson, M.; Dahl, J.; Vandenberghe, L., CVXOPT: Python software for convex optimization, version 1.1 (2015) |
| [30] | Mosek ApS, M., MOSEK optimizer API (2019) |
| [31] | Niven, R. K.; Mohammad-Djafari, A.; Cordier, L.; Abel, M.; Quade, M., Bayesian identification of dynamical systems, Proceedings, 33, 1, 33 (2020) |
| [32] | Galioto, N.; Gorodetsky, A. A., Bayesian system ID: optimal management of parameter, model, and measurement uncertainty, Nonlinear Dynam., 102, 1, 241-267 (2020) · Zbl 1517.93021 |
| [33] | Hirsh, S. M.; Barajas-Solano, D. A.; Kutz, J. N., Sparsifying priors for Bayesian uncertainty quantification in model discovery, Royal Soc. Open Sci., 9, 2 (2022) |
| [34] | Fasel, U.; Kutz, J. N.; Brunton, B. W.; Brunton, S. L., Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control, 478 (2022) |
| [35] | Abdullah, F.; Alhajeri, M. S.; Christofides, P. D., Modeling and control of nonlinear processes using sparse identification: Using dropout to handle noisy data, 61, 17976-17992 (2022) |
| [36] | Zou, H., The adaptive Lasso and its oracle properties, J. Amer. Statist. Assoc., 101, 476, 1418-1429 (2006) · Zbl 1171.62326 |
| [37] | Candès, E. J.; Wakin, M. B.; Boyd, S. P., Enhancing sparsity by reweighted \(\ell 1\) minimization, J. Fourier Anal. Appl., 14, 5, 877-905 (2008) · Zbl 1176.94014 |
| [38] | Tibshirani, R., Regression shrinkage and selection via the Lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 58, 1, 267-288 (1996) · Zbl 0850.62538 |
| [39] | Pati, Y.; Rezaiifar, R.; Krishnaprasad, P., Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition, (Proceedings of 27th Asilomar Conference on Signals, Systems and Computers, vol. 1 (1993)), 40-44, ISSN: 1058-6393 |
| [40] | Chen, S. S.; Donoho, D. L.; Saunders, M. A., Atomic decomposition by basis pursuit, SIAM Rev., 43, 1, 129-159 (2001) · Zbl 0979.94010 |
| [41] | Hansen, P. C., Analysis of discrete ill-posed problems by means of the L-curve, SIAM Rev., 34, 4, 561-580 (1992) · Zbl 0770.65026 |
| [42] | van den Berg, E.; Friedlander, M. P., Probing the Pareto frontier for basis pursuit solutions, SIAM J. Sci. Comput., 31, 2, 890-912 (2009) · Zbl 1193.49033 |
| [43] | Singh, H. K.; Isaacs, A.; Ray, T., A Pareto corner search evolutionary algorithm and dimensionality reduction in many-objective optimization problems, IEEE Trans. Evol. Comput., 15, 4, 539-556 (2011) |
| [44] | Cultrera, A.; Callegaro, L., A simple algorithm to find the L-curve corner in the regularisation of ill-posed inverse problems, IOP SciNotes, 1, 2, Article 025004 pp. (2020) |
| [45] | Cuate, O.; Schütze, O., Pareto explorer for finding the knee for many objective optimization problems, Mathematics, 8, 10, 1651 (2020) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
