×

Weak-form latent space dynamics identification. (English) Zbl 07867404

Summary: Recent work in data-driven modeling has demonstrated that a weak formulation of model equations enhances the noise robustness of a wide range of computational methods. In this paper, we demonstrate the power of the weak form to enhance the LaSDI (Latent Space Dynamics Identification) algorithm, a recently developed data-driven reduced order modeling technique.
We introduce a weak form-based version WLaSDI (Weak-form Latent Space Dynamics Identification). WLaSDI first compresses data, then projects onto the test functions and learns the local latent space models. Notably, WLaSDI demonstrates significantly enhanced robustness to noise. With WLaSDI, the local latent space is obtained using weak-form equation learning techniques. Compared to the standard sparse identification of nonlinear dynamics (SINDy) used in LaSDI, the variance reduction of the weak form guarantees a robust and precise latent space recovery, hence allowing for a fast, robust, and accurate simulation. We demonstrate the efficacy of WLaSDI vs. LaSDI on several common benchmark examples including viscid and inviscid Burgers’, radial advection, and heat conduction. For instance, in the case of 1D inviscid Burgers’ simulations with the addition of up to 100% Gaussian white noise, the relative error remains consistently below 6% for WLaSDI, while it can exceed 10,000% for LaSDI. Similarly, for radial advection simulations, the relative errors stay below 16% for WLaSDI, in stark contrast to the potential errors of up to 10,000% with LaSDI. Moreover, speedups of several orders of magnitude can be obtained with WLaSDI. For example applying WLaSDI to 1D Burgers’ yields a 140X speedup compared to the corresponding full order model. Python code to reproduce the results in this work is available at (https://github.com/MathBioCU/PyWSINDy_ODE) and (https://github.com/MathBioCU/PyWLaSDI).

MSC:

76-XX Fluid mechanics
93-XX Systems theory; control

References:

[1] Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 113, 15, 3932-3937, 2016 · Zbl 1355.94013
[2] Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan, Data-driven discovery of partial differential equations, Sci. Adv., 3, 4, Article e1602614 pp., 2017
[3] Schaeffer, Hayden; McCalla, Scott G., Sparse model selection via integral terms, Phys. Rev. E, 96, 2, Article 023302 pp., 2017
[4] de Silva, Brian; Champion, Kathleen; Quade, Markus; Loiseau, Jean-Christophe; Kutz, J.; Brunton, Steven, PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data, JOSS, 5, 49, 2104, 2020
[5] Kaptanoglu, Alan; de Silva, Brian; Fasel, Urban; Kaheman, Kadierdan; Goldschmidt, Andy; Callaham, Jared; Delahunt, Charles; Nicolaou, Zachary; Champion, Kathleen; Loiseau, Jean-Christophe; Kutz, J.; Brunton, Steven, PySINDy: A comprehensive Python package for robust sparse system identification, JOSS, 7, 69, 3994, 2022
[6] Bertsimas, Dimitris; Gurnee, Wes, Learning sparse nonlinear dynamics via mixed-integer optimization, Nonlinear Dynam., 111, 6585-6604, 2023 · Zbl 1523.37082
[7] Brunton, Steven L.; Kutz, J. Nathan, Machine learning for partial differential equations, 2023, arXiv:2303.17078
[8] Nicolaou, Zachary G.; Huo, Guanyu; Chen, Yihui; Brunton, Steven L.; Kutz, J. Nathan, Data-driven discovery and extrapolation of parameterized pattern-forming dynamics, Phys. Rev. Res., 5, 4, L042017, 2023
[9] Kaptanoglu, Alan A.; Hansen, Christopher; Lore, Jeremy D.; Landreman, Matt; Brunton, Steven L., Sparse regression for plasma physics, Phys. Plasmas, 30, 3, Article 033906 pp., 2023
[10] Kaptanoglu, Alan A.; Zhang, Lanyue; Nicolaou, Zachary G.; Fasel, Urban; Brunton, Steven L., Benchmarking sparse system identification with low-dimensional chaos, Nonlinear Dyn., 111, 14, 13143-13164, 2023
[11] Berkooz, G.; Holmes, P.; Lumley, J. L., The proper orthogonal decomposition in the analysis of turbulent flows, Annu. Rev. Fluid Mech., 25, 1, 539-575, 1993
[12] Cheung, Siu Wun; Choi, Youngsoo; Copeland, Dylan Matthew; Huynh, Kevin, Local Lagrangian reduced-order modeling for the Rayleigh-Taylor instability by solution manifold decomposition, J. Comput. Phys., 472, Article 111655 pp., 2023 · Zbl 07620355
[13] Choi, Youngsoo; Boncoraglio, Gabriele; Anderson, Spenser; Amsallem, David; Farhat, Charbel, Gradient-based constrained optimization using a database of linear reduced-order models, J. Comput. Phys., 423, Article 109787 pp., 2020 · Zbl 07508406
[14] Choi, Youngsoo; Brown, Peter; Arrighi, William; Anderson, Robert; Huynh, Kevin, Space-time reduced order model for large-scale linear dynamical systems with application to Boltzmann transport problems, J. Comput. Phys., 424, Article 109845 pp., 2021 · Zbl 07508450
[15] Choi, Youngsoo; Carlberg, Kevin, Space-Time least-squares Petrov-Galerkin projection for nonlinear model reduction, SIAM J. Sci. Comput., 41, 1, A26-A58, 2019 · Zbl 1405.65140
[16] Choi, Youngsoo; Coombs, Deshawn; Anderson, Robert, SNS: A solution-based nonlinear subspace method for time-dependent model order reduction, SIAM J. Sci. Comput., 42, 2, A1116-A1146, 2020 · Zbl 1442.37111
[17] Choi, Youngsoo; Oxberry, Geoffrey; White, Daniel; Kirchdoerfer, Trenton, Accelerating design optimization using reduced order models, 2019
[18] Rozza, G.; Huynh, D. B.P.; Patera, A. T., Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations: Application to transport and continuum mechanics, Arch. Comput. Methods Eng., 15, 3, 229-275, 2008 · Zbl 1304.65251
[19] Safonov, M. G.; Chiang, R. Y., A Schur method for balanced-truncation model reduction, IEEE Trans. Automat. Control, 34, 7, 729-733, 1989 · Zbl 0687.93027
[20] Kim, Youngkyu; Choi, Youngsoo; Widemann, David; Zohdi, Tarek, Efficient nonlinear manifold reduced order model, 2020
[21] Kim, Youngkyu; Choi, Youngsoo; Widemann, David; Zohdi, Tarek, A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder, J. Comput. Phys., 451, Article 110841 pp., 2022 · Zbl 07517153
[22] Lee, Kookjin; Carlberg, Kevin T., Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders, J. Comput. Phys., 404, Article 108973 pp., 2020 · Zbl 1454.65184
[23] Maulik, Romit; Lusch, Bethany; Balaprakash, Prasanna, Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders, Phys. Fluids, 33, 3, Article 037106 pp., 2021
[24] Tapia, Gustavo; Khairallah, Saad; Matthews, Manyalibo; King, Wayne E.; Elwany, Alaa, Gaussian process-based surrogate modeling framework for process planning in laser powder-bed fusion additive manufacturing of 316L stainless steel, Int. J. Adv. Manuf. Technol., 94, 9-12, 3591-3603, 2018
[25] Qian, Zhiguang; Seepersad, Carolyn Conner; Joseph, V. Roshan; Allen, Janet K.; Jeff Wu, C. F., Building surrogate models based on detailed and approximate simulations, J. Mech. Des., 128, 4, 668-677, 2006
[26] Daniel Marjavaara, B.; Staffan Lundström, T.; Goel, Tushar; Mack, Yolanda; Shyy, Wei, Hydraulic turbine diffuser shape optimization by multiple surrogate model approximations of Pareto fronts, J. Fluids Eng., 129, 9, 1228-1240, 2007
[27] Fuxin Huang, Lijue Wang, Chi Yang, Hull form optimization for reduced drag and improved seakeeping using a surrogate-based method, in: The Twenty-Fifth International Ocean and Polar Engineering Conference, 2015, pp. ISOPE-I-15-846.
[28] Han, Zhong-Hua; Görtz, Stefan, Hierarchical Kriging model for variable-fidelity surrogate modeling, AIAA J., 50, 9, 1885-1896, 2012
[29] Han, Zhong-Hua; Görtz, Stefan; Zimmermann, Ralf, Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function, Aerosp. Sci. Technol., 25, 1, 177-189, 2013
[30] Guo, Xiaoxiao; Li, Wei; Iorio, Francesco, Convolutional neural networks for steady flow approximation, (Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., 2016, ACM: ACM San Francisco California USA), 481-490
[31] Zhang, Yao; Sung, Woong Je; Mavris, Dimitri N., Application of convolutional neural network to predict airfoil lift coefficient, (2018 AIAAASCEAHSASC Struct. Struct. Dyn. Mater. Conf., 2018, American Institute of Aeronautics and Astronautics: American Institute of Aeronautics and Astronautics Kissimmee, Florida)
[32] Kim, Byungsoo; Azevedo, Vinicius C.; Thuerey, Nils; Kim, Theodore; Gross, Markus; Solenthaler, Barbara, Deep fluids: A generative network for parameterized fluid simulations, Comput. Graph. Forum, 38, 2, 59-70, 2019
[33] Kadeethum, T.; Ballarin, F.; Choi, Y.; O’Malley, D.; Yoon, H.; Bouklas, N., Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: Comparison with linear subspace techniques, Adv. Water Resour., 160, Article 104098 pp., 2022
[34] Fresca, Stefania; Dede’, Luca; Manzoni, Andrea, A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs, J. Sci. Comput., 87, 2, 61, 2021 · Zbl 1470.65166
[35] Xie, Xuping; Zhang, Guannan; Webster, Clayton G., Non-intrusive inference reduced order model for fluids using deep multistep neural network, Mathematics, 7, 8, 757, 2019
[36] Champion, Kathleen; Lusch, Bethany; Kutz, J. Nathan; Brunton, Steven L., Data-driven discovery of coordinates and governing equations, Proc. Natl. Acad. Sci., 116, 45, 22445-22451, 2019 · Zbl 1433.68396
[37] Fries, William D.; He, Xiaolong; Choi, Youngsoo, LaSDI: Parametric latent space dynamics identification, Comput. Methods Appl. Mech. Engrg., 399, Article 115436 pp., 2022 · Zbl 1507.65078
[38] He, Xiaolong; Choi, Youngsoo; Fries, William D.; Belof, Jon; Chen, Jiun-Shyan, gLaSDI: Parametric physics-informed greedy latent space dynamics identification, 2022
[39] He, Xiaolong; Choi, Youngsoo; Fries, William D.; Belof, Jonathan L.; Chen, Jiun-Shyan, Certified data-driven physics-informed greedy auto-encoder simulator, 2022
[40] Bonneville, Christophe; Choi, Youngsoo; Ghosh, Debojyoti; Belof, Jonathan L., GPLaSDI: Gaussian process-based interpretable latent space dynamics identification through deep autoencoder, Comput. Methods Appl. Mech. Engrg., 418, Article 116535 pp., 2024 · Zbl 1539.65085
[41] Bortz, David M.; Messenger, Daniel A.; Dukic, Vanja, Direct estimation of parameters in ODE Models Using WENDy: weak-form estimation of nonlinear dynamics, Bull. Math. Biol., 85, 110, 2023 · Zbl 1526.92048
[42] Schwartz, Laurent, Théorie Des Distributions, 1950, Hermann et Cie: Hermann et Cie Paris, France
[43] Lax, P. D.; Milgram, A. N., IX. Parabolic equations, (Contributions To the Theory of Partial Differential Equations. Contributions To the Theory of Partial Differential Equations, Annals of Mathematical Studies, vol. 33, 1955, Princeton University Press), 167-190 · Zbl 0058.08703
[44] Messenger, Daniel A.; Bortz, David M., Weak SINDy: Galerkin-based data-driven model selection, Multiscale Model. Simul., 19, 3, 1474-1497, 2021 · Zbl 1512.65163
[45] Messenger, Daniel A.; Bortz, David M., Weak SINDy for partial differential equations, J. Comput. Phys., 443, Article 110525 pp., 2021 · Zbl 07515424
[46] Messenger, Daniel A.; Dall’Anese, Emiliano; Bortz, David M., Online weak-form sparse identification of partial differential equations, (Proc. Third Math. Sci. Mach. Learn. Conf.. Proc. Third Math. Sci. Mach. Learn. Conf., Proceedings of Machine Learning Research, vol. 190, 2022, PMLR), 241-256
[47] Messenger, Daniel A.; Bortz, David M., Learning mean-field equations from particle data using WSINDy, Physica D, 439, Article 133406 pp., 2022 · Zbl 1498.93111
[48] Messenger, Daniel A.; Wheeler, Graycen E.; Liu, Xuedong; Bortz, David M., Learning anisotropic interaction rules from individual trajectories in a heterogeneous cellular population, J. R. Soc. Interface, 19, 195, Article 20220412 pp., 2022
[49] Messenger, Daniel A.; Burby, Joshua W.; Bortz, David M., Coarse-graining hamiltonian systems using SINDy, 2024, arXiv:2310.05879, Scientific Reports, accepted
[50] Messenger, Daniel A.; Bortz, David M., Asymptotic consistency of the WSINDy algorithm in the limit of continuum data, 2022, arXiv:2211.16000, submitted for publication
[51] Total Least Squares and Errors-in-Variables Modeling: Analysis, Algorithms and Applications, 2002, Springer Netherlands: Springer Netherlands Dordrecht
[52] Jorgensen, Murray, Iteratively reweighted least squares, (El-Shaarawi, Abdel H.; Piegorsch, Walter W., Encyclopedia of Environmetrics, 2012, Wiley)
[53] Xiao, D.; Fang, F.; Pain, C. C.; Navon, I. M., A parameterized non-intrusive reduced order model and error analysis for general time-dependent nonlinear partial differential equations and its applications, Comput. Methods Appl. Mech. Engrg., 317, 868-889, 2017 · Zbl 1439.65124
[54] He, Xiaolong; Choi, Youngsoo; Fries, William D.; Belof, Jonathan L.; Chen, Jiun-Shyan, gLaSDI: Parametric physics-informed greedy latent space dynamics identification, J. Comput. Phys., 489, Article 112267 pp., 2023 · Zbl 07705908
[55] LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey, Deep learning, nature, 521, 7553, 436-444, 2015
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.