Correcting model misspecification in physics-informed neural networks (PINNs). (English) Zbl 07842894
Summary: Data-driven discovery of governing equations in computational science has emerged as a new paradigm for obtaining accurate physical models and as a possible alternative to theoretical derivations. The recently developed physics-informed neural networks (PINNs) have also been employed to learn governing equations given data across diverse scientific disciplines, e.g., in biology and fluid dynamics. Despite the effectiveness of PINNs for discovering governing equations, the physical models encoded in PINNs may be misspecified in complex systems as some of the physical processes may not be fully understood, leading to the poor accuracy of PINN predictions. In this work, we present a general approach to correct the misspecified physical models in PINNs for discovering governing equations, given some sparse and/or noisy data. Specifically, we first encode the assumed physical models, which may be misspecified in PINNs, and then employ other deep neural networks (DNNs) to model the discrepancy between the imperfect models and the observational data. Due to the expressivity of DNNs, the proposed method is capable of reducing the computational errors caused by the model misspecification and thus enables the applications of PINNs in complex systems where the physical processes are not exactly known. Furthermore, we utilize the Bayesian physics-informed neural networks (B-PINNs) and/or ensemble PINNs to quantify uncertainties arising from noisy and/or gappy data in the discovered governing equations. A series of numerical examples including reaction-diffusion systems and non-Newtonian channel and cavity flows demonstrate that the added DNNs are capable of correcting the model misspecification in PINNs and thus reduce the discrepancy between the physical models encoded in PINNs and the observational data. In addition, the B-PINNs and ensemble PINNs can provide reasonable uncertainty bounds in the discovered physical models, which makes the predictions more reliable. We also demonstrate that we can seamlessly combine the present approach with the symbolic regression to obtain the explicit governing equations upon the training of PINNs. We envision that the proposed approach will extend the applications of PINNs for discovering governing equations in problems where the physico-chemical or biological processes are not well understood.
MSC:
| 68Txx | Artificial intelligence |
| 65Cxx | Probabilistic methods, stochastic differential equations |
| 35Rxx | Miscellaneous topics in partial differential equations |
Keywords:
model uncertainty; physics-informed neural networks; model misspecification; uncertainty quantification; non-Newtonian flows; symbolic regressionReferences:
| [1] | Zhang, J.; Ma, W., Data-driven discovery of governing equations for fluid dynamics based on molecular simulation, J. Fluid Mech., 892, A5, 2020 · Zbl 1460.76007 |
| [2] | Fyfe, J. C.; Kharin, V. V.; Santer, B. D.; Cole, J. N.; Gillett, N. P., Significant impact of forcing uncertainty in a large ensemble of climate model simulations, Proc. Natl. Acad. Sci., 118, 23, Article e2016549118 pp., 2021 |
| [3] | Schmidt, M.; Lipson, H., Distilling free-form natural laws from experimental data, Science, 324, 5923, 81-85, 2009 |
| [4] | Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 113, 15, 3932-3937, 2016 · Zbl 1355.94013 |
| [5] | Rudy, S. H.; Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Data-driven discovery of partial differential equations, Sci. Adv., 3, 4, Article e1602614 pp., 2017 |
| [6] | Chen, Z.; Liu, Y.; Sun, H., Physics-informed learning of governing equations from scarce data, Nat. Commun., 12, 1, 6136, 2021 |
| [7] | Zhang, Z.; Shin, Y.; Em Karniadakis, G., GFINNs: GENERIC formalism informed neural networks for deterministic and stochastic dynamical systems, Philos. Trans. R. Soc. A, 380, 2229, Article 20210207 pp., 2022 |
| [8] | Lee, K.; Trask, N.; Stinis, P., Structure-preserving sparse identification of nonlinear dynamics for data-driven modeling, (Mathematical and Scientific Machine Learning, PMLR, 2022), 65-80 |
| [9] | González-García, R.; Rico-Martìnez, R.; Kevrekidis, I. G., Identification of distributed parameter systems: a neural net based approach, Comput. Chem. Eng., 22, S965-S968, 1998 |
| [10] | Rico-Martinez, R.; Krischer, K.; Kevrekidis, I.; Kube, M.; Hudson, J., Discrete-vs. continuous-time nonlinear signal processing of cu electrodissolution data, Chem. Eng. Commun., 118, 1, 25-48, 1992 |
| [11] | Zhang, S.; Lin, G., Robust data-driven discovery of governing physical laws with error bars, Proc. R. Soc. A, Math. Phys. Eng. Sci., 474, 2217, Article 20180305 pp., 2018 · Zbl 1407.62267 |
| [12] | Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707, 2019 · Zbl 1415.68175 |
| [13] | Berger, L.; Marinacci, M., Model uncertainty in climate change economics: a review and proposed framework for future research, Environ. Resour. Econ., 77, 3, 475-501, 2020 |
| [14] | Ebers, M. R.; Steele, K. M.; Kutz, J. N., Discrepancy modeling framework: learning missing physics, modeling systematic residuals, and disambiguating between deterministic and random effects, 2022, arXiv preprint · Zbl 1532.68095 |
| [15] | Chen, Z.; Xiu, D., On generalized residual network for deep learning of unknown dynamical systems, J. Comput. Phys., 438, Article 110362 pp., 2021 · Zbl 07505957 |
| [16] | Zhang, Z.; Zou, Z.; Kuhl, E.; Karniadakis, G. E., Discovering a reaction-diffusion model for Alzheimer’s disease by combining PINNs with symbolic regression, Comput. Methods Appl. Mech. Eng., 419, Article 116647 pp., 2024 · Zbl 1539.92006 |
| [17] | Eastman, B.; Podina, L.; Kohandel, M., A PINN approach to symbolic differential operator discovery with sparse data, (The Symbiosis of Deep Learning and Differential Equations II, 2022) |
| [18] | Malani, S.; Bertalan, T. S.; Cui, T.; Avalos, J. L.; Betenbaugh, M.; Kevrekidis, I. G., Some of the variables, some of the parameters, some of the times, with some physics known: identification with partial information, 2023, arXiv preprint |
| [19] | Zhu, A.; Bertalan, T.; Zhu, B.; Tang, Y.; Kevrekidis, I. G., Implementation and (inverse modified) error analysis for implicitly-templated ode-nets, 2023, arXiv preprint |
| [20] | Psaros, A. F.; Meng, X.; Zou, Z.; Guo, L.; Karniadakis, G. E., Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons, J. Comput. Phys., 477, Article 111902 pp., 2023 · Zbl 07652802 |
| [21] | Zou, Z.; Meng, X.; Psaros, A. F.; Karniadakis, G. E., NeuralUQ: a comprehensive library for uncertainty quantification in neural differential equations and operators, SIAM Rev., 66, 1, 161-190, 2024 · Zbl 1535.65005 |
| [22] | Parker, W. S., Ensemble modeling, uncertainty and robust predictions, Wiley interdisciplinary reviews, Clim. Change, 4, 3, 213-223, 2013 |
| [23] | Gosling, S. N.; McGregor, G. R.; Lowe, J. A., The benefits of quantifying climate model uncertainty in climate change impacts assessment: an example with heat-related mortality change estimates, Clim. Change, 112, 217-231, 2012 |
| [24] | Yang, L.; Meng, X.; Karniadakis, G. E., B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data, J. Comput. Phys., 425, Article 109913 pp., 2021 · Zbl 07508507 |
| [25] | Lu, L.; Meng, X.; Mao, Z.; Karniadakis, G. E., DeepXDE: a deep learning library for solving differential equations, SIAM Rev., 63, 1, 208-228, 2021 · Zbl 1459.65002 |
| [26] | Cai, S.; Mao, Z.; Wang, Z.; Yin, M.; Karniadakis, G. E., Physics-informed neural networks (PINNs) for fluid mechanics: a review, Acta Mech. Sin., 37, 12, 1727-1738, 2021 |
| [27] | Pang, G.; Lu, L.; Karniadakis, G. E., fPINNs: fractional physics-informed neural networks, SIAM J. Sci. Comput., 41, 4, A2603-A2626, 2019 · Zbl 1420.35459 |
| [28] | Kharazmi, E.; Zhang, Z.; Karniadakis, G. E., hp-VPINNs: variational physics-informed neural networks with domain decomposition, Comput. Methods Appl. Mech. Eng., 374, Article 113547 pp., 2021 · Zbl 1506.68105 |
| [29] | Jagtap, A. D.; Karniadakis, G. E., Extended physics-informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, (AAAI Spring Symposium: MLPS, Vol. 10, 2021) |
| [30] | Jagtap, A. D.; Kharazmi, E.; Karniadakis, G. E., Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems, Comput. Methods Appl. Mech. Eng., 365, Article 113028 pp., 2020 · Zbl 1442.92002 |
| [31] | Zou, Z.; Karniadakis, G. E., L-HYDRA: multi-head physics-informed neural networks, 2023, arXiv preprint |
| [32] | Zou, Z.; Meng, X.; Karniadakis, G. E., Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators, 2023, arXiv preprint |
| [33] | Linka, K.; Schäfer, A.; Meng, X.; Zou, Z.; Karniadakis, G. E.; Kuhl, E., Bayesian physics informed neural networks for real-world nonlinear dynamical systems, Comput. Methods Appl. Mech. Eng., 402, Article 115346 pp., 2022 · Zbl 1507.68246 |
| [34] | Chen, P.; Meng, T.; Zou, Z.; Darbon, J.; Karniadakis, G. E., Leveraging multi-time Hamilton-Jacobi PDEs for certain scientific machine learning problems, 2023, arXiv preprint |
| [35] | Chen, P.; Meng, T.; Zou, Z.; Darbon, J.; Karniadakis, G. E., Leveraging Hamilton-Jacobi PDEs with time-dependent hamiltonians for continual scientific machine learning, 2023, arXiv preprint |
| [36] | Lin, G.; Wang, Y.; Zhang, Z., Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN, J. Comput. Phys., 460, Article 111173 pp., 2022 · Zbl 07525156 |
| [37] | Leung, W. T.; Lin, G.; Zhang NH-PINN, Z., Neural homogenization-based physics-informed neural network for multiscale problems, J. Comput. Phys., 470, Article 111539 pp., 2022 · Zbl 07599595 |
| [38] | Tang, K.; Wan, X.; Yang, C., DAS-PINNs: a deep adaptive sampling method for solving high-dimensional partial differential equations, J. Comput. Phys., 476, Article 111868 pp., 2023 · Zbl 07652779 |
| [39] | Tang, K.; Zhai, J.; Wan, X.; Yang, C., Adversarial adaptive sampling: unify PINN and optimal transport for the approximation of PDEs, 2023, arXiv preprint |
| [40] | Meng, X.; Babaee, H.; Karniadakis, G. E., Multi-fidelity Bayesian neural networks: algorithms and applications, J. Comput. Phys., 438, Article 110361 pp., 2021 · Zbl 07505956 |
| [41] | Sarabian, M.; Babaee, H.; Laksari, K., Physics-informed neural networks for brain hemodynamic predictions using medical imaging, IEEE Trans. Med. Imaging, 41, 9, 2285-2303, 2022 |
| [42] | Guo, L.; Wu, H.; Yu, X.; Zhou, T., Monte Carlo fPINNs: deep learning method for forward and inverse problems involving high dimensional fractional partial differential equations, Comput. Methods Appl. Mech. Eng., 400, Article 115523 pp., 2022 · Zbl 1507.65012 |
| [43] | Gao, Z.; Yan, L.; Zhou, T., Failure-informed adaptive sampling for PINNs, SIAM J. Sci. Comput., 45, 4, A1971-A1994, 2023 · Zbl 07729492 |
| [44] | Yin, M.; Zou, Z.; Zhang, E.; Cavinato, C.; Humphrey, J. D.; Karniadakis, G. E., A generative modeling framework for inferring families of biomechanical constitutive laws in data-sparse regimes, J. Mech. Phys. Solids, 181, Article 105424 pp., 2023 |
| [45] | Ruder, S., An overview of gradient descent optimization algorithms, 2016, arXiv preprint |
| [46] | Lakshminarayanan, B.; Pritzel, A.; Blundell, C., Simple and scalable predictive uncertainty estimation using deep ensembles, Adv. Neural Inf. Process. Syst., 30, 2017 |
| [47] | Kiyani, E.; Shukla, K.; Karniadakis, G. E.; Karttunen, M., A framework based on symbolic regression coupled with eXtended physics-informed neural networks for gray-box learning of equations of motion from data, 2023, arXiv preprint · Zbl 1539.65153 |
| [48] | Cranmer, M., Interpretable machine learning for science with PySR and SymbolicRegression.jl, 2023, arXiv preprint |
| [49] | 2022, T. M. Inc., Matlab version: 9.13.0 (r2022b) |
| [50] | Kingma, D. P.; Adam, J. Ba, A method for stochastic optimization, 2014, arXiv preprint |
| [51] | Wang, L.; Mi, J.; Meng, X.; Guo, Z., A localized mass-conserving lattice Boltzmann approach for non-Newtonian fluid flows, Commun. Comput. Phys., 17, 4, 908-924, 2015 · Zbl 1373.76272 |
| [52] | Neal, R. M., MCMC using Hamiltonian dynamics, (Handbook of Markov Chain Monte Carlo 2 (11), 2011), 2 · Zbl 1229.65018 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
