HomPINNs: homotopy physics-informed neural networks for learning multiple solutions of nonlinear elliptic differential equations. (English) Zbl 1524.65937
Summary: Physics-informed neural networks (PINNs) based machine learning is an emerging framework for solving nonlinear differential equations. However, due to the implicit regularity of neural network structure, PINNs can only find the flattest solution in most cases by minimizing the loss functions. In this paper, we combine PINNs with the homotopy continuation method, a classical numerical method to compute isolated roots of polynomial systems, and propose a new deep learning framework, named homotopy physics-informed neural networks (HomPINNs), for solving multiple solutions of nonlinear elliptic differential equations. The implementation of an HomPINN is a homotopy process that is composed of the training of a fully connected neural network, named the starting neural network, and training processes of several PINNs with different tracking parameters. The starting neural network is to approximate a starting function constructed by the trivial solutions, while other PINNs are to minimize the loss functions defined by boundary condition and homotopy functions, varying with different tracking parameters. These training processes are regraded as different steps of a homotopy process, and a PINN is initialized by the well-trained neural network of the previous step, while the first starting neural network is initialized using the default initialization method. Several numerical examples are presented to show the efficiency of our proposed HomPINNs, including reaction-diffusion equations with a heart-shaped domain.
MSC:
| 65N99 | Numerical methods for partial differential equations, boundary value problems |
| 68T07 | Artificial neural networks and deep learning |
| 35K57 | Reaction-diffusion equations |
Keywords:
machine learning; physics-informed neural networks; nonlinear elliptic differential equation; multiple solutions; homotopy method; data-driven computationSoftware:
DeepONet; DiffSharp; FPINNs; DGM; XPINNs; Adam; NumericalAlgebraicGeometry; Bertini; TensorFlow; PyTorchReferences:
| [1] | Cristini, V.; Li, X.; Lowengrub, J. S.; Wise, S. M., Nonlinear simulations of solid tumor growth using a mixture model: invasion and branching, J. Math. Biol., 58, 723-763 (2009) · Zbl 1311.92039 |
| [2] | Hao, W.; Hauenstein, J. D.; Hu, B.; Sommese, A. J., A three-dimensional steady-state tumor system, Appl. Math. Comput., 218, 2661-2669 (2011) · Zbl 1238.92019 |
| [3] | Puu, T., Nonlinear economic dynamics, (Nonlinear Economic Dynamics (1991), Springer), 1-7 |
| [4] | Petrov, L. F., Nonlinear effects in economic dynamic models, Nonlinear Anal., Theory Methods Appl., 71, e2366-e2371 (2009) · Zbl 1239.91093 |
| [5] | Hao, W.; Nepomechie, R. I.; Sommese, A. J., Completeness of solutions of Bethe’s equations, Phys. Rev. E, 88, Article 052113 pp. (2013) |
| [6] | Parand, K.; Shahini, M.; Dehghan, M., Rational Legendre pseudospectral approach for solving nonlinear differential equations of Lane-Emden type, J. Comput. Phys., 228, 8830-8840 (2009) · Zbl 1177.65100 |
| [7] | Gray, P.; Scott, S., Autocatalytic reactions in the isothermal, continuous stirred tank reactor: isolas and other forms of multistability, Chem. Eng. Sci., 38, 29-43 (1983) |
| [8] | Gray, P.; Scott, S., Autocatalytic reactions in the isothermal, continuous stirred tank reactor: oscillations and instabilities in the system a + 2b → 3b; b → c, Chem. Eng. Sci., 39, 1087-1097 (1984) |
| [9] | Gray, P.; Scott, S., Sustained oscillations and other exotic patterns of behavior in isothermal reactions, J. Phys. Chem., 89, 22-32 (1985) |
| [10] | Hou, T. Y.; Lowengrub, J. S.; Shelley, M. J., Boundary integral methods for multicomponent fluids and multiphase materials, J. Comput. Phys., 169, 302-362 (2001) · Zbl 1046.76029 |
| [11] | Pearson, J. E., Complex patterns in a simple system, Science, 261, 189-192 (1993) |
| [12] | Frisch, U.; Matarrese, S.; Mohayaee, R.; Sobolevski, A., A reconstruction of the initial conditions of the universe by optimal mass transportation, Nature, 417, 260-262 (2002) |
| [13] | Caffarelli, L. A.; Milman, M., Monge Ampere equation: applications to geometry and optimization: applications to geometry and optimization: NSF-CBMS conference on the Monge Ampère equation, applications to geometry and optimization, July 9-13, 1997, Florida Atlantic University, (vol. 226 (1999), American Mathematical Soc.) · Zbl 0903.00039 |
| [14] | Tadmor, E., A review of numerical methods for nonlinear partial differential equations, Bull. Am. Math. Soc., 49, 507-554 (2012) · Zbl 1258.65073 |
| [15] | Feng, X.; Glowinski, R.; Neilan, M., Recent developments in numerical methods for fully nonlinear second order partial differential equations, SIAM Rev., 55, 205-267 (2013) · Zbl 1270.65062 |
| [16] | Lo, W.; Chen, L.; Wang, M.; Nie, Q., A robust and efficient method for steady state patterns in reaction-diffusion systems, J. Comput. Phys., 231, 5062-5077 (2012) · Zbl 1250.35017 |
| [17] | Chang, K.-C., Variational methods for non-differentiable functionals and their applications to partial differential equations, J. Math. Anal. Appl., 80, 102-129 (1981) · Zbl 0487.49027 |
| [18] | Rabinowitz, P., Minimax Methods in Critical Point Theory with Applications to Differential Equations, Vol. 65 (1986), American Mathematical Soc. · Zbl 0609.58002 |
| [19] | Zhou, J., Solving multiple solution problems: computational methods and theory revisited, Commun. Appl. Math. Comput., 31, 1-31 (2017) · Zbl 1389.35014 |
| [20] | Li, Z.; Wang, Z.; Zhou, J., A new augmented singular transform and its partial Newton-correction method for finding more solutions, J. Sci. Comput., 71, 634-659 (2017) · Zbl 1384.65075 |
| [21] | Lin, L.; Cheng, X.; Weinan, E.; Shi, A.; Zhang, P., A numerical method for the study of nucleation of ordered phases, J. Comput. Phys., 229, 1797-1809 (2010) · Zbl 1329.82041 |
| [22] | Allgower, E.; Georg, K., Introduction to Numerical Continuation Methods, Vol. 45 (2003), SIAM · Zbl 1036.65047 |
| [23] | Cai, X.; Gropp, W.; Keyes, D.; Melvin, R.; Young, D., Parallel Newton-Krylov-Schwarz algorithms for the transonic full potential equation, SIAM J. Sci. Comput., 19, 246-265 (1998) · Zbl 0917.76035 |
| [24] | Cai, X.; Keyes, D., Nonlinearly preconditioned inexact Newton algorithms, SIAM J. Sci. Comput., 24, 183-200 (2002) · Zbl 1015.65058 |
| [25] | Wilkinson, J., Rounding Errors in Algebraic Processes (1994), Courier Corporation · Zbl 0868.65027 |
| [26] | Robinson, M.; Luo, C.; Farrell, P.; Erban, R.; Majumdar, A., From molecular to continuum modelling of bistable liquid crystal devices, Liq. Cryst., 44, 2267-2284 (2017) |
| [27] | Farrell, P.; Birkisson, A.; Funke, S., Deflation techniques for finding distinct solutions of nonlinear partial differential equations, SIAM J. Sci. Comput., 37, A2026-A2045 (2015) · Zbl 1327.65237 |
| [28] | Sommese, A. J.; Wampler, C. W., The Numerical Solution of Systems of Polynomials Arising in Engineering and Science (2005), World Scientific · Zbl 1091.65049 |
| [29] | Wampler, C. W.; Sommese, A. J., Numerical algebraic geometry and algebraic kinematics, Acta Numer., 20, 469-567 (2011) · Zbl 1254.13031 |
| [30] | Hao, W.; Hu, B.; Sommese, A. J., Numerical algebraic geometry and differential equations, (Future Vision and Trends on Shapes, Geometry and Algebra (2014), Springer), 39-53 · Zbl 07905659 |
| [31] | Wang, Y.; Hao, W.; Lin, G., Two-level spectral methods for nonlinear elliptic equations with multiple solutions, SIAM J. Sci. Comput., 40, B1180-B1205 (2018) · Zbl 1398.65107 |
| [32] | Hao, W.; Hesthaven, J.; Lin, G.; Zheng, B., A homotopy method with adaptive basis selection for computing multiple solutions of differential equations, J. Sci. Comput., 82, 1-17 (2020) · Zbl 07161476 |
| [33] | Hao, W.; Hauenstein, J. D.; Hu, B.; Sommese, A. J., A bootstrapping approach for computing multiple solutions of differential equations, J. Comput. Appl. Math., 258, 181-190 (2014) · Zbl 1294.65085 |
| [34] | Owens, A.; Filkin, D., Efficient training of the back propagation network by solving a system of stiff ordinary differential equations, (Proceedings IEEE/INNS International Joint Conference of Neural Networks (1989)), 381-386 |
| [35] | Dissanayake, M.; Phan-Thien, N., Neural-network-based approximations for solving partial differential equations, Commun. Numer. Methods Eng., 10, 195-201 (1994) · Zbl 0802.65102 |
| [36] | Lagaris, I. E.; Likas, A.; Fotiadis, D. I., Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9, 987-1000 (1998) |
| [37] | Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal approximators, Neural Netw., 2, 359-366 (1989) · Zbl 1383.92015 |
| [38] | Chen, T.; Chen, H., Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems, IEEE Trans. Neural Netw., 6, 911-917 (1995) |
| [39] | Siegel, J. W.; Xu, J., Approximation rates for neural networks with general activation functions, Neural Netw., 128, 313-321 (2020) · Zbl 1480.41007 |
| [40] | Lu, L.; Jin, P.; Pang, G.; Zhang, Z.; Karniadakis, G. E., Learning nonlinear operators via deeponet based on the universal approximation theorem of operators, Nat. Mach. Intell., 3, 218-229 (2021) |
| [41] | Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175 |
| [42] | Pang, G.; Lu, L.; Karniadakis, G. E., fPINNs: fractional physics-informed neural networks, SIAM J. Sci. Comput., 41, A2603-A2626 (2019) · Zbl 1420.35459 |
| [43] | Zhang, D.; Guo, L.; Karniadakis, G. E., Learning in modal space: solving time-dependent stochastic PDEs using physics-informed neural networks, SIAM J. Sci. Comput., 42, A639-A665 (2020) · Zbl 1440.60067 |
| [44] | Mao, Z.; Jagtap, A. D.; Karniadakis, G. E., Physics-informed neural networks for high-speed flows, Comput. Methods Appl. Mech. Eng., 360, Article 112789 pp. (2020) · Zbl 1442.76092 |
| [45] | Lou, Q.; Meng, X.; Karniadakis, G. E., Physics-informed neural networks for solving forward and inverse flow problems via the Boltzmann-BGK formulation, J. Comput. Phys., Article 110676 pp. (2021) · Zbl 07516434 |
| [46] | Shukla, K.; Jagtap, A. D.; Blackshire, J. L.; Sparkman, D.; Karniadakis, G. E., A physics-informed neural network for quantifying the microstructural properties of polycrystalline nickel using ultrasound data: a promising approach for solving inverse problems, IEEE Signal Process. Mag., 39, 68-77 (2021) |
| [47] | Jagtap, A. D.; Shin, Y.; Kawaguchi, K.; Karniadakis, G. E., Deep Kronecker neural networks: a general framework for neural networks with adaptive activation functions, Neurocomputing, 468, 165-180 (2022) |
| [48] | Shukla, K.; Jagtap, A. D.; Karniadakis, G. E., Parallel physics-informed neural networks via domain decomposition, J. Comput. Phys., 447, Article 110683 pp. (2021) · Zbl 07516435 |
| [49] | Jagtap, A. D.; Karniadakis, G. E., Extended physics-informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Commun. Comput. Phys., 28, 2002-2041 (2020) · Zbl 1542.65182 |
| [50] | Jagtap, A. D.; Kharazmi, E.; Karniadakis, G. E., Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems, Comput. Methods Appl. Mech. Eng., 365, Article 113028 pp. (2020) · Zbl 1442.92002 |
| [51] | Jagtap, A. D.; Mao, Z.; Adams, N.; Karniadakis, G. E., Physics-informed neural networks for inverse problems in supersonic flows (2022), arXiv preprint · Zbl 07561079 |
| [52] | De Ryck, T.; Jagtap, A. D.; Mishra, S., Error estimates for physics informed neural networks approximating the Navier-Stokes equations (2022), arXiv preprint |
| [53] | Sirignano, J.; Spiliopoulos, K., DGM: a deep learning algorithm for solving partial differential equations, J. Comput. Phys., 375, 1339-1364 (2018) · Zbl 1416.65394 |
| [54] | Weinan, E.; Yu, B., The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., 1, 1-12 (2018) · Zbl 1392.35306 |
| [55] | Baydin, A. G.; Pearlmutter, B. A.; Radul, A. A.; Siskind, J. M., Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res., 18 (2018) · Zbl 06982909 |
| [56] | Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; Lerer, A., Automatic Differentiation in Pytorch (2017) |
| [57] | Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M., Tensorflow: a system for large-scale machine learning, (12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16) (2016)), 265-283 |
| [58] | Forsythe, G.; Wasow, W., Finite Difference Methods for Partial Differential Equations (1960), John Wiley and Sons · Zbl 0099.11103 |
| [59] | Ciarlet, P., The Finite Element Method for Elliptic Problems (2002), SIAM · Zbl 0999.65129 |
| [60] | Shen, J.; Tang, T.; Wang, L., Spectral Methods: Algorithms, Analysis and Applications, Vol. 41 (2011), Springer Science & Business Media · Zbl 1227.65117 |
| [61] | Sommese, A.; Wampler, C., The Numerical Solution of Systems of Polynomials Arising in Engineering and Science, Vol. 99 (2005), World Scientific · Zbl 1091.65049 |
| [62] | Hao, W.; Hu, B.; Sommese, A., Numerical algebraic geometry and differential equations, (Future Vision and Trends on Shapes, Geometry and Algebra (2014), Springer), 39-53 · Zbl 07905659 |
| [63] | Bates, D.; Hauenstein, J.; Sommese, A.; Wampler, C., Numerically Solving Polynomial Systems with Bertini, Vol. 25 (2013), SIAM · Zbl 1295.65057 |
| [64] | Leykin, A., Numerical algebraic geometry, J. Softw. Algebra Geom., 3, 5-10 (2011) · Zbl 1311.14057 |
| [65] | Morgan, A.; Sommese, A.; Wampler, C., A product-decomposition bound for Bezout numbers, SIAM J. Numer. Anal., 32, 1308-1325 (1995) · Zbl 0854.65038 |
| [66] | Babolian, E.; Masouri, Z.; Hatamzadeh-Varmazyar, S., Numerical solution of nonlinear Volterra-Fredholm integro-differential equations via direct method using triangular functions, Comput. Math. Appl., 58, 239-247 (2009) · Zbl 1189.65306 |
| [67] | He, K.; Zhang, X.; Ren, S.; Sun, J., Delving deep into rectifiers: surpassing human-level performance on ImageNet classification, (2015 IEEE International Conference on Computer Vision (ICCV) (2015)), 1026-1034 |
| [68] | Kingma, D. P.; Ba Adam, J., A method for stochastic optimization (2014), arXiv preprint |
| [69] | Gu, Y.; Wang, C.; Yang, H., Structure probing neural network deflation, J. Comput. Phys., 434, Article 110231 pp. (2021) · Zbl 07508532 |
| [70] | Chen, C.; Xie, Z., Structure of multiple solutions for nonlinear differential equations, Sci. China Ser. A, Math., 47, 172-180 (2004) · Zbl 1090.35075 |
| [71] | Hao, W.; Zheng, C., An adaptive homotopy method for computing bifurcations of nonlinear parametric systems, J. Sci. Comput., 82, 1-19 (2020) · Zbl 1437.37111 |
| [72] | Li, Z.-X.; Yang, Z.-H.; Zhu, H.-L., A bifurcation method for solving multiple positive solutions to the boundary value problem of the Henon equation on a unit disk, Comput. Math. Appl., 62, 3775-3784 (2011) · Zbl 1236.65137 |
| [73] | Hao, W.; Xue, C., Spatial pattern formation in reaction-diffusion models: a computational approach, J. Math. Biol., 80, 521-543 (2020) · Zbl 1432.92009 |
| [74] | Jagtap, A. D.; Kawaguchi, K.; Karniadakis, G. E., Adaptive activation functions accelerate convergence in deep and physics-informed neural networks, J. Comput. Phys., 404, Article 109136 pp. (2020) · Zbl 1453.68165 |
| [75] | Jagtap, A. D.; Kawaguchi, K.; Em Karniadakis, G., Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks, Proc. R. Soc. A, 476, Article 20200334 pp. (2020) · Zbl 1472.68175 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
