Thermodynamically consistent physics-informed neural networks for hyperbolic systems. (English) Zbl 07524762
Summary: Physics-informed neural network architectures have emerged as a powerful tool for developing flexible PDE solvers that easily assimilate data. When applied to problems in shock physics however, these approaches face challenges related to the collocation-based PDE discretization underpinning them. By instead adopting a least squares space-time control volume scheme, we obtain a scheme which more naturally handles: regularity requirements, imposition of boundary conditions, entropy compatibility, and conservation, substantially reducing requisite hyperparameters in the process. Additionally, connections to classical finite volume methods allows application of inductive biases toward entropy solutions and total variation diminishing properties. For inverse problems in shock hydrodynamics, we propose inductive biases for discovering thermodynamically consistent equations of state that guarantee hyperbolicity. This framework therefore provides a means of discovering continuum shock models from molecular simulations of rarefied gases and metals. The output of the learning process provides a data-driven equation of state which may be incorporated into traditional shock hydrodynamics codes.
MSC:
| 35Lxx | Hyperbolic equations and hyperbolic systems |
| 65Mxx | Numerical methods for partial differential equations, initial value and time-dependent initial-boundary value problems |
| 68Txx | Artificial intelligence |
Keywords:
physics-informed neural networks; inverse problems; machine learning; equation of state; conservation laws; shock hydrodynamicsReferences:
| [1] | Weinan, E.; Yu, B., The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., 6, 1, 1-12 (2018) · Zbl 1392.35306 |
| [2] | He, J.; Li, L.; Xu, J.; Zheng, C., ReLU deep neural networks and linear finite elements (2018), arXiv preprint |
| [3] | Daubechies, I.; DeVore, R.; Foucart, S.; Hanin, B.; Petrova, G., Nonlinear approximation and (deep) ReLU networks (2019), arXiv preprint · Zbl 1501.41003 |
| [4] | Yarotsky, D., Error bounds for approximations with deep ReLU networks, Neural Netw., 94, 103-114 (2017) · Zbl 1429.68260 |
| [5] | Yarotsky, D., Optimal approximation of continuous functions by very deep relu networks (2018), arXiv preprint |
| [6] | Opschoor, J. A.; Petersen, P.; Schwab, C., Deep ReLU Networks and High-Order Finite Element Methods (2019), SAM, ETH Zürich · Zbl 1452.65354 |
| [7] | Bach, F., Breaking the curse of dimensionality with convex neural networks, J. Mach. Learn. Res., 18, 1, 629-681 (2017) · Zbl 1433.68390 |
| [8] | Bengio, S.; Bengio, Y., Taking on the curse of dimensionality in joint distributions using neural networks, IEEE Trans. Neural Netw., 11, 3, 550-557 (2000) |
| [9] | Han, J.; Jentzen, A.; Weinan, E., Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci., 115, 34, 8505-8510 (2018) · Zbl 1416.35137 |
| [10] | Wang, S.; Teng, Y.; Perdikaris, P., Understanding and mitigating gradient pathologies in physics-informed neural networks (2020), arXiv preprint |
| [11] | Beck, C.; Jentzen, A.; Kuckuck, B., Full error analysis for the training of deep neural networks (2019), arXiv preprint |
| [12] | Fokina, D.; Oseledets, I., Growing axons: greedy learning of neural networks with application to function approximation (2019), arXiv preprint |
| [13] | Adcock, B.; Dexter, N., The gap between theory and practice in function approximation with deep neural networks (2020), arXiv preprint |
| [14] | Lagaris, I. E.; Likas, A.; Fotiadis, D. I., Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9, 5, 987-1000 (1998) |
| [15] | Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175 |
| [16] | Raissi, M., Deep hidden physics models: deep learning of nonlinear partial differential equations, J. Mach. Learn. Res., 19, 1, 932-955 (2018) · Zbl 1439.68021 |
| [17] | Sun, L.; Gao, H.; Pan, S.; Wang, J.-X., Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data, Comput. Methods Appl. Mech. Eng., 361, Article 112732 pp. (2020) · Zbl 1442.76096 |
| [18] | Zhang, D.; Lu, L.; Guo, L.; Karniadakis, G. E., Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems, J. Comput. Phys., 397, Article 108850 pp. (2019) · Zbl 1454.65008 |
| [19] | Meng, X.; Karniadakis, G. E., A composite neural network that learns from multi-fidelity data: application to function approximation and inverse pde problems, J. Comput. Phys., 401, Article 109020 pp. (2020) · Zbl 1454.76006 |
| [20] | Mao, Z.; Jagtap, A. D.; Karniadakis, G. E., Physics-informed neural networks for high-speed flows, Comput. Methods Appl. Mech. Eng., 360, Article 112789 pp. (2020) · Zbl 1442.76092 |
| [21] | Zhang, D.; Guo, L.; Karniadakis, G. E., Learning in modal space: solving time-dependent stochastic PDEs using physics-informed neural networks, SIAM J. Sci. Comput., 42, 2, A639-A665 (2020) · Zbl 1440.60067 |
| [22] | Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G. S.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Goodfellow, I.; Harp, A.; Irving, G.; Isard, M.; Jia, Y.; Jozefowicz, R.; Kaiser, L.; Kudlur, M.; Levenberg, J.; Mané, D.; Monga, R.; Moore, S.; Murray, D.; Olah, C.; Schuster, M.; Shlens, J.; Steiner, B.; Sutskever, I.; Talwar, K.; Tucker, P.; Vanhoucke, V.; Vasudevan, V.; Viégas, F.; Vinyals, O.; Warden, P.; Wattenberg, M.; Wicke, M.; Yu, Y.; Zheng, X., TensorFlow: large-scale machine learning on heterogeneous systems (2015), software available from |
| [23] | Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; Desmaison, A.; Kopf, A.; Yang, E.; DeVito, Z.; Raison, M.; Tejani, A.; Chilamkurthy, S.; Steiner, B.; Fang, L.; Bai, J.; Chintala, S., PyTorch: an imperative style, high-performance deep learning library, (Wallach, H.; Larochelle, H.; Beygelzimer, A.; d’ Alché-Buc, F.; Fox, E.; Garnett, R., Advances in Neural Information Processing Systems, vol. 32 (2019), Curran Associates, Inc.), 8024-8035 |
| [24] | Robinson, A.; Berry, R.; Carpenter, J.; Debusschere, B.; Drake, R. R.; Mattsson, A.; Rider, W. J., Fundamental issues in the representation and propagation of uncertain equation of state information in shock hydrodynamics, Comput. Fluids, 83, 187-193 (2013) · Zbl 1290.76058 |
| [25] | Carpenter, J. H.; Robinson, A. C.; Debusschere, B.; Wills, A. E., Automated generation of tabular equations of state with uncertainty information (2015), Sandia National Lab. (SNL-NM): Sandia National Lab. (SNL-NM) Albuquerque, NM (United States), Sandia…, Tech. Rep. |
| [26] | Jagtap, A. D.; Kharazmi, E.; Karniadakis, G. E., Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems, Comput. Methods Appl. Mech. Eng., 365, Article 113028 pp. (2020) · Zbl 1442.92002 |
| [27] | Lax, P. D., Hyperbolic Systems of Conservation Laws and the Mathematical Theory of Shock Waves (1973), SIAM · Zbl 0268.35062 |
| [28] | Menikoff, R.; Plohr, B. J., The Riemann problem for fluid flow of real materials, Rev. Mod. Phys., 61, 1, 75-130 (1989) · Zbl 1129.35439 |
| [29] | Guermond, J.-L.; Popov, B., Viscous regularization of the Euler equations and entropy principles, SIAM J. Appl. Math., 74, 2, 284-305 (2014) · Zbl 1446.76147 |
| [30] | Xiong, S.; He, X.; Tong, Y.; Liu, R.; Zhu, B., Roenets: predicting discontinuity of hyperbolic systems from continuous data (2020), arXiv preprint |
| [31] | Tokareva, S.; Shashkov, M. J.; Skurikhin, A. N., Machine learning approach for the solution of the Riemann problem in fluid dynamics (2019), Los Alamos National Lab. (LANL): Los Alamos National Lab. (LANL) Los Alamos, NM (United States), Tech. Rep. |
| [32] | Magiera, J.; Ray, D.; Hesthaven, J. S.; Rohde, C., Constraint-aware neural networks for Riemann problems, J. Comput. Phys., 409, Article 109345 pp. (2020) · Zbl 1435.76046 |
| [33] | Kharazmi, E.; Zhang, Z.; Karniadakis, G. E., hp-VPINNs: variational physics-informed neural networks with domain decomposition (2020), arXiv preprint · Zbl 1506.68105 |
| [34] | Jagtap, A. D.; Karniadakis, G. E., Extended physics-informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Commun. Comput. Phys., 28, 2002-2041 (2020) · Zbl 1542.65182 |
| [35] | Dafermos, C. M., Hyperbolic Conservation Laws in Continuum Physics, Grundlehren der Mathematischen Wissenschaften, vol. 325 (2000), Springer-Verlag: Springer-Verlag Berlin · Zbl 0940.35002 |
| [36] | Bianchini, S.; Bressan, A., Vanishing viscosity solutions of nonlinear hyperbolic systems, Ann. Math. (2), 161, 1, 223-342 (2005) · Zbl 1082.35095 |
| [37] | Godlewski, E.; Raviart, P.-A., Numerical Approximation of Hyperbolic Systems of Conservation Laws, Applied Mathematical Sciences, vol. 118 (1996), Springer-Verlag: Springer-Verlag New York · Zbl 1063.65080 |
| [38] | Godlewski, E.; Raviart, P.-A., Numerical Approximation of Hyperbolic Systems of Conservation Laws, vol. 118 (1996), Springer-Verlag · Zbl 0860.65075 |
| [39] | Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M., Tensorflow: a system for large-scale machine learning, (12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16) (2016)), 265-283 |
| [40] | Moritz, H., Least-squares collocation, Rev. Geophys., 16, 3, 421-430 (1978) |
| [41] | Rummel, R.; Schwarz, K.-P.; Gerstl, M., Least squares collocation and regularization, Bull. Géod., 53, 4, 343-361 (1979) |
| [42] | Ling, L.; Kansa, E. J., A least-squares preconditioner for radial basis functions collocation methods, Adv. Comput. Math., 23, 1-2, 31-54 (2005) · Zbl 1067.65136 |
| [43] | Hu, H.; Chen, J.; Hu, W., Weighted radial basis collocation method for boundary value problems, Int. J. Numer. Methods Eng., 69, 13, 2736-2757 (2007) · Zbl 1194.74525 |
| [44] | Zhang, X.; Liu, X.-H.; Song, K.-Z.; Lu, M.-W., Least-squares collocation meshless method, Int. J. Numer. Methods Eng., 51, 9, 1089-1100 (2001) · Zbl 1056.74064 |
| [45] | Cheng, H.; Sandu, A., Collocation least-squares polynomial chaos method, (Proceedings of the 2010 Spring Simulation Multiconference (2010)), 1-6 |
| [46] | Bochev, P. B.; Gunzburger, M. D., Least-Squares Finite Element Methods, Applied Mathematical Sciences, vol. 166 (2009), Springer: Springer New York · Zbl 1168.65067 |
| [47] | Bochev, P. B.; Gunzburger, M. D., Least-squares methods for hyperbolic problems, (Handbook of Numerical Analysis Handbook of Numerical Methods for Hyperbolic Problems - Basic and Fundamental Issues (2016)), 289-317 · Zbl 1352.65001 |
| [48] | Guermond, J.-L.; Marpeau, F.; Popov, B., A fast algorithm for solving first-order PDEs by \(L^1\)-minimization, Commun. Math. Sci., 6, 1, 199-216 (2008) · Zbl 1143.65060 |
| [49] | Guermond, J.-L.; Popov, B., \( L^1\)-minimization methods for Hamilton-Jacobi equations: the one-dimensional case, Numer. Math., 109, 2, 269-284 (2008) · Zbl 1143.65061 |
| [50] | Reisner, J.; Serencsa, J.; Shkoller, S., A space-time smooth artificial viscosity method for nonlinear conservation laws, J. Comput. Phys., 235, 912-933 (2013) |
| [51] | Rauch, J., BV estimates fail for most quasilinear hyperbolic systems in dimensions greater than one, Commun. Math. Phys., 106, 3, 481-484 (1986) · Zbl 0619.35073 |
| [52] | Toro, E. F.; Billett, S. J., Centred TVD schemes for hyperbolic conservation laws, IMA J. Numer. Anal., 20, 1, 47-79 (2000) · Zbl 0943.65100 |
| [53] | Kaiser, E.; Kutz, J. N.; Brunton, S. L., Sparse identification of nonlinear dynamics for model predictive control in the low-data limit, Proc. R. Soc. A, 474, 2219, Article 20180335 pp. (2018) · Zbl 1425.93175 |
| [54] | Lu, L.; Jin, P.; Karniadakis, G. E., Deeponet: learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators (2019), arXiv preprint |
| [55] | Harten, A.; Lax, P. D.; Levermore, C. D.; Morokoff, W. J., Convex entropies and hyperbolicity for general Euler equations, SIAM J. Numer. Anal., 35, 6, 2117-2127 (1998) · Zbl 0922.35089 |
| [56] | Kivva, S., Entropy stable flux correction for scalar hyperbolic conservation laws (2020), arXiv preprint · Zbl 1486.65110 |
| [57] | LeVeque, R. J., Finite Volume Methods for Hyperbolic Problems (2002), Cambridge University Press · Zbl 1010.65040 |
| [58] | Cyr, E. C.; Gulian, M. A.; Patel, R. G.; Perego, M.; Trask, N. A., Robust training and initialization of deep neural networks: an adaptive basis viewpoint, (Lu, J.; Ward, R., Proceedings of the First Mathematical and Scientific Machine Learning Conference. Proceedings of the First Mathematical and Scientific Machine Learning Conference, Proceedings of Machine Learning Research, vol. 107 (2020), PMLR, Princeton University: PMLR, Princeton University Princeton, NJ, USA), 512-536 |
| [59] | Schlömer, N.; Cervone, A.; McBain, G. D.; Tryfon mw; Nate; Gokstorp, F.; van Staden, R.; Toothstone; Dokken, J. S.; Kempf, D.; Sanchez, J.; Anzil; Bussonnier, M.; Fu, F.; Ivanmultiwave; Wagner, N.; Chen, S.; Tayebzaidi; Maric, T.; awa5114; Feng, Y., nschloe/pygmsh v7.1.5 (Dec. 2020) |
| [60] | Fuks, O.; Tchelepi, H. A., Limitations of physics informed machine learning for nonlinear two-phase transport in porous media, J. Mach. Learn. Model. Comput., 1, 1 (2020) |
| [61] | Sod, G. A., A survey of several finite difference methods for systems of nonlinear hyperbolic conservation laws, J. Comput. Phys., 27, 1-31 (1978) · Zbl 0387.76063 |
| [62] | Bird, G. A., Molecular Gas Dynamics and the Direct Simulation of Gas Flows (1994), Oxford University Press |
| [63] | Gallis, M. A.; Koehler, T. P.; Torczynski, J. R.; Plimpton, S. J., Direct simulation Monte Carlo investigation of the Richtmyer-Meshkov instability, Phys. Fluids, 27, Article 84105 pp. (2015) |
| [64] | Gallis, M. A.; Koehler, T. P.; Torczynski, J. R.; Plimpton, S. J., Direct simulation Monte Carlo investigation of the Rayleigh-Taylor instability, Phys. Rev. Fluids, 1, Article 43403 pp. (2016) |
| [65] | Gallis, M. A.; Torczynski, J. R.; Bitter, N. P.; Koehler, T. P.; Plimpton, S. J.; Papadakis, G., Gas-kinetic simulation of sustained turbulence in minimal Couette flow, Phys. Rev. Fluids, 3, Article 71402 pp. (2018) |
| [66] | Plimpton, S. J.; Moore, S. G.; Borner, A.; Stagg, A. K.; Koehler, T. P.; Torczynski, J. R.; Gallis, M. A., Direct simulation Monte Carlo on petaflop supercomputers and beyond, Phys. Fluids, 31, 8, Article 086101 pp. (2019) |
| [67] | Stukowski, A., Visualization and analysis of atomistic simulation data with OVITO-the open visualization tool, Model. Simul. Mater. Sci. Eng., 18, 1 (2010) |
| [68] | Plimpton, S., Fast parallel algorithms for short-range molecular dynamics, J. Comput. Phys., 117, 1, 1-19 (1995) · Zbl 0830.65120 |
| [69] | Wood, M. A.; Kittell, D. E.; Yarrington, C. D.; Thompson, A. P., Multiscale modeling of shock wave localization in porous energetic material, Phys. Rev. B, 97, 1, Article 014109 pp. (2018) |
| [70] | Mishin, Y.; Mehl, M.; Papaconstantopoulos, D.; Voter, A.; Kress, J., Structural stability and lattice defects in copper: ab initio, tight-binding, and embedded-atom calculations, Phys. Rev. B, 63, 22, Article 224106 pp. (2001) |
| [71] | Bringa, E.; Cazamias, J.; Erhart, P.; Stölken, J.; Tanushev, N.; Wirth, B.; Rudd, R.; Caturla, M., Atomistic shock Hugoniot simulation of single-crystal copper, J. Appl. Phys., 96, 7, 3793-3799 (2004) |
| [72] | Tuckerman, M. E.; Alejandre, J.; López-Rendón, R.; Jochim, A. L.; Martyna, G. J., A Liouville-operator derived measure-preserving integrator for molecular dynamics simulations in the isothermal-isobaric ensemble, J. Phys. A, Math. Gen., 39, 19, 5629 (2006) · Zbl 1098.81096 |
| [73] | Parrinello, M.; Rahman, A., Polymorphic transitions in single crystals: a new molecular dynamics method, J. Appl. Phys., 52, 12, 7182-7190 (1981) |
| [74] | Shu, C.-W., High order weno and dg methods for time-dependent convection-dominated pdes: a brief survey of several recent developments, J. Comput. Phys., 316, 598-613 (2016) · Zbl 1349.65486 |
| [75] | Bishop, C. M., Neural Networks for Pattern Recognition (1995), Oxford University Press |
| [76] | Ramani, R.; Reisner, J.; Shkoller, S., A space-time smooth artificial viscosity method with wavelet noise indicator and shock collision scheme, part 1: the 1-d case, J. Comput. Phys., 387, 81-116 (2019) · Zbl 1452.76159 |
| [77] | Lee, K.; Trask, N. A.; Patel, R. G.; Gulian, M. A.; Cyr, E. C., Partition of unity networks: deep hp-approximation (2021), arXiv preprint |
| [78] | You, H.; Yu, Y.; Trask, N.; Gulian, M.; D’Elia, M., Data-driven learning of robust nonlocal physics from high-fidelity synthetic data (2020), arXiv preprint |
| [79] | Patel, R. G.; Trask, N. A.; Wood, M. A.; Cyr, E. C., A physics-informed operator regression framework for extracting data-driven continuum models, Comput. Methods Appl. Mech. Eng., 373, Article 113500 pp. (2021) · Zbl 1506.62383 |
| [80] | Kingma, D. P.; Ba, J., Adam: a method for stochastic optimization (2014), arXiv preprint |
| [81] | Glorot, X.; Bengio, Y., Understanding the difficulty of training deep feedforward neural networks, (Teh, Y. W.; Titterington, M., Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy, Proceedings of Machine Learning Research, JMLR Workshop and Conference Proceedings, Chia Laguna Resort, vol. 9 (2010)), 249-256 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
