A physics-informed operator regression framework for extracting data-driven continuum models. (English) Zbl 1506.62383
Summary: The application of deep learning toward discovery of data-driven models requires careful application of inductive biases to obtain a description of physics which is both accurate and robust. We present here a framework for discovering continuum models from high fidelity molecular simulation data. Our approach applies a neural network parameterization of governing physics in modal space, allowing a characterization of differential operators while providing structure which may be used to impose biases related to symmetry, isotropy, and conservation form. We demonstrate the effectiveness of our framework for a variety of physics, including local and nonlocal diffusion processes and single and multiphase flows. For the flow physics we demonstrate this approach leads to a learned operator that generalizes to system characteristics not included in the training sets, such as variable particle sizes, densities, and concentration.
MSC:
| 62M45 | Neural nets and related approaches to inference from stochastic processes |
| 65C20 | Probabilistic models, generic numerical methods in probability and statistics |
Keywords:
physics-informed machine learning; operator regression; spectral methods; continuum scale modelingReferences:
| [1] | Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 113, 15, 3932-3937 (2016) · Zbl 1355.94013 |
| [2] | Rudy, S. H.; Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Data-driven discovery of partial differential equations, Sci. Adv., 3, 4, Article e1602614 pp. (2017) |
| [3] | Montáns, F. J.; Chinesta, F.; Gómez-Bombarelli, R.; Kutz, J. N., Data-driven modeling and learning in science and engineering, C. R. Méc., 347, 11, 845-855 (2019) |
| [4] | Pavliotis, G.; Stuart, A., Multiscale Methods: Averaging and Homogenization (2008), Springer Science & Business Media · Zbl 1160.35006 |
| [5] | Arbogast, T., Mixed multiscale methods for heterogeneous elliptic problems, (Numerical Analysis of Multiscale Problems (2012), Springer), 243-283 · Zbl 1248.65119 |
| [6] | Zwanzig, R., Nonequilibrium Statistical Mechanics (2001), Oxford University Press · Zbl 1267.82001 |
| [7] | Michel, J.-C.; Moulinec, H.; Suquet, P., Effective properties of composite materials with periodic microstructure: a computational approach, Comput. Methods Appl. Mech. Engrg., 172, 1-4, 109-143 (1999) · Zbl 0964.74054 |
| [8] | Mori, H., Transport, collective motion, and Brownian motion, Progr. Theoret. Phys., 33, 3, 423-455 (1965) · Zbl 0127.45002 |
| [9] | Li, Z.; Bian, X.; Li, X.; Karniadakis, G. E., Incorporation of memory effects in coarse-grained modeling via the Mori-Zwanzig formalism, J. Chem. Phys., 143, 24, Article 243128 pp. (2015) |
| [10] | LeCun, Y.; Bengio, Y.; Hinton, G., Deep learning, Nature, 521, 7553, 436-444 (2015) |
| [11] | Goodfellow, I.; Bengio, Y.; Courville, A., Deep Learning (2016), MIT Press · Zbl 1373.68009 |
| [12] | Baker, N.; Alexander, F.; Bremer, T.; Hagberg, A.; Kevrekidis, Y.; Najm, H.; Parashar, M.; Patra, A.; Sethian, J.; Wild, S., Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial IntelligenceTech. Rep. (2019), USDOE Office of Science: USDOE Office of Science Washington, DC |
| [13] | Battaglia, P. W.; Hamrick, J. B.; Bapst, V.; Sanchez-Gonzalez, A.; Zambaldi, V.; Malinowski, M.; Tacchetti, A.; Raposo, D.; Santoro, A.; Faulkner, R., Relational inductive biases, deep learning, and graph networks (2018), arXiv preprint arXiv:1806.01261 |
| [14] | Wu, J.-L.; Xiao, H.; Paterson, E., Physics-informed machine learning approach for augmenting turbulence models: A comprehensive framework, Phys. Rev. Fluids, 3, 7, Article 074602 pp. (2018) |
| [15] | Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175 |
| [16] | Ling, J.; Jones, R.; Templeton, J., Machine learning strategies for systems with invariance properties, J. Comput. Phys., 318, 22-35 (2016) · Zbl 1349.76124 |
| [17] | Biegler, L. T.; Ghattas, O.; Heinkenschloss, M.; van Bloemen Waanders, B., Large-scale PDE-constrained optimization: an introduction, (Large-Scale PDE-Constrained Optimization (2003), Springer), 3-13 · Zbl 1094.49500 |
| [18] | Haber, E.; Ascher, U. M.; Oldenburg, D. W., Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach, Geophysics, 69, 5, 1216-1228 (2004) |
| [19] | Epanomeritakis, I.; Akçelik, V.; Ghattas, O.; Bielak, J., A Newton-CG method for large-scale three-dimensional elastic full-waveform seismic inversion, Inverse Problems, 24, 3, Article 034015 pp. (2008) · Zbl 1142.65052 |
| [20] | Stuart, A. M., Inverse problems: a Bayesian perspective, Acta Numer., 19, 451-559 (2010) · Zbl 1242.65142 |
| [21] | Lu, L.; Jin, P.; Karniadakis, G. E., DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators (2019), arXiv preprint arXiv:1910.03193 |
| [22] | Bongard, J.; Lipson, H., Automated reverse engineering of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 104, 24, 9943-9948 (2007) · Zbl 1155.37044 |
| [23] | Schmidt, M.; Lipson, H., Distilling free-form natural laws from experimental data, Science, 324, 5923, 81-85 (2009) |
| [24] | Bright, I.; Lin, G.; Kutz, J., Compressive sensing based machine learning strategy for characterizing the flow around a cylinder with limited pressure measurements, Phys. Fluids, 25, 12, Article 127102 pp. (2013) · Zbl 1320.76003 |
| [25] | Long, Z.; Lu, Y.; Dong, B., PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network, J. Comput. Phys., 399, Article 108925 pp. (2019) · Zbl 1454.65131 |
| [26] | Patel, R. G.; Desjardins, O., Nonlinear integro-differential operator regression with neural networks (2018), arXiv preprint arXiv:1810.08552 |
| [27] | Boyd, J. P., Chebyshev and Fourier Spectral Methods (2001), Courier Corporation · Zbl 0994.65128 |
| [28] | Hesthaven, J. S.; Gottlieb, S.; Gottlieb, D., Spectral Methods for Time-Dependent Problems, Vol. 21 (2007), Cambridge University Press · Zbl 1111.65093 |
| [29] | Karniadakis, G.; Sherwin, S., Spectral/hp Element Methods for Computational Fluid Dynamics (2013), Oxford University Press · Zbl 1256.76003 |
| [30] | Wu, K.; Xiu, D., Data-driven deep learning of partial differential equations in modal space, J. Comput. Phys., 408, Article 109307 pp. (2020) · Zbl 07505629 |
| [31] | Zhang, D.; Guo, L.; Karniadakis, G. E., Learning in modal space: Solving time-dependent stochastic PDEs using physics-informed neural networks, SIAM J. Sci. Comput., 42, 2, A639-A665 (2020) · Zbl 1440.60067 |
| [32] | Qin, T.; Wu, K.; Xiu, D., Data driven governing equations approximation using deep neural networks, J. Comput. Phys., 395, 620-635 (2019) · Zbl 1455.65125 |
| [33] | Wu, K.; Xiu, D., Numerical aspects for approximating governing equations using data, J. Comput. Phys., 384, 200-221 (2019) · Zbl 1451.65008 |
| [34] | Zhang, D.; Guo, L.; Karniadakis, G. E., Learning in modal space: Solving time-dependent stochastic PDEs using physics-informed neural networks, SIAM J. Sci. Comput., 42, A639-A665 (2020) · Zbl 1440.60067 |
| [35] | Nocedal, J.; Wright, S., Numerical Optimization (2006), Springer Science & Business Media · Zbl 1104.65059 |
| [36] | J. Hicken, J. Alonso, Comparison of reduced-and full-space algorithms for PDE-constrained optimization, in: 51st AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, 2013, p. 1043. |
| [37] | Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G. S.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Goodfellow, I.; Harp, A.; Irving, G.; Isard, M.; Jia, Y.; Jozefowicz, R.; Kaiser, L.; Kudlur, M.; Levenberg, J.; Mané, D.; Monga, R.; Moore, S.; Murray, D.; Olah, C.; Schuster, M.; Shlens, J.; Steiner, B.; Sutskever, I.; Talwar, K.; Tucker, P.; Vanhoucke, V.; Vasudevan, V.; Viégas, F.; Vinyals, O.; Warden, P.; Wattenberg, M.; Wicke, M.; Yu, Y.; Zheng, X., TensorFlow: Large-scale machine learning on heterogeneous systems (2015), Software available from tensorflow.org |
| [38] | Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; Desmaison, A.; Kopf, A.; Yang, E.; DeVito, Z.; Raison, M.; Tejani, A.; Chilamkurthy, S.; Steiner, B.; Fang, L.; Bai, J.; Chintala, S., PyTorch: An imperative style, high-performance deep learning library, (Wallach, H.; Larochelle, H.; Beygelzimer, A.; d’ Alché-Buc, F.; Fox, E.; Garnett, R., Advances in Neural Information Processing Systems 32 (2019), Curran Associates, Inc.), 8024-8035 |
| [39] | Press, W. H.; Teukolsky, S. A.; Vetterling, W. T.; Flannery, B. P., Numerical Recipes 3rd Edition: The Art of Scientific Computing (2007), Cambridge University Press · Zbl 1132.65001 |
| [40] | Trask, N.; Patel, R. G.; Gross, B. J.; Atzberger, P. J., GMLS-Nets: A framework for learning from unstructured data (2019), arXiv preprint arXiv:1909.05371 |
| [41] | Bar-Sinai, Y.; Hoyer, S.; Hickey, J.; Brenner, M. P., Learning data-driven discretizations for partial differential equations, Proc. Natl. Acad. Sci., 116, 31, 15344-15349 (2019) · Zbl 1431.65195 |
| [42] | Dieleman, S.; Willett, K. W.; Dambre, J., Rotation-invariant convolutional neural networks for galaxy morphology prediction, Mon. Not. R. Astron. Soc. (2015) |
| [43] | Wong, S. C.; Gatt, A.; Stamatescu, V.; McDonnell, M. D., Understanding data augmentation for classification: when to warp? (2016), arXiv preprint arXiv:1609.08764 |
| [44] | Wang, J.; Perez, L., The effectiveness of data augmentation in image classification using deep learning (2017), arXiv preprint arXiv:1712.04621v1 |
| [45] | Inoue, H., Data augmentation by pairing samples for images classification (2018), arXiv preprint arXiv:1801.02929 |
| [46] | Lawrence, S.; Giles, C.; Ah Chung Tsoi, V.; Back, A., Face recognition: a convolutional neural-network approach, IEEE Trans. Neural Netw., 8, 1, 98-113 (1997) |
| [47] | A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, in: Conference on Neural Information Processing Systems, 2012, pp. 1097-1105. |
| [48] | Meltzer, P.; Mallea, M. D.G.; Bentley, P. J., PiNet: A permutation invariant graph neural network for graph classification (2019), arXiv preprint arXiv:1905.03046 |
| [49] | Thomas, N.; Smidt, T.; Kearnes, S.; Yang, L.; Li, L.; Kohlhoff, K.; Riley, P., Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds (2018), arXiv preprint arXiv:1802.08219 |
| [50] | Esteves, C.; Allen-Blanchette, C.; Makadia, A.; Daniilidis, K., Learning SO(3) equivariant representations with spherical CNNs (2017), arXiv preprint arXiv:1711.06721 |
| [51] | Worrall, D.; Brostow, G., CubeNet: Equivariance to 3D rotation and translation (2018), arXiv preprint arXiv:1804.04458 |
| [52] | Sokolov, I. M.; Klafter, J., From diffusion to anomalous diffusion: a century after Einstein’s Brownian motion, Chaos, 15, 2, Article 026103 pp. (2005) · Zbl 1080.82022 |
| [53] | Gulian, M.; Pang, G., Stochastic solution of elliptic and parabolic boundary value problems for the spectral fractional Laplacian (2018), arXiv preprint arXiv:1812.01206 |
| [54] | Lischke, A.; Pang, G.; Gulian, M.; Song, F.; Glusa, C.; Zheng, X.; Mao, Z.; Cai, W.; Meerschaert, M. M.; Ainsworth, M., What is the fractional Laplacian? (2018), arXiv preprint arXiv:1801.09767 |
| [55] | Plimpton, S., Fast Parallel Algorithms for Short-Range Molecular DynamicsTech. Rep. (1993), Sandia National Labs.: Sandia National Labs. Albuquerque, NM |
| [56] | Tuckerman, M.; Berne, B. J.; Martyna, G. J., Reversible multiple time scale molecular dynamics, J. Chem. Phys., 97, 3, 1990-2001 (1992) |
| [57] | Parrinello, M.; Rahman, A., Polymorphic transitions in single crystals: A new molecular dynamics method, J. Appl. Phys., 52, 12, 7182-7190 (1981) |
| [58] | Zuo, Y.; Chen, C.; Li, X.; Deng, Z.; Chen, Y.; Behler, J.; Csányi, G.; Shapeev, A. V.; Thompson, A. P.; Wood, M. A., Performance and cost assessment of machine learning interatomic potentials, J. Phys. Chem. A, 124, 4, 731-745 (2020) |
| [59] | Everaers, R.; Ejtehadi, M., Interaction potentials for soft and hard ellipsoids, Phys. Rev. E, 67, 4, Article 041710 pp. (2003) |
| [60] | Head, T.; MechCoder, M.; Louppe, G.; Shcherbatyi, I.; fcharras, Y.; cius, Z. V.; cmmalone, G.; Schröder, C.; nel215, A. P.; Campos, N.; Young, T.; Cereda, S.; Fan, T.; rene rex, Z.; Shi, K. K.; Schwabedal, J.; carlosdanielcsantos, S.; Hvass-Labs, B.; Pak, M.; SoManyUsernamesTaken, J.; Callaway, F.; Estève, L.; Besson, L.; Cherti, M.; Pfannschmidt, K.; Linzberger, F.; Cauet, C.; Gut, A.; Mueller, A.; Fabisch, A., scikit-optimize/scikit-optimize: v0.5.2 (2018) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
