Deep learning discrete calculus (DLDC): a family of discrete numerical methods by universal approximation for STEM education to frontier research. (English) Zbl 07719575
Summary: The article proposes formulating and codifying a set of applied numerical methods, coined as Deep Learning Discrete Calculus (DLDC), that uses the knowledge from discrete numerical methods to interpret the deep learning algorithms through the lens of applied mathematics. The DLDC methods aim to leverage the flexibility and ever-increasing resources of deep learning and rich literature on numerical analysis to formulate a general class of numerical methods that can directly use data with uncertainty to predict the behavior of an unknown system as well as elevate the speed and accuracy of numerical solution of the governing equations for known systems. The article is structured into two major sections. In the first section, the building blocks of the DLDC methods are presented and deep learning structures analogous to traditional numerical methods such as finite difference and finite element methods are constructed with a view to incorporate these techniques in Science, Technology, Engineering, Mathematics syllabus for K-12 students. The second section builds upon the building blocks of the previous discussion and proposes new solution schemes for differential and integral equations pertinent to multiscale mechanics. Each section is accompanied by a mathematical formulation of the numerical methods, analogous DLDC formulation, and suitable examples.
MSC:
| 74S99 | Numerical and other methods in solid mechanics |
| 74S05 | Finite element methods applied to problems in solid mechanics |
| 74K10 | Rods (beams, columns, shafts, arches, rings, etc.) |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68T07 | Artificial neural networks and deep learning |
Keywords:
kernel learning; deep learning algorithm; convolution; data uncertainty; space-time finite element method; one-dimensional elastic bar; spring mass damper systemReferences:
| [1] | Saha, S.; Gan, Z.; Cheng, L.; Gao, J.; Kafka, OL; Xie, X.; Li, H.; Tajdari, M.; Kim, HA; Liu, WK, Hierarchical deep learning neural network (hidenn): an artificial intelligence (ai) framework for computational science and engineering, Comput Methods Appl Mech Eng, 373, 113452 (2021) · Zbl 1506.68110 |
| [2] | Xie, X.; Bennett, J.; Saha, S.; Lu, Y.; Cao, J.; Liu, WK; Gan, Z., Mechanistic data-driven prediction of as-built mechanical properties in metal additive manufacturing, npj Comput Mater, 7, 1, 1-12 (2021) |
| [3] | Gan, Z.; Jones, KK; Lu, Y.; Liu, WK, Benchmark study of melted track geometries in laser powder bed fusion of inconel 625, Integr Mater Manuf Innov, 10, 2, 177-195 (2021) |
| [4] | Kafka, OL; Jones, KK; Yu, C.; Cheng, P.; Liu, WK, Image-based multiscale modeling with spatially varying microstructures from experiments: demonstration with additively manufactured metal in fatigue and fracture, J Mech Phys Solids, 150, 104350 (2021) |
| [5] | Sirignano, J.; Spiliopoulos, K., Dgm: a deep learning algorithm for solving partial differential equations, J Comput Phys, 375, 1339-1364 (2018) · Zbl 1416.65394 |
| [6] | Raissi, M., Deep hidden physics models: deep learning of nonlinear partial differential equations, J Mach Learn Res, 19, 1, 932-955 (2018) · Zbl 1439.68021 |
| [7] | Han, J.; Jentzen, A., Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Commun Math Stat, 5, 4, 349-380 (2017) · Zbl 1382.65016 |
| [8] | Yu, B., The deep ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun Math Stat, 6, 1, 1-12 (2018) · Zbl 1392.35306 |
| [9] | Xiao, L.; Liao, B.; Li, S.; Chen, K., Nonlinear recurrent neural networks for finite-time solution of general time-varying linear matrix equations, Neural Netw, 98, 102-113 (2018) · Zbl 1441.93265 |
| [10] | Zhu M, Chang B, Fu C (2018) Convolutional neural networks combined with runge-kutta methods, arXiv preprint arXiv:1802.08831 |
| [11] | Kani JN, Elsheikh AH (2017) Dr-rnn: a deep residual recurrent neural network for model reduction, arXiv preprint arXiv:1709.00939 |
| [12] | Karl M, Soelch M, Bayer J, Van der Smagt P (2016) Deep variational bayes filters: unsupervised learning of state space models from raw data, arXiv preprint arXiv:1605.06432 |
| [13] | Chakraverty, S.; Mall, S., Artificial neural networks for engineers and scientists: solving ordinary differential equations (2017), Boca Raton: CRC Press, Boca Raton · Zbl 1375.00005 |
| [14] | Magill M, Qureshi F, de Haan H (2018) Neural networks trained to solve differential equations learn general representations. Adv Neural Inf Process Syst. Vol. 31 |
| [15] | Liu, WK; Gan, Z.; Fleming, M., Mechanistic data science for stem education and applications (2021), Berlin: Springer, Berlin |
| [16] | Lubliner, J.; Oliver, J.; Oller, S.; Oñate, E., A plastic-damage model for concrete, Int J Solids Struct, 25, 3, 299-326 (1989) |
| [17] | Raissi, M.; Perdikaris, P.; Karniadakis, GE, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J Comput Phys, 378, 686-707 (2019) · Zbl 1415.68175 |
| [18] | Karniadakis, GE; Kevrekidis, IG; Lu, L.; Perdikaris, P.; Wang, S.; Yang, L., Physics-informed machine learning, Nat Rev Phys, 3, 6, 422-440 (2021) |
| [19] | Pang, G.; Lu, L.; Karniadakis, GE, fpinns: fractional physics-informed neural networks, SIAM J Sci Comput, 41, 4, A2603-A2626 (2019) · Zbl 1420.35459 |
| [20] | Krishnapriyan, A.; Gholami, A.; Zhe, S.; Kirby, R.; Mahoney, MW, Characterizing possible failure modes in physics-informed neural networks, Adv Neural Inf Process Syst, 34, 26548-26560 (2021) |
| [21] | Li, R.; Lee, E.; Luo, T., Physics-informed neural networks for solving multiscale mode-resolved phonon Boltzmann transport equation, Mater Today Phys, 19, 100429 (2021) |
| [22] | Leung WT, Lin G, Zhang Z (2021) Nh-pinn: neural homogenization based physics-informed neural network for multiscale problems. arXiv preprint arXiv:2108.12942 · Zbl 07599595 |
| [23] | Brunton, SL; Proctor, JL; Kutz, JN, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc Natl Acad Sci, 113, 15, 3932-3937 (2016) · Zbl 1355.94013 |
| [24] | Kaiser, E.; Kutz, JN; Brunton, SL, Sparse identification of nonlinear dynamics for model predictive control in the low-data limit, Proc R Soc A, 474, 2219, 20180335 (2018) · Zbl 1425.93175 |
| [25] | Kaheman, K.; Kutz, JN; Brunton, SL, Sindy-pi: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics, Proc R Soc A, 476, 2242, 20200279 (2020) · Zbl 1473.93007 |
| [26] | Gan, Z.; Kafka, OL; Parab, N.; Zhao, C.; Fang, L.; Heinonen, O.; Sun, T.; Liu, WK, Universal scaling laws of keyhole stability and porosity in 3d printing of metals, Nat Commun, 12, 1, 1-8 (2021) |
| [27] | Xie X, Liu WK, Gan Z (2021) Data-driven discovery of dimensionless numbers and scaling laws from experimental measurements. arXiv preprint arXiv:2111.03583 |
| [28] | Chen RT, Rubanova Y,Bettencourt J, Duvenaud DK (2018) Neural ordinary differential equations. Adv Neural Inf Process Syst. Vol. 31 |
| [29] | Zhang, L.; Cheng, L.; Li, H.; Gao, J.; Yu, C.; Domel, R.; Yang, Y.; Tang, S.; Liu, WK, Hierarchical deep-learning neural networks: finite elements and beyond, Comput Mech, 67, 1, 207-230 (2021) · Zbl 07360501 |
| [30] | Niu MY, Horesh L, Chuang I (2019) Recurrent neural networks in the eye of differential equations. arXiv preprint arXiv:1904.12933 |
| [31] | Hughes, TW; Williamson, IA; Minkov, M.; Fan, S., Wave physics as an analog recurrent neural network, Sci Adv, 5, 12, eaay6946 (2019) |
| [32] | He, J.; Xu, J., Mgnet: a unified framework of multigrid and convolutional neural network, Sci China Math, 62, 7, 1331-1354 (2019) · Zbl 1476.65026 |
| [33] | Kovachki N, Li Z, Liu B, Azizzadenesheli K, Bhattacharya K, Stuart A, Anandkumar A (2021) Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.08481 |
| [34] | Li Z, Kovachki N, Azizzadenesheli K, Liu B, Bhattacharya K, Stuart A, Anandkumar A (2020)Fourier neural operator for parametric partial differential equations arXiv preprint arXiv:2010.08895 |
| [35] | Lu L, Jin P, Karniadakis GE (2019) Deeponet: learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193 |
| [36] | Lu, L.; Meng, X.; Cai, S.; Mao, Z.; Goswami, S.; Zhang, Z.; Karniadakis, GE, A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data, Comput Methods Appl Mech Eng, 393, 114778 (2022) · Zbl 1507.65050 |
| [37] | Anandkumar A, Azizzadenesheli K, Bhattacharya K, Kovachki N, Li Z, Liu B, Stuart A (2020) Neural operator: graph kernel network for partial differential equations. In: ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations |
| [38] | You H, Yu Y, D’Elia M, Gao T, Silling S (2022) Nonlocal kernel network (nkn): a stable and resolution-independent deep neural network. arXiv preprint arXiv:2201.02217 · Zbl 07592139 |
| [39] | Tucker C, Jackson K, Park JJ (2020) Exploring the future of engineering education: Perspectives from a workshop on artificial intelligence and the future of stem and societies. In: American Society of Engineering Education |
| [40] | Wang N, Tonko P, Ragav N, Chungyoun M, Plucker J (2022) A perspective on k-12 ai education. Technol Innov |
| [41] | Touretzky DS, Gardner-McCune C (2022) Artificial intelligence thinking in k-12, Computational Thinking Education in K-12: Artificial Intelligence Literacy and Physical Computing, pp. 153-180 |
| [42] | Touretzky D, Gardner-McCune C, Seehorn D (2022) Machine learning and the five big ideas in ai. Int J Artif Intell Educ. pp. 1-34 |
| [43] | Yin Y (2022) Ai4all: Ai education for k-12, tech. rep., EasyChair |
| [44] | Chapra, SC; Canale, RP, Numerical methods for engineers (2011), New York: Mcgraw-hill, New York |
| [45] | Bellman, R.; Casti, J., Differential quadrature and long-term integration, J Math Anal Appl, 34, 2, 235-238 (1971) · Zbl 0236.65020 |
| [46] | Bert CW, Malik M (1996) Differential quadrature method in computational mechanics: a review. Appl Mech Rev |
| [47] | Shu, C., Differential quadrature and its application in engineering (2012), Berlin: Springer, Berlin |
| [48] | Savitzky, A.; Golay, MJ, Smoothing and differentiation of data by simplified least squares procedures, Anal Chem, 36, 8, 1627-1639 (1964) |
| [49] | Gondara L (2016) Medical image denoising using convolutional denoising autoencoders. In: 2016 IEEE 16th international conference on data mining workshops (ICDMW), pp. 241-246, IEEE |
| [50] | Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, et al (2018) Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261 |
| [51] | Sanchez-Gonzalez A, Godwin J, Pfaff T, Ying R, Leskovec J, Battaglia P (2020) Learning to simulate complex physics with graph networks. In: International Conference on Machine Learning, pp. 8459-8468, PMLR |
| [52] | Park C, Lu Y, Saha S, Xue T, Guo J, Mojumder S, Wagner G, Liu W (2023) Convolution hierarchical deep-learning neural network (c-hidenn) with graphics processing unit (gpu) acceleration. Computational Mechanics · Zbl 07719578 |
| [53] | Liu, G.; Gu, Y., A local radial point interpolation method (lrpim) for free vibration analyses of 2-d solids, J Sound Vib, 246, 1, 29-46 (2001) |
| [54] | Liu, G-R; Gu, Y-T, An introduction to meshfree methods and their programming (2005), Berlin: Springer, Berlin |
| [55] | Hughes, TJ; Hulbert, GM, Space-time finite element methods for elastodynamics: formulations and error estimates, Comput Methods Appl Mech Eng, 66, 3, 339-363 (1988) · Zbl 0616.73063 |
| [56] | Hulbert, GM; Hughes, TJ, Space-time finite element methods for second-order hyperbolic equations, Comput Methods Appl Mech Eng, 84, 3, 327-348 (1990) · Zbl 0754.73085 |
| [57] | Wang, L.; Zhong, H., A time finite element method for structural dynamics, Appl Math Model, 41, 445-461 (2017) · Zbl 1443.65227 |
| [58] | Taskaya-Temizel, T.; Casey, MC, A comparative study of autoregressive neural network hybrids, Neural Netw, 18, 5-6, 781-789 (2005) |
| [59] | Triebe O, Laptev N, Rajagopal R (2018) Ar-net: a simple auto-regressive neural network for time-series. arXiv preprint arXiv:1911.12436 |
| [60] | Lippmann, BA; Schwinger, J., Variational principles for scattering processes. i, Phys Rev, 79, 3, 469 (1950) · Zbl 0039.42406 |
| [61] | Gopal, A.; Martinsson, P-G, An accelerated, high-order accurate direct solver for the lippmann-schwinger equation for acoustic scattering in the plane, Adv Comput Math, 48, 4, 1-31 (2022) · Zbl 1492.65358 |
| [62] | Zecevic, M.; Lebensohn, RA; Capolungo, L., New large-strain fft-based formulation and its application to model strain localization in nano-metallic laminates and other strongly anisotropic crystalline materials, Mech Mater, 166, 104208 (2022) |
| [63] | Moulinec, H.; Suquet, P., A numerical method for computing the overall response of nonlinear composites with complex microstructure, Comput Methods Appl Mech Eng, 157, 1-2, 69-94 (1998) · Zbl 0954.74079 |
| [64] | Li Z, Kovachki N, Azizzadenesheli K, Liu B, Bhattacharya K, Stuart A, Anandkumar A (2020)Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485 |
| [65] | Saha, S.; Kafka, OL; Lu, Y.; Yu, C.; Liu, WK, Macroscale property prediction for additively manufactured in625 from microstructure through advanced homogenization, Integr Mater Manuf Innov, 10, 3, 360-372 (2021) |
| [66] | Saha, S.; Kafka, OL; Lu, Y.; Yu, C.; Liu, WK, Microscale structure to property prediction for additively manufactured in625 through advanced material model parameter identification, Integr Mater Manuf Innov, 10, 2, 142-156 (2021) |
| [67] | Liu, Z.; Bessa, MA; Liu, WK, Self-consistent clustering analysis: an efficient multi-scale scheme for inelastic heterogeneous materials, Comput Methods Appl Mech Eng, 306, 319-341 (2016) · Zbl 1436.74070 |
| [68] | Yu, C.; Kafka, OL; Liu, WK, Self-consistent clustering analysis for multiscale modeling at finite strains, Comput Methods Appl Mech Eng, 349, 339-359 (2019) · Zbl 1441.74309 |
| [69] | Hartigan, JA; Wong, MA, Algorithm as 136: a k-means clustering algorithm, J R Stat Soc Ser C (Appl Stat), 28, 1, 100-108 (1979) · Zbl 0447.62062 |
| [70] | Huang O, Saha S, Guo J, Liu WK (2023) An introductory treatment on mechanistic neural operators for homogenization by self-consistent clustering analysis: Part 1, Comput Mech (submitted) |
| [71] | McVeigh, C.; Vernerey, F.; Liu, WK; Brinson, LC, Multiresolution analysis for material design, Comput Methods Appl Mech Eng, 195, 37-40, 5053-5076 (2006) · Zbl 1118.74040 |
| [72] | McVeigh, C.; Liu, WK, Multiresolution modeling of ductile reinforced brittle composites, J Mech Phys Solids, 57, 2, 244-267 (2009) · Zbl 1170.74321 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
