N-adaptive Ritz method: a neural network enriched partition of unity for boundary value problems. (English) Zbl 07875014
Summary: Conventional finite element methods are known to be tedious in adaptive refinements due to their conformal regularity requirements. Further, the enrichment functions for adaptive refinements are often not readily available in general applications. This work introduces a novel neural network-enriched Partition of Unity (NN-PU) approach for solving boundary value problems via artificial neural networks with a potential energy-based loss function minimization. The flexibility and adaptivity of the NN function space are utilized to capture complex solution patterns that the conventional Galerkin methods fail to capture. The NN enrichment is constructed by combining pre-trained feature-encoded NN blocks with an additional untrained NN block. The pre-trained NN blocks learn specific local features during the offline stage, enabling efficient enrichment of the approximation space during the online stage through the Ritz-type energy minimization. The NN enrichment is introduced under the Partition of Unity (PU) framework, ensuring convergence of the proposed method. The proposed NN-PU approximation and feature-encoded transfer learning form an adaptive approximation framework, termed the neural-refinement (n-refinement), for solving boundary value problems. Demonstrated by solving various elasticity problems, the proposed method offers accurate solutions while notably reducing the computational cost compared to the conventional adaptive refinement in the mesh-based methods.
Keywords:
Ritz method; partition of unity; neural network enrichment; energy minimization; transfer-learning; adaptivityReferences:
| [1] | Hornik, K., Approximation capabilities of multilayer feedforward networks, Neural Netw., 4, 2, 251-257, 1991 |
| [2] | Barron, A. R., Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Inf. Theory, 39, 3, 930-945, 1993 · Zbl 0818.68126 |
| [3] | Vlassis, N. N.; Ma, R.; Sun, W. C., Geometric deep learning for computational mechanics Part I: anisotropic hyperelasticity, Comput. Methods Appl. Mech. Eng., 371, Article 113299 pp., 2020 · Zbl 1506.74504 |
| [4] | Vlassis, N. N.; Sun, W. C., Sobolev training of thermodynamic-informed neural networks for interpretable elasto-plasticity models with level set hardening, Comput. Methods Appl. Mech. Eng., 377, Article 113695 pp., 2021 · Zbl 1506.74449 |
| [5] | Xu, K.; Huang, D. Z.; Darve, E., Learning constitutive relations using symmetric positive definite neural networks, J. Comput. Phys., 428, Article 110072 pp., 2021 · Zbl 07511428 |
| [6] | He, X.; Chen, J. S., Thermodynamically consistent machine-learned internal state variable approach for data-driven modeling of path-dependent materials, Comput. Methods Appl. Mech. Eng., 402, Article 115348 pp., 2022 · Zbl 1507.74007 |
| [7] | He, Y.; Semnani, S. J., Machine learning based modeling of path-dependent materials for finite element analysis, Comput. Geotech., 156, Article 105254 pp., 2023 |
| [8] | Xiong, Z.; Xiao, M.; Vlassis, N.; Sun, W. C., A neural kernel method for capturing multiscale high-dimensional micromorphic plasticity of materials with internal structures, Comput. Methods Appl. Mech. Eng., 416, Article 116317 pp., 2023 · Zbl 1543.74015 |
| [9] | Kirchdoerfer, T.; Ortiz, M., Data-driven computational mechanics, Comput. Methods Appl. Mech. Eng., 304, 81-101, 2016 · Zbl 1425.74503 |
| [10] | Ibañez, R.; Abisset-Chavanne, E.; Aguado, J. V.; Gonzalez, D.; Cueto, E.; Chinesta, F., A manifold learning approach to data-driven computational elasticity and inelasticity, Arch. Comput. Methods Eng., 25, 1, 47-57, 2018 · Zbl 1390.74195 |
| [11] | Eggersmann, R.; Kirchdoerfer, T.; Reese, S.; Stainier, L.; Ortiz, M., Model-free data-driven inelasticity, Comput. Methods Appl. Mech. Eng., 350, 81-99, 2019 · Zbl 1441.74048 |
| [12] | He, Q.; Chen, J. S., A physics-constrained data-driven approach based on locally convex reconstruction for noisy database, Comput. Methods Appl. Mech. Eng., 363, Article 112791 pp., 2020 · Zbl 1436.62725 |
| [13] | He, X.; He, Q.; Chen, J. S.; Sinha, U.; Sinha, S., Physics-constrained local convexity data-driven modeling of anisotropic nonlinear elastic solids, Data Centr. Eng, 1, 5, e19, 2020 |
| [14] | He, X.; He, Q.; Chen, J. S., Deep autoencoders for physics-constrained data-driven nonlinear materials modeling, Comput. Methods Appl. Mech. Eng., 385, Article 114034 pp., 2021 · Zbl 1502.74109 |
| [15] | Bahmani, B.; Sun, W. C., Distance-preserving manifold denoising for data-driven mechanics, Comput. Methods Appl. Mech. Eng., 405, Article 115857 pp., 2023 · Zbl 1539.74538 |
| [16] | Liu, Z.; Bessa, M. A.; Liu, W. K., Self-consistent clustering analysis: an efficient multi-scale scheme for inelastic heterogeneous materials, Comput. Methods Appl. Mech. Eng., 306, 319-341, 2016 · Zbl 1436.74070 |
| [17] | Wang, K.; Sun, W. C., A multiscale multi-permeability poroplasticity model linked by recursive homogenizations and deep learning, Comput. Methods Appl. Mech. Eng., 334, 337-380, 2018 · Zbl 1440.74130 |
| [18] | He, X.; Taneja, K.; Chen, J. S.; Lee, C. H.; Hodgson, J.; Malis, V.; Sinha, U.; Sinha, S., Multiscale modeling of passive material influences on deformation and force output of skeletal muscles, Int. J. Numer. Method. Biomed. Eng., 38, 4, e3571, 2022 |
| [19] | Bishara, D.; Xie, Y.; Liu, W. K.; Li, S., A state-of-the-art review on machine learning-based multiscale modeling, simulation, homogenization and design of materials, Arch. Comput. Methods Eng., 30, 1, 191-222, 2023 |
| [20] | Wei, H.; Wu, C. T.; Hu, W.; Su, T. H.; Oura, H.; Nishi, M.; Naito, T.; Chung, S.; Shen, L., LS-DYNA machine learning-based multiscale method for nonlinear modeling of short fiber-reinforced composites, J. Eng. Mech., 149, 3, Article 04023003 pp., 2023 |
| [21] | Baek, J.; Chen, J. S.; Susuki, K., A neural network‑enhanced reproducing kernel particle method for modeling strain localization, Int. J. Numer. Methods Eng., 123, 18, 4422-4454, 2022 · Zbl 07768033 |
| [22] | Baek, J.; Chen, J. S., A neural network-based enrichment of reproducing kernel approximation for modeling brittle fracture, Comput. Methods Appl. Mech. Eng., 419, Article 116590 pp., 2024 · Zbl 1539.74363 |
| [23] | Duarte, C. A.; Babuška, I.; Oden, J. T., Generalized finite element methods for three-dimensional structural mechanics problems, Comput. Struct., 77, 2, 215-232, 2000 |
| [24] | Duarte, C. A.; Kim, D. J., Analysis and applications of a generalized finite element method with global-local enrichment functions, Comput. Methods Appl. Mech. Eng., 197, 6-8, 487-504, 2008 · Zbl 1169.74597 |
| [25] | Bergstra, J.; Breuleux, O.; Bastien, F.; Lamblin, P.; Pascanu, R.; Desjardins, G.; Turian, J.; Warde-Farley, D.; Bengio, Y., Theano: a CPU and GPU math expression compiler, Proc. Python Sci. Comput. Conf., 4, 3, 1-7, 2010 |
| [26] | Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; Kudlur, M., TensorFlow: a system for large-scale machine learning, (Proc. 12th USENIX Symp. Oper. Syst. Des. Implementation, OSDI 2016, 2016), 265-283, Available: |
| [27] | T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang, “MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems,” 2015, Available: https://arxiv.org/abs/1512.01274v1. |
| [28] | Chollet, F., Deep Learning With Python, 2021, Simon and Schuster · Zbl 1467.68002 |
| [29] | Baydin, A. G.; Pearlmutter, B. A.; Radul, A. A.; Siskind, J. M., Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res., 18, 1-43, 2018 · Zbl 06982909 |
| [30] | Jacobs, R. A.; Jordan, M. I.; Nowlan, S. J.; Hinton, G. E., Adaptive mixtures of local experts, Neural Comput, 3, 1, 79-87, 1991 |
| [31] | K. Lee, N.A. Trask, R.G. Patel, M.A. Gulian, and E.C. Cyr, “Partition of unity networks: deep hp-approximation,” arXiv preprint arXiv:2101.11256, 2021. |
| [32] | Trask, N.; Henriksen, A.; Martinez, C.; Cyr, E., Hierarchical partition of unity networks: fast multilevel training, Proc. Mach. Learn. Res., 145, 1-20, 2022 |
| [33] | Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707, 2019 · Zbl 1415.68175 |
| [34] | Haghighat, E.; Juanes, R., SciANN: a Keras/TensorFlow wrapper for scientific computations and physics-informed deep learning using artificial neural networks, Comput. Methods Appl. Mech. Eng., 373, Article 113552 pp., 2021 · Zbl 1506.65251 |
| [35] | Sirignano, J.; Spiliopoulos, K., DGM: a deep learning algorithm for solving partial differential equations, J. Comput. Phys., 375, 1339-1364, 2018 · Zbl 1416.65394 |
| [36] | Weinan, E.; Yu, B., The deep ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., 6, 1, 1-12, 2018 · Zbl 1392.35306 |
| [37] | Samaniego, E.; Anitescu, C.; Goswami, S.; Nguyen-Thanh, V. M.; Guo, H.; Hamdia, K.; Zhuang, X.; Rabczuk, T., An energy approach to the solution of partial differential equations in computational mechanics via machine learning: concepts, implementation and applications, Comput. Methods Appl. Mech. Eng., 362, Article 112790 pp., 2020 · Zbl 1439.74466 |
| [38] | Nguyen-Thanh, V. M.; Zhuang, X.; Rabczuk, T., A deep energy method for finite deformation hyperelasticity, Eur. J. Mech. A/Solids, 80, Article 103874 pp., 2020 · Zbl 1472.74213 |
| [39] | Nguyen-Thanh, V. M.; Anitescu, C.; Alajlan, N.; Rabczuk, T.; Zhuang, X., Parametric deep energy approach for elasticity accounting for strain gradient effects, Comput. Methods Appl. Mech. Eng., 386, Article 114096 pp., 2021 · Zbl 1507.74571 |
| [40] | Saha, S.; Gan, Z.; Cheng, L.; Gao, J.; Kafka, O. L.; Xie, X.; Li, H.; Tajdari, M.; Kim, H. A.; Liu, W. K., Hierarchical deep learning Neural Network (HiDeNN): an artificial intelligence (AI) framework for computational science and engineering, Comput. Methods Appl. Mech. Eng., 373, Article 113452 pp., 2021 · Zbl 1506.68110 |
| [41] | Melenk, J. M.; Babuška, I., The partition of unity finite element method: basic theory and applications, Comput. Methods Appl. Mech. Eng., 139, 1-4, 289-314, 1996 · Zbl 0881.65099 |
| [42] | Babuška, I. K.; Melenk, J. M., The partition of unity method, Int. J. Numer. Methods Eng., 40, 727-758, 1997 · Zbl 0949.65117 |
| [43] | Duarte, C. A.; Oden, J. T., An h-p adaptive method using clouds, Comput. Methods Appl. Mech. Eng., 139, 1-4, 237-262, 1996 · Zbl 0918.73328 |
| [44] | Duarte, A.; Tinsley Odent, J., H-p clouds-An h-p meshless method, Numer. Methods Partial Differ. Equations An Int. J., 12, 673-705, 1996 · Zbl 0869.65069 |
| [45] | Hughes, T. J.R.; Cottrell, J. A.; Bazilevs, Y., Isogeometric analysis: CAD, finite elements, NURBS, exact geometry and mesh refinement, Comput. Methods Appl. Mech. Eng., 194, 39-41, 4135-4195, 2005, Oct. · Zbl 1151.74419 |
| [46] | Belytschko, T.; Lu, Y. Y.; Gu, L., Element-free Galerkin methods, Int. J. Numer. Methods Eng., 37, 2, 229-256, 1994, Jan. · Zbl 0796.73077 |
| [47] | Liu, W. K.; Jun, S.; Zhang, Y. F., Reproducing kernel particle methods, Int. J. Numer. Methods Fluids, 20, 8-9, 1081-1106, 1995 · Zbl 0881.76072 |
| [48] | Chen, J. S.; Pan, C.; Wu, C. T.; Liu, W. K., Reproducing kernel particle methods for large deformation analysis of non-linear structures, Comput. Methods Appl. Mech. Eng., 139, 1-4, 195-227, 1996, Dec. · Zbl 0918.73330 |
| [49] | Fleming, M.; Chur, Y. A.; Belytschko, T., Enriched element-free Galerkin methods for crack tip fields, Int. J. Numer. Methods Eng., 29, 1483-1504, 1996 |
| [50] | Belytschko, T.; Fleming, M., Smoothing, enrichment and contact in the element-free Galerkin method, Comput. Struct., 71, 2, 173-195, 1999 |
| [51] | Chen, J. S.; Hillman, M.; Chi, W., Meshfree methods: progress made after 20 years, Am. Soc. Civ. Eng., 143, 4, 2017 |
| [52] | Belytschko, T.; Chen, J. S.; Hillman, M. C., Meshfree and Particle methods: Fundamentals and Applications, 2024, John Wiley & Sons |
| [53] | Clevert, D. A.; Unterthiner, T.; Hochreiter, S., Fast and accurate deep network learning by exponential linear units (ELUs), (4th Int. Conf. Learn. Represent. ICLR 2016 - Conf. Track Proc, 2015), Available: |
| [54] | He, K.; Zhang, X.; Ren, S.; Sun, J., Deep residual learning for image recognition, (Proceedings of the IEEE conference on computer vision and pattern recognition, 2016), 770-778, Available: |
| [55] | Han, W.; Liu, W. K.; Wagner, G. J., Convergence analysis of a hierarchical enrichment of Dirichlet boundary conditions in a mesh-free method, Int. J. Numer. Methods Eng., 53, 6, 1323-1336, 2002, Feb. · Zbl 0995.65108 |
| [56] | Chen, J. S.; Han, W.; You, Y.; Meng, X., A reproducing kernel method with nodal interpolation property, Int. J. Numer. Methods Eng., 56, 935-960, 2003 · Zbl 1106.74424 |
| [57] | Weinan, E.; Wang, Q., Exponential convergence of the deep neural network approximation for analytic functions, Sci. China Math., 61, 10, 1733-1740, 2018 · Zbl 1475.65007 |
| [58] | Chen, J. S.; Wu, C. T.; Yoon, S.; You, Y., Stabilized conforming nodal integration for Galerkin mesh-free methods, Int. J. Numer. Methods Eng., 50, 2, 435-466, 2001 · Zbl 1011.74081 |
| [59] | You, Y.; Chen, J. S.; Lu, H., Filters, reproducing kernel, and adaptive meshfree method, Comput. Mech., 31, 3, 316-326, 2003 · Zbl 1038.74681 |
| [60] | Lu, H.; Chen, J. S., Adaptive Galerkin Particle Method, Meshfree Methods for Partial Differential Equations, 251-265, 2003, Springer: Springer Berlin Heidelberg · Zbl 1090.65547 |
| [61] | Wang, Y.; Baek, J.; Tang, Y.; Du, J.; Hillman, M.; Chen, J. S., Support vector machine guided reproducing kernel particle method for image-based modeling of microstructures, Comput. Mech., 2023, 1-36, Oct. 2023 |
| [62] | D.P. Kingma and J.L. Ba, “Adam: a method for stochastic optimization,” 2015, Available: https://arxiv.org/abs/1412.6980v9. |
| [63] | Hughes, T. J., The Finite Element Method: Linear Static and Dynamic Finite Element Analysis, 2012, Courier Corporation |
| [64] | Kirsch, E. G., Die Theorie der Elastizit t und die bed Rfnisse der Festigkeitslehre, Zeitshrift des Vereines Dtsch. Ingenieure, 42, 797-807, 1898 |
| [65] | Kurtz, J.; Demkowicz, L., A fully automatic hp-adaptivity for elliptic PDEs in three dimensions, Comput. Methods Appl. Mech. Eng., 196, 37-40, 3534-3545, 2007, Aug. · Zbl 1173.65357 |
| [66] | Rachowicz, W.; Oden, J. T.; Demkowicz, L., Toward a universal h-p adaptive finite element strategy part 3. design of h-p meshes, Comput. Methods Appl. Mech. Eng., 77, 1-2, 181-212, 1989 · Zbl 0723.73076 |
| [67] | Chen, J. S.; Pan, C.; Roque, C. M.O. L.; Wang, H. P., A Lagrangian reproducing kernel particle method for metal forming analysis, Comput. Mech., 22, 3, 289-307, 1998 · Zbl 0928.74115 |
| [68] | Hu, H. Y.; Chen, J. S.; Hu, W., Error analysis of collocation method based on reproducing kernel approximation, Numer. Methods Partial Differ. Equ., 27, 3, 554-580, 2011 · Zbl 1223.65091 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
