×

AI-enhanced iterative solvers for accelerating the solution of large-scale parametrized systems. (English) Zbl 1532.65122

Summary: Recent advances in the field of machine learning open a new era in high performance computing for challenging computational science and engineering applications. In this framework, the use of advanced machine learning algorithms for the development of accurate and cost-efficient surrogate models of complex physical processes has already attracted major attention from scientists. However, despite their powerful approximation capabilities, surrogate model predictions are still far from being near to the ‘exact’ solution of the problem. To address this issue, the present work proposes the use of up-to-date machine learning tools in order to equip a new generation of iterative solvers of linear equation systems, capable of very efficiently solving large-scale parametrized problems at any desired level of accuracy. The proposed approach consists of the following two steps. At first, a reduced set of model evaluations is performed using a standard finite element methodology and the corresponding solutions are used to establish an approximate mapping from the problem’s parametric space to its solution space using a combination of deep feedforward neural networks and convolutional autoencoders. This mapping serves as a means of obtaining very accurate initial predictions of the system’s response to new query points at negligible computational cost. Subsequently, an iterative solver inspired by the Algebraic Multigrid method in combination with Proper Orthogonal Decomposition, termed POD-2G, is developed that successively refines the initial predictions of the surrogate model towards the exact solution. The application of POD-2G as a standalone solver or as preconditioner in the context of preconditioned conjugate gradient methods is demonstrated on several numerical examples of large scale systems, with the results indicating its strong superiority over conventional iterative solution schemes.
© 2023 John Wiley & Sons, Ltd.

MSC:

65N99 Numerical methods for partial differential equations, boundary value problems
65F10 Iterative numerical methods for linear systems
65N30 Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs
65N55 Multigrid methods; domain decomposition for boundary value problems involving PDEs
68T07 Artificial neural networks and deep learning

Software:

TensorFlow

References:

[1] BarrettR, BerryM, ChanTF, et al. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. 2nd ed.SIAM; 1994.
[2] BenziM, CullumJK, TumaM. Robust approximate inverse preconditioning for the conjugate gradient method. SIAM J Sci Comput. 2000;22(4):1318‐1332. doi:10.1137/S1064827599356900 · Zbl 0985.65035
[3] LinFR, YangSW, JinXQ. Preconditioned iterative methods for fractional diffusion equation. J Comput Phys. 2014;256:109‐117. doi:10.1016/j.jcp.2013.07.040 · Zbl 1349.65314
[4] HerzogR, SachsE. Preconditioned conjugate gradient method for optimal control problems with control and state constraints. SIAM J Matrix Anal Appl. 2010;31(5):2291‐2317. doi:10.1137/090779127 · Zbl 1209.49038
[5] SaadY, SchultzMH. GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J Sci Stat Comput. 1986;7(3):856‐869. doi:10.1137/0907058 · Zbl 0599.65018
[6] ShakibF, HughesTJ, JohanZ. A multi‐element group preconditioned GMRES algorithm for nonsymmetric systems arising in finite element analysis. Comput Methods Appl Mech Eng. 1989;75(1‐3):415‐456. doi:10.1016/0045‐7825(89)90040‐6 · Zbl 0687.76065
[7] BaglamaJ, CalvettiD, GolubG, ReichelL. Adaptively preconditioned gmres algorithms. SIAM J Sci Comput. 1998;20(1):243‐269. doi:10.1137/S1064827596305258 · Zbl 0954.65026
[8] ConcusP, GolubGH, MeurantG. Block preconditioning for the conjugate gradient method. SIAM J Sci Stat Comput. 1985;6(1):220‐252. doi:10.1137/0906018 · Zbl 0556.65022
[9] ToselliA, WidlundOB. Domain Decomposition Methods: Algorithms and Theory. Springer; 2005. · Zbl 1069.65138
[10] TallecP, RoeckY, VidrascuM. Domain decomposition methods for large linearly elliptic three‐dimensional problems. J Comput Appl Math. 1991;34(1):93‐117. doi:10.1016/0377‐0427(91)90150‐I · Zbl 0719.65083
[11] FarhatC, RouxFX. A method of finite element tearing and interconnecting and its parallel solution algorithm. Int J Numer Methods Eng. 1991;32(6):1205‐1227. doi:10.1002/nme.1620320604 · Zbl 0758.65075
[12] FarhatC, MandelJ, RouxFX. Optimal convergence properties of the FETI domain decomposition method. Comput Methods Appl Mech Eng. 1994;115(3):365‐385. doi:10.1016/0045‐7825(94)90068‐X
[13] FragakisY, PapadrakakisM. The mosaic of high performance domain decomposition methods for structural mechanics: formulation, interrelation and numerical efficiency of primal and dual methods. Comput Methods Appl Mech Eng. 2003;192:3799‐3830. · Zbl 1054.74069
[14] CaiXC, SarkisM. A restricted additive Schwarz preconditioner for general sparse linear systems. SIAM J Sci Comput. 1999;21(2):792‐797. doi:10.1137/S106482759732678X · Zbl 0944.65031
[15] Al DaasH, GrigoriL, JolivetP, TournierPH. A multilevel Schwarz preconditioner based on a hierarchy of robust coarse spaces. SIAM J Sci Comput. 2021;43(3):A1907‐A1928. doi:10.1137/19M1266964 · Zbl 1512.65289
[16] TrottenbergU, OosteleeC, SchullerA. Multigrid 1st Edition. Academic Press; 2000.
[17] IwamuraC, CostaFS, SbarskiI, EastonA, LiN. An efficient algebraic multigrid preconditioned conjugate gradient solver. Comput Methods Appl Mech Eng. 2003;192(20‐21):2299‐2318. doi:10.1016/S0045‐7825(02)00378‐X · Zbl 1045.74047
[18] HeysJ, ManteuffelT, McCormickSF, OlsonL. Algebraic multigrid for higher‐order finite elements. J Comput Phys. 2005;204(2):520‐532. doi:10.1016/j.jcp.2004.10.021 · Zbl 1060.65673
[19] LangerU, PuschD, ReitzingerS. Efficient preconditioners for boundary element matrices based on grey‐box algebraic multigrid methods. Int J Numer Methods Eng. 2003;58(13):1937‐1953. doi:10.1002/nme.839 · Zbl 1035.65144
[20] RamageA. A multigrid preconditioner for stabilised discretisations of advection‐diffusion problems. J Comput Appl Math. 1999;110(1):187‐203. doi:10.1016/S0377‐0427(99)00234‐4 · Zbl 0939.65135
[21] WienandsR, OosterleeCW, WashioT. Fourier analysis of GMRES(m) preconditioned by multigrid. SIAM J Sci Comput. 2000;22(2):582‐603. doi:10.1137/S1064827599353014 · Zbl 0967.65101
[22] VakiliS, DarbandiM. Recommendations on enhancing the efficiency of algebraic multigrid preconditioned GMRES in solving coupled fluid flow equations. Numer Heat Transf Part B: Fund. 2009;55(3):232‐256. doi:10.1080/10407790802628879
[23] ZahmO, NouyA. Interpolation of inverse operators for preconditioning parameter‐dependent equations. SIAM J Sci Comput. 2016;38(2):A1044‐A1074. doi:10.1137/15M1019210 · Zbl 1382.65035
[24] BergamaschiL. A survey of low‐rank updates of preconditioners for sequences of symmetric linear systems. Algorithms. 2020;13(4). doi:10.3390/A13040100
[25] CarrA, SturlerE, GugercinS. Preconditioning parametrized linear systems. SIAM J Sci Comput. 2021;43(3):A2242‐A2267. doi:10.1137/20M1331123 · Zbl 1512.65043
[26] StavroulakisG, GiovanisDG, PapadrakakisM, PapadopoulosV. A new perspective on the solution of uncertainty quantification and reliability analysis of large‐scale problems. Comput Methods Appl Mech Eng. 2014;276:627‐658. doi:10.1016/j.cma.2014.03.009 · Zbl 1425.65012
[27] SaadY. Analysis of augmented Krylov subspace methods. SIAM J Matrix Anal Appl. 1997;18(2):435‐449. doi:10.1137/S0895479895294289 · Zbl 0871.65026
[28] SturlerE. Truncation strategies for optimal Krylov subspace methods. SIAM J Numer Anal. 1999;36(3):864‐889. doi:10.1137/S0036142997315950 · Zbl 0960.65031
[29] PanagiotopoulosD, DeckersE, DesmetW. Krylov subspaces recycling based model order reduction for acoustic BEM systems and an error estimator. Comput Methods Appl Mech Eng. 2020;359. doi:10.1016/j.cma.2019.112755 · Zbl 1441.76083
[30] ChapmanA, SaadY. Deflated and augmented Krylov subspace techniques. Numer Linear Algebra Appl. 1997;4(1):43‐66. doi:10.1002/(SICI)1099‐1506(199701/02)4:1<43::AID‐NLA99>3.0.CO;2‐Z · Zbl 0889.65028
[31] SaadY, YeungM, ErhelJ, Guyomarc’hF. A deflated version of the conjugate gradient algorithm. SIAM J Sci Comput. 2000;21(5):1909‐1926. doi:10.1137/S1064829598339761 · Zbl 0955.65021
[32] DaasH, GrigoriL, HénonP, RicouxP. Recycling Krylov subspaces and truncating deflation subspaces for solving sequence of linear systems. ACM Trans Math Softw. 2021;47(2). doi:10.1145/3439746 · Zbl 07467973
[33] PapadrakakisM, PapadopoulosV, LagarosND. Structural reliability analyis of elastic‐plastic structures using neural networks and Monte Carlo simulation. Comput Methods Appl Mech Eng. 1996;136(1‐2):145‐163. doi:10.1016/0045‐7825(96)01011‐0 · Zbl 0893.73079
[34] PapadrakakisM, LagarosND. Reliability‐based structural optimization using neural networks and Monte Carlo simulation. Comput Methods Appl Mech Eng. 2002;191(32):3491‐3507. doi:10.1016/S0045‐7825(02)00287‐6 · Zbl 1101.74377
[35] SeyhanAT, TayfurG, KarakurtM, TanogluM. Artificial neural network (ANN) prediction of compressive strength of VARTM processed polymer composites. Comput Mater Sci. 2005;34(1):99‐105. doi:10.1016/j.commatsci.2004.11.001
[36] Hosni ElhewyA, MesbahiE, PuY. Reliability analysis of structures using neural network method. Probab Eng Mech. 2006;21(1):44‐53. doi:10.1016/j.probengmech.2005.07.002
[37] ChojaczykA, TeixeiraA, NevesL, CardosoJ, GuedesSC. Review and application of artificial neural networks models in reliability analysis of steel structures. Struct Safe. 2015;52:78‐89. doi:10.1016/j.strusafe.2014.09.002
[38] NikolopoulosS, KalogerisI, PapadopoulosV. Non‐intrusive surrogate modeling for parametrized time‐dependent partial differential equations using convolutional autoencoders. Eng Appl Artif Intel. 2022;109:104652. doi:10.1016/j.engappai.2021.104652
[39] NikolopoulosS, KalogerisI, PapadopoulosV. Machine learning accelerated transient analysis of stochastic nonlinear structures. Eng Struct. 2022;257:114020. doi:10.1016/j.engstruct.2022.114020
[40] XuJ, DuraisamyK. Multi‐level convolutional autoencoder networks for parametric prediction of spatio‐temporal dynamics. Comput Methods Appl Mech Eng. 2020;372:113379. doi:10.1016/j.cma.2020.113379 · Zbl 1506.68111
[41] FrescaS, DedeL, ManzoniA. A comprehensive deep learning‐based approach to reduced order Modeling of nonlinear time‐dependent parametrized PDEs. J Sci Comput. 2021;87(61). doi:10.1007/s10915‐021‐01462‐7 · Zbl 1470.65166
[42] GruberA, GunzburgerM, JuL, WangZ. A comparison of neural network architectures for data‐driven reduced‐order modeling. Comput Methods Appl Mech Eng. 2022;393. doi:10.1016/j.cma.2022.114764 · Zbl 1507.65157
[43] LeeK, CarlbergKT. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J Comput Phys. 2020;404. doi:10.1016/j.jcp.2019.108973 · Zbl 1454.65184
[44] SanO, MaulikR, AhmedM. An artificial neural network framework for reduced order modeling of transient flows. Commun Nonlinear Sci Numer Simul. 2019;77:271‐287. doi:10.1016/j.cnsns.2019.04.025 · Zbl 1479.76082
[45] YuY, YaoH, LiuY. Aircraft dynamics simulation using a novel physics‐based learning method. Aerosp Sci Technol. 2019;87:254‐264. doi:10.1016/j.ast.2019.02.021
[46] ZhouJM, DongL, GuanW, YanJ. Impact load identification of nonlinear structures using deep recurrent neural network. Mech Syst Signal Process. 2019;133. doi:10.1016/j.ymssp.2019.106292
[47] GonzalezFJ, BalajewiczM. Deep convolutional recurrent autoencoders for learning low‐dimensional feature dynamics of fluid systems. arXiv. 2018.
[48] CarlbergK, FarhatC. A low‐cost, goal‐oriented ‘compact proper orthogonal decomposition’ basis for model reduction of static systems. Int J Numer Methods Eng. 2011;86(3):381‐402. doi:10.1002/nme.3074 · Zbl 1235.74352
[49] ZahrMJ, AveryP, FarhatC. A multilevel projection‐based model order reduction framework for nonlinear dynamic multiscale problems in structural and solid mechanics. Int J Numer Methods Eng. 2017;112(8):855‐881. doi:10.1002/nme.5535 · Zbl 07867235
[50] AgathosK, BordasSPA, ChatziE. Parametrized reduced order modeling for cracked solids. Int J Numer Methods Eng. 2020;121(20):4537‐4565. doi:10.1002/nme.6447 · Zbl 07863230
[51] ChinestaF, AmmarA, CuetoE. Proper generalized decomposition of multiscale models. Int J Numer Methods Eng. 2010;83(8‐9):1114‐1132. doi:10.1002/nme.2794 · Zbl 1197.76093
[52] LadevèzeP, PassieuxJC, NéronD. The LATIN multiscale computational method and the proper generalized decomposition. Comput Methods Appl Mech Eng. 2010;199(21‐22):1287‐1296. doi:10.1016/j.cma.2009.06.023 · Zbl 1227.74111
[53] LadevèzeP, ChamoinL. On the verification of model reduction methods based on the proper generalized decomposition. Comput Methods Appl Mech Eng. 2011;200(23‐24):2032‐2047. doi:10.1016/j.cma.2011.02.019 · Zbl 1228.76089
[54] Dal SantoN, DeparisS, PegolottiL. Data driven approximation of parametrized PDEs by reduced basis and neural networks. J Comput Phys. 2020;416:109550. doi:10.1016/j.jcp.2020.109550 · Zbl 1437.65224
[55] SalvadorM, DedèL, ManzoniA. Non intrusive reduced order modeling of parametrized PDEs by kernel POD and neural networks. Comput Math Appl. 2021;104:1‐13. doi:10.1016/j.camwa.2021.11.001 · Zbl 1538.65448
[56] KalogerisI, PapadopoulosV. Diffusion maps‐aided neural networks for the solution of parametrized PDEs. Comput Methods Appl Mech Eng. 2021;376:113568. doi:10.1016/j.cma.2020.113568 · Zbl 1506.65180
[57] SantosKR, GiovanisDG, ShieldsMD. Grassmannian diffusion maps-based dimension reduction and classification for high‐dimensional data. SIAM J Sci Comput. 2022;44(2):B250‐B274. doi:10.1137/20M137001X · Zbl 1489.53136
[58] KadeethumT, BallarinF, ChoiY, O’MalleyD, YoonH, BouklasN. Non‐intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniques. Adv Water Resour. 2022;160. doi:10.1016/j.advwatres.2021.104098
[59] VlachasK, TatsisK, AgathosK, BrinkAR, ChatziE. A local basis approximation approach for nonlinear parametric model order reduction. J Sound Vib. 2021;502:116055. doi:10.1016/j.jsv.2021.116055
[60] HeaneyCE, WolffsZ, TómassonJA, et al. An AI‐based non‐intrusive reduced‐order model for extended domains applied to multiphase flow in pipes. Phys Fluids. 2022;34(5):055111. doi:10.1063/5.0088070
[61] LazzaraM, ChevalierM, ColomboM, Garay GarciaJ, LapeyreC, TesteO. Surrogate modelling for an aircraft dynamic landing loads simulation using an LSTM AutoEncoder‐based dimensionality reduction approach. Aerosp Sci Technol. 2022;126:107629. doi:10.1016/j.ast.2022.107629
[62] FrescaS, ManzoniA. POD‐DL‐ROM: enhancing deep learning‐based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition. Comput Methods Appl Mech Eng. 2022;388. doi:10.1016/j.cma.2021.114181 · Zbl 1507.65181
[63] CarlbergK, ForstallV, TuminaroR. Krylov‐subspace recycling via the POD‐augmented conjugate‐gradient method. SIAM J Matrix Anal Appl. 2016;37(3):1304‐1336. doi:10.1137/16M1057693 · Zbl 1348.15005
[64] HeinleinA, KlawonnA, LanserM, WeberJ. Machine learning in adaptive domain decomposition methods—predicting the geometric location of constraints. SIAM J Sci Comput. 2019;41(6):A3887‐A3912. doi:10.1137/18M1205364 · Zbl 1429.65065
[65] ChenY, DongB, XuJ. Meta‐MgNet: meta multigrid networks for solving parameterized partial differential equations. J Comput Phys. 2022;455:110996. doi:10.1016/j.jcp.2022.110996 · Zbl 07518081
[66] LuzI, GalunM, MaronH, BasriR, YavnehI. Learning algebraic multigrid using graph neural networks. PMLR. 2020 6489‐6499.
[67] StübenK. A review of algebraic multigrid. J Comput Appl Math. 2001;128(1):281‐309. doi:10.1016/S0377‐0427(00)00516‐1 · Zbl 0979.65111
[68] BrezinaM, ClearyAJ, FalgoutRD, et al. Algebraic multigrid based on element interpolation (AMGe). SIAM J Sci Comput. 2001;22(5):1570‐1592. doi:10.1137/S1064827598344303 · Zbl 0991.65133
[69] TreisterE, YavnehI. Square and stretch multigrid for stochastic matrix eigenproblems. Numer Linear Algebra Appl. 2010;17(2‐3):229‐251. doi:10.1002/nla.708 · Zbl 1240.65124
[70] NapovA, NotayY. An efficient Multigrid method for graph Laplacian systems. Electron Trans Numer Anal. 2016;45:201‐218. · Zbl 1347.65059
[71] FaccaE, BenziM. Fast iterative solution of the optimal transport problem on graphs. SIAM J Sci Comput. 2021;43(3):A2295‐A2319. doi:10.1137/20M137015X · Zbl 1512.65103
[72] RugeJW, StübenK. Algebraic Multigridch. 4. Algebraic Multigrid: 73‐130. Frontiers in Applied Mathematics. SIAM; 1987.
[73] VanekP, MandelJ, BrezinaM. Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems. Comput Secur. 1996;56(3):179‐196. doi:10.1007/BF02238511 · Zbl 0851.65087
[74] MoS, ZhuY, ZabarasN, ShiX, WuJ. Deep convolutional encoder‐decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media. Water Resour Res. 2019;55(1):703‐728. doi:10.1029/2018WR023528
[75] PasettoD, FerronatoM, PuttiM. A reduced order model‐based preconditioner for the efficient solution of transient diffusion equations. Int J Numer Methods Eng. 2017;109(8):1159‐1179. doi:10.1002/nme.5320 · Zbl 07874340
[76] Dal SantoN, DeparisS, ManzoniA, QuarteroniA. Multi space reduced basis preconditioners for large‐scale parametrized PDEs. SIAM J Sci Comput. 2018;40(2):A954‐A983. doi:10.1137/16M1089149 · Zbl 1453.65416
[77] Diaz CortesG, VuikC, JansenJ. On POD‐based deflation vectors for DPCG applied to porous media problems. J Comput Appl Math. 2018;330:193‐213. doi:10.1016/j.cam.2017.06.032 · Zbl 1432.76253
[78] OlssonAM, SandbergGE. Latin hypercube sampling for stochastic finite element analysis. J Eng Mech. 2002;128(1):121‐125.
[79] RathinamM, PetzoldLR. A new look at proper orthogonal decomposition. SIAM J Numer Anal. 2003;41(5):1893‐1925. doi:10.1137/S0036142901389049 · Zbl 1053.65106
[80] LieuT, FarhatC, LesoinneM. Reduced‐order fluid/structure modeling of a complete aircraft configuration. Comput Methods Appl Mech Eng. 2006;195(41):5730‐5742. doi:10.1016/j.cma.2005.08.026 · Zbl 1124.76042
[81] RapúnML, VegaJM. Reduced order models based on local POD plus Galerkin projection. J Comput Phys. 2010;229(8):3046‐3063. doi:10.1016/j.jcp.2009.12.029 · Zbl 1187.65111
[82] AbadiM, BarhamP, ChenJ, et al. Tensorflow: a system for large‐scale machine learning. Paper presented at: 12th USENIX Symposium on Operating Systems Design and Implementation. 2016 265-283.
[83] CarcioneJM. Chapter 7‐Biot theory for porous media. Wave Fields in Real Media (Third Edition). Elsevier; 2015.
[84] BergamaschiL, FerronatoM, GambolatiG. Novel preconditioners for the iterative solution to FE‐discretized coupled consolidation equations. Comput Methods Appl Mech Eng. 2007;196(25‐28):2647‐2656. doi:10.1016/j.cma.2007.01.013 · Zbl 1173.76330
[85] LeeJJ, MardalKA, WintherR. Parameter‐robust discretization and preconditioning of Biot’s consolidation model. SIAM J Sci Comput. 2017;39(1):A1‐A24. doi:10.1137/15M1029473 · Zbl 1381.76183
[86] CastellettoN, KlevtsovS, HajibeygiH, TchelepiH. Multiscale two‐stage solver for Biot’s poroelasticity equations in subsurface media. Comput Geosci. 2019;23(2):207‐224. doi:10.1007/s10596‐018‐9791‐z · Zbl 1414.76053
[87] GasparFJ, RodrigoC. On the fixed‐stress split scheme as smoother in multigrid methods for coupling flow and geomechanics. Comput Methods Appl Mech Eng. 2017;326:526‐540. doi:10.1016/j.cma.2017.08.025 · Zbl 1439.74413
[88] AxelssonO, KarátsonJ. Superlinearly convergent CG methods via equivalent preconditioning for nonsymmetric elliptic operators. Numer Math. 2004;99(2):197‐223. doi:10.1007/s00211‐004‐0557‐2 · Zbl 1061.65043
[89] KarátsonJ, KuricsT. Superlinearly convergent PCG algorithms for some nonsymmetric elliptic systems. J Comput Appl Math. 2008;212(2):214‐230. doi:10.1016/j.cam.2006.12.004 · Zbl 1151.65082
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.