3D elastic wave propagation with a factorized Fourier neural operator (F-FNO). (English) Zbl 1536.74102
Summary: Numerical simulations are computationally demanding in three-dimensional (3D) settings but they are often required to accurately represent physical phenomena. Neural operators have emerged as powerful surrogate models to alleviate the computational costs of simulations. However, neural operators applications in 3D remain sparse, mainly due to the difficulty of obtaining training databases for supervised learning and the size of 3D neural operators that poses memory challenges. This work focuses on the propagation of elastic waves in 3D domains and showcases the Factorized Fourier Neural Operator (F-FNO) as an efficient and accurate surrogate model. The F-FNO is trained on the publicly available HEMEW-3D database of 30 000 wavefields simulations in realistic heterogeneous domains. The F-FNO predicts space- and time-dependent (3D) surface wavefields depending on the characteristics of the propagation domain (characterized by the velocity of shear waves). Four FNO variants are compared and extensive investigations on the influence of hyperparameters and training strategies are conducted. The two most influential hyperparameters are the number of layers and the number of channels, meaning that richer models are more accurate. On the contrary, increasing the number of Fourier modes had little influence and did not reduce the spectral bias that causes an underestimation of high-frequency patterns. The F-FNO is sensitive to heterogeneities in the inputs but robust to the addition of noise. Additionally, it possesses good generalization ability to out-of-distribution data and transfer learning is very beneficial to improve the predictions in tailored applications.
MSC:
| 74J20 | Wave scattering in solid mechanics |
| 65M70 | Spectral, collocation and related methods for initial value and initial-boundary value problems involving PDEs |
References:
| [1] | Bonev, B.; Kurth, T.; Hundt, C.; Pathak, J.; Baust, M.; Kashinath, K., Modelling Atmospheric Dynamics with Spherical Fourier Neural Operators, URL https://www.climatechange.ai/papers/iclr2023/47 |
| [2] | Pathak, J.; Subramanian, S.; Harrington, P.; Raja, S.; Chattopadhyay, A.; Mardani, M.; Kurth, T.; Hall, D.; Li, Z.; Azizzadenesheli, K.; Hassanzadeh, P.; Kashinath, K.; Anandkumar, A., FourCastNet: A global data-driven high-resolution weather model using adaptive Fourier neural operators (2022), arXiv. arXiv:2202.11214 |
| [3] | Chu, A. K.; Benson, S. M.; Wen, G., Deep-learning-based flow prediction for CO2 storage in shale-sandstone formations. Energies, 1 (2023) |
| [4] | Wen, G.; Li, Z.; Long, Q.; Azizzadenesheli, K.; Anandkumar, A.; Benson, S., Real-time high-resolution co2 geological storage prediction using nested fourier neural operators. Energy & Environmental Science, 1732-1741 (2023) |
| [5] | Witte, P. A.; Konuk, T.; Skjetne, E.; Chandra, R., Fast CO2 saturation simulations on large-scale geomodels with artificial intelligence-based wavelet neural operators. Int. J. Greenh. Gas Control (2023) |
| [6] | Goswami, S.; Yin, M.; Yu, Y.; Karniadakis, G. E., A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials. Comput. Methods Appl. Mech. Engrg. (2022) · Zbl 1507.74383 |
| [7] | Rashid, M. M.; Pittie, T.; Chakraborty, S.; Krishnan, N. A., Learning the stress-strain fields in digital composites using Fourier neural operator. iScience, 11 (2022) |
| [8] | Yang, Y.; Gao, A. F.; Azizzadenesheli, K.; Clayton, R. W.; Ross, Z. E., Rapid seismic waveform modeling and inversion with neural operators. IEEE Trans. Geosci. Remote Sens., 1 (2023) |
| [9] | Zhang, T.; Trad, D.; Innanen, K., Learning to solve the elastic wave equation with Fourier neural operators. Geophysics, 1-63 (2023) |
| [10] | Clark Di Leoni, P.; Lu, L.; Meneveau, C.; Karniadakis, G. E.; Zaki, T. A., Neural operator prediction of linear instability waves in high-speed boundary layers. J. Comput. Phys. (2023) · Zbl 07640550 |
| [11] | Peng, W.; Yuan, Z.; Li, Z.; Wang, J., Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence. Phys. Fluids, 1 (2023) |
| [12] | Grady, T. J.; Khan, R.; Louboutin, M.; Yin, Z.; Witte, P. A.; Chandra, R.; Hewett, R. J.; Herrmann, F. J., Model-parallel Fourier neural operators as learned surrogates for large-scale parametric PDEs. Comput. Geosci. (2023) |
| [13] | Kong, Q.; Rodgers, A., Feasibility of Using Fourier Neural Operators for 3D Elastic Seismic SimulationsTechnical Report LLNL-TR-854521 (2023), Lawrence Livermore National Laboratory, URL https://www.osti.gov/servlets/purl/2001189 |
| [14] | Lehmann, F.; Gatti, F.; Bertin, M.; Clouteau, D., Fourier neural operator surrogate model to predict 3D seismic waves propagation |
| [15] | Li, Z.; Peng, W.; Yuan, Z.; Wang, J., Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator. Phys. Fluids, 7 (2023) |
| [16] | Li, Z.; Kovachki, N. B.; Choy, C.; Li, B.; Kossaifi, J.; Otta, S. P.; Nabian, M. A.; Stadler, M.; Hundt, C.; Azizzadenesheli, K.; Anandkumar, A., Geometry-informed neural operator for large-scale 3D PDEs (2023), arXiv. arXiv:2309.00583 |
| [17] | Moczo, P.; Kristek, J.; Bard, P.-Y.; Stripajová, S.; Hollender, F.; Chovanová, Z.; Kristeková, M.; Sicilia, D., Key structural parameters affecting earthquake ground motion in 2D and 3D sedimentary structures. Bull. Earthq. Eng., 6, 2421-2450 (2018) |
| [18] | Smerzini, C.; Paolucci, R.; Stupazzini, M., Comparison of 3D, 2D and 1D numerical approaches to predict long period earthquake ground motion in the Gubbio plain, Central Italy. Bull. Earthq. Eng., 6, 2007-2029 (2011) |
| [19] | Zhu, C.; Riga, E.; Pitilakis, K.; Zhang, J.; Thambiratnam, D., Seismic aggravation in shallow basins in addition to one-dimensional site amplification. J. Earthq. Eng., 9, 1477-1499 (2020) |
| [20] | Bloemheuvel, S.; Van Den Hoogen, J.; Jozinović, D.; Michelini, A.; Atzmueller, M., Graph neural networks for multivariate time series regression with application to seismic data. Int. J. Data Sci. Anal., 3, 317-332 (2023) |
| [21] | Sarkar, S.; Kumar, S.; Roy, A.; Das, B., Multilayer perceptron based early on-site estimation of PGA during an earthquake, 313-326, URL https://link.springer.com/10.1007/978-981-19-5936-3_29 |
| [22] | Clements, T.; Denolle, M. A., Tracking groundwater levels using the ambient seismic field. Geophys. Res. Lett., 13, 6459-6465 (2018) |
| [23] | Hansen, D.; Maddix, D. C.; Alizadeh, S.; Gupta, G.; Mahoney, M. W., Learning Physical Models that Can Respect Conservation Laws (2023), arXiv. arXiv:2302.11002 |
| [24] | Raissi, M.; Perdikaris, P.; Karniadakis, G., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 686-707 (2019) · Zbl 1415.68175 |
| [25] | Ding, Y.; Chen, S.; Li, X.; Wang, S.; Luan, S.; Sun, H., Self-adaptive physics-driven deep learning for seismic wave modeling in complex topography. Eng. Appl. Artif. Intell. (2023) |
| [26] | Karimpouli, S.; Tahmasebi, P., Physics informed machine learning: Seismic wave equation. Geosci. Front., 6, 1993-2001 (2020) |
| [27] | Moseley, B.; Markham, A.; Nissen-Meyer, T., Solving the wave equation with physics-informed deep learning (2020) |
| [28] | Rasht-Behesht, M.; Huber, C.; Shukla, K.; Karniadakis, G. E., Physics-informed neural networks (PINNs) for wave propagation and full waveform inversions. J. Geophys. Res. Solid Earth, 5 (2022) |
| [29] | Ren, P.; Rao, C.; Chen, S.; Wang, J. X.; Sun, H.; Liu, Y., SeismicNet: Physics-informed neural networks for seismic wave modeling in semi-infinite domain. Comput. Phys. Comm. (2024) · Zbl 1532.74142 |
| [30] | Song, C.; Liu, Y.; Zhao, P.; Zhao, T.; Zou, J.; Liu, C., Simulating multi-component elastic seismic wavefield using deep learning. IEEE Geosci. Remote Sens. Lett., 1 (2023) |
| [31] | Wu, Y.; Aghamiry, H. S.; Operto, S.; Ma, J., Helmholtz equation solution in non-smooth media by physics-informed neural network with incorporating quadratic terms and a perfectly matching layer condition. Geophysics, 1-66 (2023) |
| [32] | Aldirany, Z.; Cottereau, R.; Laforest, M.; Prudhomme, S., Operator approximation of the wave equation based on deep learning of Green’s function (2023) |
| [33] | Wei, W.; Fu, L. Y., Small-data-driven fast seismic simulations for complex media using physics-informed Fourier neural operators. Geophysics, 6, T435-T446 (2022) |
| [34] | Ovadia, O.; Turkel, E.; Kahana, A.; Karniadakis, G. E., DiTTO: Diffusion-inspired Temporal Transformer Operator (2023), arXiv:2307.09072 |
| [35] | Benitez, J. A.L.; Furuya, T.; Faucher, F.; Kratsios, A.; Tricoche, X.; de Hoop, M. V., Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation (2023), arXiv:2301.11509 |
| [36] | Yang, Y.; Gao, A. F.; Castellanos, J. C.; Ross, Z. E.; Azizzadenesheli, K.; Clayton, R. W., Seismic wave propagation and inversion with neural operators. Seism. Rec., 3, 126-134 (2021) |
| [37] | O’Brien, G. S.; Bean, C. J.; Meiland, H.; Witte, P., Imaging and seismic modelling inside volcanoes using machine learning. Sci. Rep., 1, 630 (2023) |
| [38] | Deng, C.; Feng, S.; Wang, H.; Zhang, X.; Jin, P.; Feng, Y.; Zeng, Q.; Chen, Y.; Lin, Y., OpenFWI: Large-scale multi-structural benchmark datasets for full waveform inversion, 6007-6020, URL https://proceedings.neurips.cc/paper_files/paper/2022/file/27d3ef263c7cb8d542c4f9815a49b69b-Paper-Datasets_and_Benchmarks.pdf |
| [39] | Garg, S.; Chakraborty, S., Randomized prior wavelet neural operator for uncertainty quantification (2023), arXiv. arXiv:2302.01051 |
| [40] | Li, B.; Wang, H.; Yang, X.; Lin, Y., Solving seismic wave equations on variable velocity models with Fourier neural operator (2022), arXiv. arXiv:2209.12340 |
| [41] | Li, Z.; Kovachki, N. B.; Azizzadenesheli, K.; liu, B.; Bhattacharya, K.; Stuart, A.; Anandkumar, A., Fourier neural operator for parametric partial differential equations |
| [42] | Ovadia, O.; Kahana, A.; Stinis, P.; Turkel, E.; Karniadakis, G. E., ViTO: Vision transformer-operator (2023), arXiv. arXiv:2303.08891 |
| [43] | Zhu, M.; Feng, S.; Lin, Y.; Lu, L., Fourier-DeepONet: Fourier-enhanced deep operator networks for full waveform inversion with improved accuracy, generalizability, and robustness. Comput. Methods Appl. Mech. Engrg. (2023) · Zbl 1539.86010 |
| [44] | Lehmann, F., Physics-Based Simulations of 3D Wave Propagation: A Dataset for Scientific Machine Learning (2023), Recherche Data Gouv |
| [45] | Rahman, M. A.; Ross, Z. E.; Azizzadenesheli, K., U-NO: U-shaped neural operators. Transactions on Machine Learning Research (2023), URL https://openreview.net/forum?id=j3oQF9coJd |
| [46] | Lu, L.; Jin, P.; Pang, G.; Zhang, Z.; Karniadakis, G. E., Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat. Mach. Intell., 3, 218-229 (2021) |
| [47] | Brandstetter, J.; Worrall, D.; Welling, M., Message Passing Neural PDE Solvers, URL https://openreview.net/forum?id=vSix3HPYKSU |
| [48] | Fanaskov, V.; Oseledets, I., Spectral neural operators (2022), arXiv. arXiv:2205.10573 |
| [49] | Tripura, T.; Chakraborty, S., Wavelet Neural Operator for solving parametric partial differential equations in computational mechanics problems. Comput. Methods Appl. Mech. Engrg. (2023) · Zbl 07644856 |
| [50] | Raonic, B.; Molinaro, R.; Rohner, T.; Mishra, S.; de Bezenac, E., Convolutional neural operators, URL https://openreview.net/forum?id=GT8p40tiVlB |
| [51] | Ronneberger, O.; Fischer, P.; Brox, T., U-Net: convolutional networks for biomedical image segmentation, 234-241 |
| [52] | Helwig, J.; Zhang, X.; Fu, C.; Kurtin, J.; Wojtowytsch, S.; Ji, S., Group equivariant Fourier neural operators for partial differential equations, URL https://openreview.net/pdf?id=kgAOY5x4fi |
| [53] | Tran, A.; Mathews, A.; Xie, L.; Ong, C. S., Factorized Fourier neural operators, URL https://openreview.net/forum?id=tmIiMPl4IPa |
| [54] | Hendrycks, D.; Gimpel, K., Gaussian Error Linear Units (GELUs) (2023), arXiv. arXiv:1606.08415 |
| [55] | Chernov, L. A., Wave Propagation in a Random Medium (1960), Dover publications: Dover publications Mineola, New York, Translated by Richard A. Silverman |
| [56] | de Carvalho Paludo, L.; Bouvier, V.; Cottereau, R., Scalable parallel scheme for sampling of Gaussian random fields over very large domains: Parallel scheme for sampling of random fields over very large domains. Internat. J. Numer. Methods Engrg., 8, 845-859 (2019) · Zbl 07865241 |
| [57] | Khazaie, S.; Cottereau, R.; Clouteau, D., Influence of the spatial correlation structure of an elastic random medium on its scattering properties. J. Sound Vib., 132-148 (2016) |
| [58] | Scalise, M.; Pitarka, A.; Louie, J. N.; Smith, K. D., Effect of random 3D correlated velocity perturbations on numerical modeling of ground motion from the source physics experiment. Bull. Seismol. Soc. Am., 1, 139-156 (2021) |
| [59] | Delouis, B.; Oral, E.; Menager, M.; Ampuero, J. P.; Guilhem Trilla, A.; Régnier, M.; Deschamps, A., Constraining the point source parameters of the 11 November 2019 Mw 4.9 Le Teil earthquake using multiple relocation approaches, first motion and full waveform inversions. C. R. Géosci., S1, 1-24 (2021) |
| [60] | Faccioli, E.; Maggio, F.; Paolucci, R.; Quarteroni, A., 2D and 3D elastic wave propagation by a pseudo-spectral domain decomposition method. J. Seismol., 3, 237-251 (1997) |
| [61] | Komatitsch, D.; Tromp, J., Introduction to the spectral element method for three-dimensional seismic wave propagation. Geophys. J. Int., 3, 806-822 (1999) |
| [62] | Touhami, S.; Gatti, F.; Lopez-Caballero, F.; Cottereau, R.; de Abreu Corrêa, L.; Aubry, L.; Clouteau, D., SEM3D: a 3D high-fidelity numerical earthquake simulator for broadband (0-10 Hz) seismic response prediction at a regional scale. Geosciences, 3, 112 (2022) |
| [63] | Rekoske, J. M.; Gabriel, A. A.; May, D. A., Instantaneous physics-based ground motion maps using reduced-order modeling. J. Geophys. Res. Solid Earth, 8 (2023) |
| [64] | Rosti, A.; Smerzini, C.; Paolucci, R.; Penna, A.; Rota, M., Validation of physics-based ground shaking scenarios for empirical fragility studies: The case of the 2009 L’Aquila earthquake. Bull. Earthq. Eng., 1, 95-123 (2023) |
| [65] | Castro-Cruz, D.; Gatti, F.; Lopez-Caballero, F., High-fidelity broadband prediction of regional seismic response: A hybrid coupling of physics-based synthetic simulation and empirical Green functions. Nat. Hazards, 2, 1997-2031 (2021) |
| [66] | De Martin, F.; Chaljub, E.; Thierry, P.; Sochala, P.; Dupros, F.; Maufroy, E.; Hadri, B.; Benaichouche, A.; Hollender, F., Influential parameters on 3-D synthetic ground motions in a sedimentary basin derived from global sensitivity analysis. Geophys. J. Int., 3, 1795-1817 (2021) |
| [67] | Heinecke, A.; Breuer, A.; Rettenberger, S.; Bader, M.; Gabriel, A. A.; Pelties, C.; Bode, A.; Barth, W.; Liao, X. K.; Vaidyanathan, K.; Smelyanskiy, M.; Dubey, P., Petascale high order dynamic rupture earthquake simulations on heterogeneous supercomputers, 3-14 |
| [68] | Rahaman, N.; Baratin, A.; Arpit, D.; Draxler, F.; Lin, M.; Hamprecht, F.; Bengio, Y.; Courville, A., On the spectral bias of neural networks, 5301-5310, URL https://proceedings.mlr.press/v97/rahaman19a.html |
| [69] | Basri, R.; Galun, M.; Geifman, A.; Jacobs, D.; Kasten, Y.; Kritchman, S., Frequency bias in neural networks for input of non-uniform density |
| [70] | Smerzini, C.; Vanini, M.; Paolucci, R.; Renault, P.; Traversa, P., Regional physics-based simulation of ground motion within the Rhône Valley, France, during the Mw 4.9 2019 Le Teil earthquake. Bull. Earthq. Eng., 4, 1747-1774 (2023) |
| [71] | Lehmann, F.; Gatti, F.; Bertin, M.; Clouteau, D., Seismic hazard analysis with a Factorized Fourier Neural Operator (F-FNO) surrogate model enhanced by transfer learning, URL https://openreview.net/forum?id=xiNRyrBAjt |
| [72] | Majumdar, R.; Karande, S.; Vig, L., How important are specialized transforms in Neural Operators? (2023), arXiv:2308.09293 |
| [73] | Loshchilov, I.; Hutter, F., SGDR: Stochastic gradient descent with warm restarts, arXiv:1608.03983v5 |
| [74] | Smith, L. N.; Topin, N., Super-convergence: Very fast training of neural networks using large learning rates (2018) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
