CNN-DP: composite neural network with differential propagation for impulsive nonlinear dynamics. (English) Zbl 07807985
Summary: In mechanical engineering, abundant high-quality data from simulations and experimental observations can help develop practical and accurate data-driven models. However, when dynamics are complex and highly nonlinear, designing a suitable model and optimizing it accurately is challenging. In particular, when data comprise impulsive signals or high-frequency components, training a data-driven model becomes increasingly challenging. This study proposes a novel and robust composite neural network for impulsive time-transient dynamics by dividing the prediction of the dynamics into tasks for three sub-networks, one for approximating simplified dynamics and the other two for mapping lower-order derivatives to higher-order derivatives. The mapping serves as the temporal differential operator, hence, the name “composite neural network with differential propagation (cNN-DP)” for the suggested model. Furthermore, numerical investigations were conducted to compare cNN-DP with two baseline models, a conventional network and another employing the autogradient approach. Regarding the convergence rate of model optimizations and the generalization accuracy, the proposed network outperformed the baseline models by many orders of magnitude. In terms of computational efficiency, numerical tests showed that cNN-DP requires an acceptable and comparable computational load. Although the numerical studies and descriptions focus on accelerations, the proposed network can be easily extended to any other application involving impulsive data.
MSC:
| 68Txx | Artificial intelligence |
| 65Dxx | Numerical approximation and computational geometry (primarily algorithms) |
| 65Lxx | Numerical methods for ordinary differential equations |
Keywords:
composite neural network; data-driven modeling; surrogate modeling; impulsive dynamics; nonlinear dynamicsReferences:
| [1] | Choi, H.-S.; An, J.; Han, S.; Kim, J.-G.; Jung, J.-Y.; Choi, J.; Orzechowski, G.; Mikkola, A.; Choi, J. H., Data-driven simulation for general-purpose multibody dynamics using deep neural networks. Multibody Syst. Dyn., 4, 419-454 (2021) · Zbl 1470.70010 |
| [2] | Han, S.; Choi, H.-S.; Choi, J.; Choi, J. H.; Kim, J.-G., A DNN-based data-driven modeling employing coarse sample data for real-time flexible multibody dynamics simulations. Comput. Methods Appl. Mech. Eng. (2021) · Zbl 1506.65097 |
| [3] | Go, M.-S.; Han, S.; Lim, J. H.; Kim, J.-G., An efficient fixed-time increment-based data-driven simulation for general multibody dynamics using deep neural networks. Eng. Comput., 1-19 (2023) |
| [4] | Kutz, J. N., Deep learning in fluid dynamics. J. Fluid Mech., 1-4 (2017) · Zbl 1383.76380 |
| [5] | Alizadehdakhel, A.; Rahimi, M.; Sanjari, J.; Alsairafi, A. A., CFD and artificial neural network modeling of two-phase flow pressure drop. Int. Commun. Heat Mass Transf., 8, 850-856 (2009) |
| [6] | Safikhani, H.; Khalkhali, A.; Farajpoor, M., Pareto based multi-objective optimization of centrifugal pumps using CFD, neural networks and genetic algorithms. Eng. Appl. Comput. Fluid Mech., 1, 37-48 (2011) |
| [7] | Elsayed, K.; Lacor, C., Modeling, analysis and optimization of aircyclones using artificial neural network, response surface methodology and CFD simulation approaches. Powder Technol., 1, 115-133 (2011) |
| [8] | Shojaeefard, M. H.; Akbari, M.; Asadi, P., Multi objective optimization of friction stir welding parameters using FEM and neural network. Int. J. Precis. Eng. Manuf., 11, 2351-2356 (2014) |
| [9] | Chen, H.; Wu, L.; Dou, Q.; Qin, J.; Li, S.; Cheng, J.-Z.; Ni, D.; Heng, P.-A., Ultrasound standard plane detection using a composite neural network framework. IEEE Trans. Cybern., 6, 1576-1586 (2017) |
| [10] | Pan, Y.; Nie, X.; Li, Z.; Gu, S., Data-driven vehicle modeling of longitudinal dynamics based on a multibody model and deep neural networks. Measurement (2021) |
| [11] | He, Q.; Barajas-Solano, D.; Tartakovsky, G.; Tartakovsky, A. M., Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport. Adv. Water Resour. (2020) |
| [12] | Cai, S.; Wang, Z.; Lu, L.; Zaki, T. A.; Karniadakis, G. E., DeepM&Mnet: inferring the electroconvection multiphysics fields based on operator approximation by neural networks. J. Comput. Phys. (2021) · Zbl 07513856 |
| [13] | Na, W.; Zhang, W.; Yan, S.; Feng, F.; Zhang, W.; Zhang, Y., Automated neural network-based multiphysics parametric modeling of microwave components. IEEE Access, 141153-141160 (2019) |
| [14] | Lee, S. Y.; Park, C.-S.; Park, K.; Lee, H. J.; Lee, S., A physics-informed and data-driven deep learning approach for wave propagation and its scattering characteristics. Eng. Comput., 1-17 (2022) |
| [15] | Rahaman, N.; Baratin, A.; Arpit, D.; Draxler, F.; Lin, M.; Hamprecht, F.; Bengio, Y.; Courville, A., On the spectral bias of neural networks, 5301-5310 |
| [16] | Yang, M.-C.; Chen, M. C., Composite neural network: theory and application to PM2.5 prediction. IEEE Trans. Knowl. Data Eng. (2021) |
| [17] | Yang, M.-C.; Chen, M. C., An analysis of composite neural network performance from function composition perspective (2018) |
| [18] | Meng, X.; Karniadakis, G. E., A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems. J. Comput. Phys. (2020) · Zbl 1454.76006 |
| [19] | Ahn, J.-G.; Yang, H.-I.; Kim, J.-G., Multi-fidelity meta modeling using composite neural network with online adaptive basis technique. Comput. Methods Appl. Mech. Eng. (2022) · Zbl 1507.65026 |
| [20] | Zhang, X.; Xie, F.; Ji, T.; Zhu, Z.; Zheng, Y., Multi-fidelity deep neural network surrogate model for aerodynamic shape optimization. Comput. Methods Appl. Mech. Eng. (2021) · Zbl 1506.74324 |
| [21] | Mahmoudabadbozchelou, M.; Caggioni, M.; Shahsavari, S.; Hartt, W. H.; Em Karniadakis, G.; Jamali, S., Data-driven physics-informed constitutive metamodeling of complex fluids: a multifidelity neural network (MFNN) framework. J. Rheol., 2, 179-198 (2021) |
| [22] | De, S.; Doostan, A., Neural network training using \(ℓ1\)-regularization and bi-fidelity data. J. Comput. Phys. (2022) |
| [23] | Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 686-707 (2019) · Zbl 1415.68175 |
| [24] | Karniadakis, G. E.; Kevrekidis, I. G.; Lu, L.; Perdikaris, P.; Wang, S.; Yang, L., Physics-informed machine learning. Nat. Rev. Phys., 6, 422-440 (2021) |
| [25] | Jagtap, A. D.; Kharazmi, E.; Karniadakis, G. E., Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems. Comput. Methods Appl. Mech. Eng. (2020) · Zbl 1442.92002 |
| [26] | Jagtap, A. D.; Karniadakis, G. E., Extended physics-informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations · Zbl 1542.65182 |
| [27] | Robbins, H.; Monro, S., A stochastic approximation method. Ann. Math. Stat., 400-407 (1951) · Zbl 0054.05901 |
| [28] | Kingma, D. P.; Ba Adam, J., A method for stochastic optimization (2014), arXiv preprint |
| [29] | Liu, L.; Jiang, H.; He, P.; Chen, W.; Liu, X.; Gao, J.; Han, J., On the variance of the adaptive learning rate and beyond (2019), arXiv preprint |
| [30] | Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; Lerer, A., Automatic differentiation in PyTorch (2017) |
| [31] | Hendrycks, D.; Gimpel, K., Gaussian error linear units (gelus) (2016), arXiv preprint |
| [32] | He, K.; Zhang, X.; Ren, S.; Sun, J., Delving deep into rectifiers: surpassing human-level performance on imagenet classification, 1026-1034 |
| [33] | Rabinovich, M. I.; Fabrikant, A. L., Stochastic self-modulation of waves in nonequilibrium media. J. Exp. Theor. Phys., 617-629 (1979) |
| [34] | Wanner, G.; Hairer, E., Solving Ordinary Differential Equations II (1996), Springer Berlin Heidelberg New York · Zbl 0859.65067 |
| [35] | Danca, M.-F.; Kuznetsov, N.; Chen, G., Unusual dynamics and hidden attractors of the Rabinovich-Fabrikant system. Nonlinear Dyn., 791-805 (2017) |
| [36] | Stein, M., Large sample properties of simulations using Latin hypercube sampling. Technometrics, 2, 143-151 (1987) · Zbl 0627.62010 |
| [37] | Lorenz, E. N., Deterministic nonperiodic flow. J. Atmos. Sci., 2, 130-141 (1963) · Zbl 1417.37129 |
| [38] | Luo, C.; Wang, X., Chaos in the fractional-order complex Lorenz system and its synchronization. Nonlinear Dyn., 241-257 (2013) · Zbl 1268.34022 |
| [39] | Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L., Pytorch: an imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. (2019) |
| [40] | Nvidia; Vingelmann, P.; Fitzek, F. H., Cuda, release: 11.8.0 (2020) |
| [41] | Wang, S.; Yu, X.; Perdikaris, P., When and why PINNs fail to train: a neural tangent kernel perspective. J. Comput. Phys. (2022) · Zbl 07524768 |
| [42] | Xiang, Z.; Peng, W.; Liu, X.; Yao, W., Self-adaptive loss balanced physics-informed neural networks. Neurocomputing, 11-34 (2022) |
| [43] | Van Der Pol, B., VII. Forced oscillations in a circuit with non-linear resistance. (Reception with reactive triode). Lond. Edinb. Dublin Philos. Mag. J. Sci., 13, 65-80 (1927) |
| [44] | (2022), Center for Engineering Strong Motion Data (CESMD) |
| [45] | Akima, H., A new method of interpolation and smooth curve fitting based on local procedures. J. ACM, 4, 589-602 (1970) · Zbl 0209.46805 |
| [46] | Griffin, D.; Lim, J., Signal estimation from modified short-time Fourier transform. IEEE Trans. Acoust. Speech Signal Process., 2, 236-243 (1984) |
| [47] | Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; Polosukhin, I., Attention is all you need. Adv. Neural Inf. Process. Syst. (2017) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
