A dual-topological graph memory network for anti-noise multivariate time series forecasting. (English) Zbl 07897654
Summary: Multivariate time series (MTS) forecasting plays an essential role in the automation and optimization process of intelligent applications. However, capturing correlations and dependencies among variables in MTS data remains a major challenge for the forecasting models. Although existing methods explain these complex relationships by deeply analyzing variables and intervariable dependencies in MTS data, they tend to be affected by the noise interference prevalent in real-world data, which increases the difficulty for forecasting models to characterize the real features, and thus makes it difficult to achieve satisfactory forecasting results. To address these challenges, this paper designs a dual-topological graph memory (DualGM) network for anti-noise multivariate time series forecasting. The core idea is to analyze the complex relationships in MTS data in detail by constructing dual-topological graphs and to learn and retain the comprehensive information of sequence features by employing a bidirectional self-attention memory module. Specifically, the model first constructs the MTS data as a graph structure and creates a probability topological graph and a distance topological graph based on the similarity between the samples, which helps the model recognize and filter the noise more efficiently in noisy environments, and valuable information is extracted from the noisy data. Then, the topological neighborhood features of the MTS data in each layer are summarized via multilayer graph groups, and the probability and distance spatial information is fused using a bidirectional self-attention mechanism to further enhance the memory and retention of the topological features, thus improving the accuracy of the forecasting model. The experimental results on four real datasets demonstrate that the DualGM model outperforms previously reported methods under various settings; in particular, the RSE values of the DualGM model are higher than those of the second-best results by 12%, 37%, 0.8% and 16% on the four datasets. Moreover, the DualGM model provides a valuable solution for multivariate time series forecasting under noise interference.
Keywords:
multivariate time series forecasting; anti-noise; dual-topological graph; bidirectional self-attentionReferences:
| [1] | Shih, S.-Y.; Sun, F.-K.; Lee, H. yi, Temporal pattern attention for multivariate time series forecasting, Mach. Learn., 108, 1421-1441, 2018 · Zbl 1493.62534 |
| [2] | Bacanin, N.; Jovanovic, L.; Zivkovic, M.; Kandasamy, V.; Antonijevic, M.; Deveci, M.; Strumberger, I., Multivariate energy forecasting via metaheuristic tuned long-short term memory and gated recurrent unit neural networks, Inf. Sci., 642, Article 119122 pp., 2023 |
| [3] | Ma, D.; Song, X. B.; Li, P., Daily traffic flow forecasting through a contextual convolutional recurrent neural network modeling inter- and intra-day traffic patterns, IEEE Trans. Intell. Transp. Syst., 22, 2627-2636, 2021 |
| [4] | Wang, L.; Shen, P., Memetic segmentation based on variable lag aware for multivariate time series, Inf. Sci., 657, Article 120003 pp., 2024 |
| [5] | Yao, J.; Han, L.; Yang, K.; Guo, G.; Liu, N.; Huang, X.; Zheng, Z.; Zhang, D.; Han, J., Contextual dependency vision transformer for spectrogram-based multivariate time series analysis, Neurocomputing, 572, Article 127215 pp., 2023 |
| [6] | Chen, L.; Chen, D.; Shang, Z.; Wu, B.; Zheng, C.; Wen, B.; Zhang, W., Multi-scale adaptive graph neural network for multivariate time series forecasting, IEEE Trans. Knowl. Data Eng., 35, 10748-10761, 2022 |
| [7] | Wang, X.; Liu, H.; Du, J.; Dong, X.; Yang, Z., A long-term multivariate time series forecasting network combining series decomposition and convolutional neural networks, Appl. Soft Comput., 139, Article 110214 pp., 2023 |
| [8] | Geng, X.; He, X.; Xu, L.; Yu, J., Graph correlated attention recurrent neural network for multivariate time series forecasting, Inf. Sci., 606, 126-142, 2022 |
| [9] | Sun, Y.; Ding, S.; Guo, L.; Zhang, Z., Hypergraph regularized semi-supervised support vector machine, Inf. Sci., 591, 400-421, 2022 · Zbl 07798547 |
| [10] | Zhao, J. ying; Song, Y.; Wang, L.; Guo, H.; Marigentti, F.; Liu, X., Forecasting the eddy current loss of a large turbo generator using hybrid ensemble Gaussian process regression, Eng. Appl. Artif. Intell., 121, Article 106022 pp., 2023 |
| [11] | Schimbinschi, F.; Moreira-Matias, L.; Nguyen, X. V.; Bailey, J., Topology-regularized universal vector autoregression for traffic forecasting in large urban areas, Expert Syst. Appl., 82, 301-316, 2017 |
| [12] | Chen, Z.; Li, X. yao; Wang, W.; Li, Y.; Shi, L.; Li, Y., Residual strength prediction of corroded pipelines using multilayer perceptron and modified feedforward neural network, Reliab. Eng. Syst. Saf., 231, Article 108980 pp., 2022 |
| [13] | Ma, X.; Zhang, J.; Du, B.; Ding, C.; Sun, L., Parallel architecture of convolutional bi-directional lstm neural networks for network-wide metro ridership prediction, IEEE Trans. Intell. Transp. Syst., 20, 2278-2288, 2019 |
| [14] | Liu, L.; He, B.; Zhang, D.; Mao, H., Deep belief network-based prediction for gear noise, (8th International Conference on Mechatronics and Robotics Engineering. 8th International Conference on Mechatronics and Robotics Engineering, ICMRE 2022, Munich, Germany, February 10-12, 2022), 50-54 |
| [15] | Xie, J.; Yan, R.; Xiao, S.; Peng, L.; Johnson, M. T.; Zhang, W., Dynamic temporal residual learning for speech recognition, (2020 IEEE International Conference on Acoustics, Speech and Signal Processing. 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020), 7709-7713 |
| [16] | Dolatabadi, A.; Abdeltawab, H. H.; Mohamed, Y. A.-R. I., Deep spatial-temporal 2-d cnn-blstm model for ultrashort-term lidar-assisted wind turbine’s power and fatigue load forecasting, IEEE Trans. Ind. Inform., 18, 2342-2353, 2021 |
| [17] | Babu, G. S.; Zhao, P.; Li, X., Deep convolutional neural network based regression approach for estimation of remaining useful life, (Database Systems for Advanced Applications - 21st International Conference, Proceedings, Part I, vol. 9642. Database Systems for Advanced Applications - 21st International Conference, Proceedings, Part I, vol. 9642, DASFAA 2016, Dallas, TX, USA, April 16-19, 2016), 214-228 |
| [18] | Hong, S.; Kang, M.; Kim, J.; Baek, J. H., Sequential application of denoising autoencoder and long-short recurrent convolutional network for noise-robust remaining-useful-life prediction framework of lithium-ion batteries, Comput. Ind. Eng., 179, Article 109231 pp., 2023 |
| [19] | Wang, Y.; Duan, Z.; Huang, Y.; Xu, H.; Feng, J.; Ren, A., Mthetgnn: a heterogeneous graph embedding framework for multivariate time series forecasting, Pattern Recognit. Lett., 153, 151-158, 2020 |
| [20] | Bai, L.; Yao, L.; Li, C.; Wang, X.; Wang, C., Adaptive graph convolutional recurrent network for traffic forecasting, (Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, 2020, December 6-12, 2020), 6-12 |
| [21] | Wu, Z.; Pan, S.; Long, G.; Jiang, J.; Zhang, C., Graph wavenet for deep spatial-temporal graph modeling, (Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019), 1907-1913 |
| [22] | Zheng, C.; Fan, X.; Pan, S.; Wu, Z.; Wang, C.; Yu, P. S., Spatio-temporal joint graph convolutional networks for traffic forecasting, IEEE Trans. Knowl. Data Eng., 36, 372-385, 2021 |
| [23] | He, K.; Chen, X.; Wu, Q.; Yu, S.; Zhou, Z., Graph attention spatial-temporal network with collaborative global-local learning for citywide mobile traffic prediction, IEEE Trans. Mob. Comput., 21, 1244-1256, 2022 |
| [24] | Wu, Z.; Pan, S.; Long, G.; Jiang, J.; Chang, X.; Zhang, C., Connecting the dots: multivariate time series forecasting with graph neural networks, (KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event. KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020), 753-763 |
| [25] | Wang, S.; Fan, Y.; Jin, S.; Takyi-Aninakwa, P.; Fernandez, C., Improved anti-noise adaptive long short-term memory neural network modeling for the robust remaining useful life prediction of lithium-ion batteries, Reliab. Eng. Syst. Saf., 230, Article 108920 pp., 2023 |
| [26] | Huang, X.; Huang, H.; Wu, J.; Yang, M.; Ding, W. P., Sound quality prediction and improving of vehicle interior noise based on deep convolutional neural networks, Expert Syst. Appl., 160, Article 113657 pp., 2020 |
| [27] | Peng, D.; Wang, H.; Liu, Z.; Zhang, W.; Zuo, M. J.; Chen, J., Multibranch and multiscale cnn for fault diagnosis of wheelset bearings under strong noise and variable load condition, IEEE Trans. Ind. Inform., 16, 4949-4960, 2020 |
| [28] | Fan, W.; Chen, Z.; Li, Y.; Zhu, F.; Xie, M., A reinforced noise resistant correlation method for bearing condition monitoring, IEEE Trans. Autom. Sci. Eng., 20, 995-1006, 2023 |
| [29] | Huang, H.; Tang, B.; Luo, J.; Pu, H.; Zhang, K., Residual gated dynamic sparse network for gearbox fault diagnosis using multisensor data, IEEE Trans. Ind. Inform., 18, 2264-2273, 2021 |
| [30] | Xiao, L.; Tang, J.; Zhang, X.; Bechhoefer, E.; Ding, S., Remaining useful life prediction based on intentional noise injection and feature reconstruction, Reliab. Eng. Syst. Saf., 215, Article 107871 pp., 2021 |
| [31] | Zhou, Z.; Zhang, M.; He, Y., Regularizing deep neural networks by noise: its interpretation and optimization, IEEE Trans. Pattern Anal. Mach. Intell., 1564-1577, 2017 |
| [32] | Yoon, Y.; Yu, J.; Jeon, M., Predictively encoded graph convolutional network for noise-robust skeleton-based action recognition, Appl. Intell., 52, 3, 2317-2331, 2022 |
| [33] | Li, X.; Zhang, W.; Ding, Q., A robust intelligent fault diagnosis method for rolling element bearings based on deep distance metric learning, Neurocomputing, 310, 77-95, 2018 |
| [34] | Liu, H.; Zhou, J. zhong; Zheng, Y.; Jiang, W.; Zhang, Y., Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders, ISA Trans., 77, 167-178, 2018 |
| [35] | Peng, X.; Wang, P.; Xia, S.; Wang, C.; Chen, W., Vpgb: a granular-ball based model for attribute reduction and classification with label noise, Inf. Sci., 611, 504-521, 2022 |
| [36] | Kadhim, Z. S.; Abdullah, H. S.; Ghathwan, K. I., Automatically avoiding overfitting in deep neural networks by using hyper-parameters optimization methods, Int. J. Online Biomed. Eng., 19, 5, 146-162, 2023 |
| [37] | Zhou, J.; Cui, G.; Zhang, Z.; Yang, C.; Liu, Z.; Sun, M., Graph neural networks: a review of methods and applications, AI Open |
| [38] | van der Maaten, L.; Hinton, G. E., Visualizing data using t-sne, J. Mach. Learn. Res., 9, 2579-2605, 2008 · Zbl 1225.68219 |
| [39] | Ren, J.; Chen, W.; Lin, X.; Huang, Q., Environmental noise and its effect on sensor reliability: a case study in industrial automation, IEEE Trans. Ind. Inform., 16, 2326-2334, 2020 |
| [40] | Feng, Y.; Sun, H., Robust optimal control for discrete-time lti systems over multiple additive white Gaussian noise channels, IEEE Trans. Autom. Control, 68, 5174-5186, 2023 · Zbl 07746852 |
| [41] | Zheng, C.; Wang, S.; Liu, Y.; Liu, C.; Xie, W.; Fang, C.; Liu, S., A novel equivalent model of active distribution networks based on lstm, IEEE Trans. Neural Netw. Learn. Syst., 30, 2611-2624, 2019 |
| [42] | Wang, R.; Fu, B.; Fu, G.; Wang, M., Deep & cross network for ad click predictions, (Proceedings of the ADKDD’17. Proceedings of the ADKDD’17, Halifax, NS, Canada, August 13 - 17, 2017), Article 12 pp. |
| [43] | Lea, C.; Flynn, M. D.; Vidal, R.; Reiter, A.; Hager, G. D., Temporal convolutional networks for action segmentation and detection, (2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017), 1003-1012 |
| [44] | Vaswani, A.; Shazeer, N. M.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; Polosukhin, I., Attention is all you need, (Neural Information Processing Systems, 2017) |
| [45] | Sabour, S.; Frosst, N.; Hinton, G. E., Dynamic routing between capsules, (Advances in Neural Information Processing Systems, vol. 30, 2017), 3856-3866 |
| [46] | Sun, H. C.; Wang, C.; Cao, X., An adaptive anti-noise gear fault diagnosis method based on attention residual prototypical network under limited samples, Appl. Soft Comput., 125, Article 109120 pp., 2022 |
| [47] | Liu, Y.; Chen, L.; Li, Y.; Zhao, J.; Wang, W., Time series prediction with input noise based on the esn and the em and its industrial applications, Expert Syst. Appl., 217, Article 119591 pp., 2023 |
| [48] | Lai, G.; Chang, W.; Yang, Y.; Liu, H., Modeling long- and short-term temporal patterns with deep neural networks, (The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018), 95-104 |
| [49] | Chang, Y.; Sun, F.; Wu, Y.; Lin, S., A memory-network based solution for multivariate time-series forecasting, CoRR |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
