×

Multivariate time-series classification with hierarchical variational graph pooling. (English) Zbl 1522.62053

Summary: In recent years, multivariate time-series classification (MTSC) has attracted considerable attention owing to the advancement of sensing technology. Existing deep-learning-based MTSC techniques, which mostly rely on convolutional or recurrent neural networks, focus primarily on the temporal dependency of a single time series. Based on this, complex pairwise dependencies among multivariate variables can be better described using advanced graph methods, where each variable is regarded as a node in the graph, and their dependencies are regarded as edges. Furthermore, current spatial-temporal modeling (e.g., graph classification) methodologies based on graph neural networks (GNNs) are inherently flat and cannot hierarchically aggregate node information. To address these limitations, we propose a novel graph-pooling-based framework, MTPool, to obtain an expressive global representation of MTS. We first convert MTS slices into graphs using the interactions of variables via a graph structure learning module and obtain the spatial-temporal graph node features via a temporal convolutional module. To obtain global graph-level representation, we design an “encoder-decoder”-based variational graph pooling module to create adaptive centroids for cluster assignments. Then, we combine GNNs and our proposed variational graph pooling layers for joint graph representation learning and graph coarsening, after which the graph is progressively coarsened to one node. Finally, a differentiable classifier uses this coarsened representation to obtain the final predicted class. Experiments on ten benchmark datasets showed that MTPool outperforms state-of-the-art strategies in the MTSC task.

MSC:

62H30 Classification and discrimination; cluster analysis (statistical aspects)
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
68T05 Learning and adaptive systems in artificial intelligence

Software:

FastGCN; ASAP; t-SNE

References:

[1] Bagnall, A.; Dau, H. A.; Lines, J.; Flynn, M.; Large, J.; Bostrom, A., The UEA multivariate time series classification archive, 2018 (2018), arXiv preprint arXiv:1811.00075
[2] Bagnall, A. J.; Flynn, M.; Large, J.; Lines, J.; Middlehurst, M., A tale of two toolkits, report the third: on the usage and performance of HIVE-cote v1. 0 (2020), CoRR
[3] Bai, Y., Ding, H., Bian, S., Chen, T., Sun, Y., & Wang, W. (2019). Simgnn: A neural network approach to fast graph similarity computation. In Proceedings of the twelfth ACM international conference on web search and data mining (pp. 384-392).
[4] Baldán, F. J.; Benítez, J. M., Distributed fastshapelet transform: a big data time series classification algorithm, Information Sciences, 496, 451-463 (2019)
[5] Chen, J.; Ma, T.; Xiao, C., Fastgcn: fast learning with graph convolutional networks via importance sampling (2018), arXiv preprint arXiv:1801.10247
[6] Defferrard, M.; Bresson, X.; Vandergheynst, P., Convolutional neural networks on graphs with fast localized spectral filtering, (Lee, D. D.; Sugiyama, M.; Luxburg, U. V.; Guyon, I.; Garnett, R., Advances in neural information processing systems, Vol. 29 (2016), Curran Associates, Inc.), 3844-3852
[7] Duan, Z.; Wang, Y.; Ye, W.; Feng, Z.; Fan, Q.; Li, X., Connecting latent ReLationships over heterogeneous attributed network for recommendation (2021), arXiv preprint arXiv:2103.05749
[8] Feng, Z.-k.; Niu, W.-j., Hybrid artificial neural network and cooperation search algorithm for nonlinear river flow time series forecasting in humid and semi-humid regions, Knowledge-Based Systems, 211, Article 106580 pp. (2021)
[9] Gao, H.; Ji, S., Graph u-nets (2019), arXiv preprint arXiv:1905.05178
[10] Hamilton, W. L.; Ying, R.; Leskovec, J., Inductive representation learning on large graphs (2017), CoRR abs/1706.02216. arXiv:1706.02216
[11] Iwana, B. K.; Frinken, V.; Uchida, S., DTW-NN: A novel neural network for time series recognition using dynamic alignment between inputs and weights, Knowledge-Based Systems, 188, Article 104971 pp. (2020)
[12] Kang, H.; Choi, S., Bayesian common spatial patterns for multi-subject EEG classification, Neural Networks, 57, 39-50 (2014) · Zbl 1368.92096
[13] Karim, F.; Majumdar, S.; Darabi, H.; Harford, S., Multivariate LSTM-FCNs for time series classification, Neural Networks, 116, 237-245 (2019)
[14] Khasahmadi, A. H.; Hassani, K.; Moradi, P.; Lee, L.; Morris, Q., Memory-based graph networks, (International conference on learning representations (2020))
[15] Kipf, T. N.; Welling, M., Semi-supervised classification with graph convolutional networks (2016), CoRR abs/1609.02907. arXiv:1609.02907
[16] Lee, J.; Lee, I.; Kang, J., Self-attention graph pooling (2019), arXiv preprint arXiv:1904.08082
[17] Li, J.; Yang, B.; Li, H.; Wang, Y.; Qi, C.; Liu, Y., DTDR-ALSTM: Extracting dynamic time-delays to reconstruct multivariate data for improving attention-based LSTM industrial time series prediction models, Knowledge-Based Systems, 211, Article 106508 pp. (2021)
[18] Liu, F.; Xue, S.; Wu, J.; Zhou, C.; Hu, W.; Paris, C., Deep learning for community detection: progress, challenges and opportunities (2020), arXiv preprint arXiv:2005.08225
[19] Ma, X.; Wu, J.; Xue, S.; Yang, J.; Zhou, C.; Sheng, Q. Z., A comprehensive survey on graph anomaly detection with deep learning, IEEE Transactions on Knowledge and Data Engineering (2021)
[20] Maaten, L.v.d.; Hinton, G., Visualizing data using t-SNE, Journal of Machine Learning Research, 9, Nov, 2579-2605 (2008) · Zbl 1225.68219
[21] Mori, U.; Mendiburu, A.; Miranda, I. M.; Lozano, J. A., Early classification of time series using multi-objective optimization techniques, Information Sciences, 492, 204-218 (2019) · Zbl 1451.62102
[22] Morris, C., Ritzert, M., Fey, M., Hamilton, W. L., Lenssen, J. E., & Rattan, G., et al. (2019). Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33 (pp. 4602-4609).
[23] Pei, W.; Dibeklioğlu, H.; Tax, D. M.; van der Maaten, L., Multivariate time-series classification using the hidden-unit logistic model, IEEE Transactions on Neural Networks and Learning Systems, 29, 4, 920-931 (2017)
[24] Ranjan, E.; Sanyal, S.; Talukdar, P. P., ASAP: Adaptive structure aware pooling for learning hierarchical graph representations, (AAAI (2020)), 5470-5477
[25] Ruiz, A. P.; Flynn, M.; Bagnall, A., Benchmarking multivariate time series classification algorithms (2020), arXiv preprint arXiv:2007.13156
[26] Scarselli, F.; Gori, M.; Tsoi, A. C.; Hagenbuchner, M.; Monfardini, G., The graph neural network model, IEEE Transactions on Neural Networks, 20, 1, 61-80 (2008)
[27] Schäfer, P.; Leser, U., Multivariate time series classification with weasel+ MUSE (2017), arXiv preprint arXiv:1711.11343
[28] Seto, S.; Zhang, W.; Zhou, Y., Multivariate time series classification using dynamic time warping template selection for human activity recognition, (2015 IEEE symposium series on computational intelligence (2015), IEEE), 1399-1406
[29] Shi, Z.; Bai, Y.; Jin, X.; Wang, X.; Su, T.; Kong, J., Parallel deep prediction with covariance intersection fusion on non-stationary time series, Knowledge-Based Systems, 211, Article 106523 pp. (2021)
[30] Su, X.; Xue, S.; Liu, F.; Wu, J.; Yang, J.; Zhou, C., A comprehensive survey on community detection with deep learning (2021), arXiv preprint arXiv:2105.12584
[31] Sun, C.; Chen, C.; Li, W.; Fan, J.; Chen, W., A hierarchical neural network for sleep stage classification based on comprehensive feature learning and multi-flow sequence learning, IEEE Journal of Biomedical and Health Informatics, 24, 5, 1351-1366 (2019)
[32] Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Lio, P.; Bengio, Y., Graph attention networks (2017), arXiv preprint arXiv:1710.10903
[33] Wang, Y.; Duan, Z.; Huang, Y.; Xu, H.; Feng, J.; Ren, A., MTHetGNN: A heterogeneous graph embedding framework for multivariate time series forecasting (2020), arXiv e-Prints, arXiv-2008
[34] Wang, Y., Duan, Z., Liao, B., Wu, F., & Zhuang, Y. (2019). Heterogeneous attributed network embedding with graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33 (pp. 10061-10062).
[35] Wang, Z.; Yan, W.; Oates, T., Time series classification from scratch with deep neural networks: A strong baseline, (2017 international joint conference on neural networks (IJCNN) (2017), IEEE), 1578-1585
[36] Wu, Z.; Pan, S.; Long, G.; Jiang, J.; Chang, X.; Zhang, C., Connecting the dots: Multivariate time series forecasting with graph neural networks (2020), arXiv preprint arXiv:2005.11650
[37] Xu, H.; Chen, R.; Bai, Y.; Duan, Z.; Feng, J.; Sun, Y., CoSimGNN: Towards large-scale graph similarity computation (2020), arXiv preprint arXiv:2005.07115
[38] Xu, H.; Duan, Z.; Wang, Y.; Feng, J.; Chen, R.; Zhang, Q., Graph partitioning and graph neural network based hierarchical graph matching for graph similarity computation, Neurocomputing, 439, 348-362 (2021)
[39] Xu, K.; Hu, W.; Leskovec, J.; Jegelka, S., How powerful are graph neural networks? (2018), ArXiv arXiv:1810.00826
[40] Xu, H.; Huang, Y.; Duan, Z.; Wang, X.; Feng, J.; Song, P., Multivariate time series forecasting with transfer entropy graph (2020), arXiv e-Prints, arXiv-2005
[41] Ying, Z.; You, J.; Morris, C.; Ren, X.; Hamilton, W.; Leskovec, J., Hierarchical graph representation learning with differentiable pooling, (Advances in neural information processing systems (2018)), 4800-4810
[42] Yu, Z.; Lee, M., Real-time human action classification using a dynamic neural model, Neural Networks, 69, 29-43 (2015)
[43] Zhang, X.; Gao, Y.; Lin, J.; Lu, C.-T., TapNet: Multivariate time series classification with attentional prototypical network, (AAAI (2020)), 6845-6852
[44] Zhou, J.; Cui, G.; Hu, S.; Zhang, Z.; Yang, C.; Liu, Z., Graph neural networks: A review of methods and applications, AI Open, 1, 57-81 (2020)
[45] Zhou, Y.; Duan, Z.; Xu, H.; Feng, J.; Ren, A.; Wang, Y., Parallel extraction of long-term trends and short-term fluctuation framework for multivariate time series forecasting (2020), arXiv preprint arXiv:2008.07730
[46] Zhou, K.; Wang, W.; Huang, L.; Liu, B., Comparative study on the time series forecasting of web traffic based on statistical model and generative adversarial model, Knowledge-Based Systems, 213, Article 106467 pp. (2021)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.