×

Semi-supervised anomaly detection in dynamic communication networks. (English) Zbl 1531.68093

Summary: To ensure the security and stabilization of the communication networks, anomaly detection is the first line of defense. However, their learning process suffers two major issues: (1) inadequate labels: there are many different kinds of attacks but rare abnormal nodes in mt of these atstacks; and (2) inaccurate labels: considering the heavy network flows and new emerging attacks, providing accurate labels for all nodes is very expensive. The inadequate and inaccurate label problem challenges many existing methods because the majority normal nodes result in a biased classifier while the noisy labels will further degrade the performance of the classifier. To tackle these issues, we propose SemiADC, a Semi-supervised Anomaly Detection framework for dynamic Communication networks. SemiADC first approximately learns the feature distribution of normal nodes with regularization from abnormal ones. It then cleans the datasets and extracts the nodes sasainaccurate labels by the learned feature distribution and structure-based temporal correlations. These self-learning processes run iteratively with mutual promotion, and finally help increase the accuracy of anomaly detection. Experimental evaluations on real-world datasets demonstrate the effectiveness of our SemiADC, which performs substantially better than the state-of-art anomaly detection approaches without the demand of adequate and accurate supervision.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
90B18 Communication networks in operations research

Software:

GANomaly

References:

[1] Y. Ban, X. Liu, L. Huang, Y. Duan, X. Liu, W. Xu, No place to hide: Catching fraudulent entities in tensors, in: WWW, 2019.
[2] Bars, B. L.; Kalogeratos, A., A probabilistic framework to node-level anomaly detection in communication networks (2019), INFOCOM: INFOCOM in
[3] S. Ranshous, S. Shen, D. Koutra, S. Harenberg, C. Faloutsos, N.F. Samatova, Anomaly detection in dynamic networks: a survey, Wiley Interdisc. Rev.: Comput. Stat. 7 (3). · Zbl 07912769
[4] W. Cheng, K. Zhang, H. Chen, G. Jiang, Z. Chen, W. Wang, Ranking causal anomalies via temporal and dynamical analysis on vanishing correlations, in: KDD, 2016.
[5] Rossi, R. A.; Gallagher, B.; Neville, J.; Henderson, K., Modeling dynamic behavior in large evolving graphs (2013), WSDM: WSDM in
[6] M. Ahmed, A.N. Mahmood, J. Hu, A survey of network anomaly detection techniques, J. Network Comput. Appl.
[7] R. Chalapathy, S. Chawla, Deep learning for anomaly detection: A survey, arXiv preprintarXiv:1901.03407.
[8] Jyothsna, V.; Prasad, V. V.R., A review of anomaly based intrusiondetection systems, Int. J. Comput. Appl., 28, 7, 26-35 (2011)
[9] S. Mohurle, M. Patil, A brief study of wannacry threat: Ransomware attack 2017, Int. J. Adv. Res. Comput. Sci. 8 (5).
[10] Mukkamala, S.; Sung, A. H., Detecting denial of service attacks using support vector machines (2003), FUZZ-IEEE: FUZZ-IEEE in
[11] Y. Zhou, M. Han, L. Liu, J.S. He, Y. Wang, Deep learning approach for cyberattack detection, in: INFOCOM Workshops, 2018.
[12] J. Kim, S. Bu, S. Cho, Zero-day malware detection using transferred generative adversarial networks based on deep autoencoders, Inf. Sci.
[13] Q. Huang, P.P.C. Lee, Ld-sketch: A distributed sketching design for accurate and scalable anomaly detection in network data streams, in: INFOCOM, 2014.
[14] I. Nevat, D.M. Divakaran, S.G. Nagarajan, P. Zhang, L. Su, L.L. Ko, V.L.L. Thing, Anomaly detection and attribution in networks with temporally correlated traffic, IEEE/ACM Trans. Netw. 26 (1).
[15] Yu, J. J.Q.; Hou, Y.; Li, V. O.K., Online false data injection attack detection with wavelet transform and deep neural networks, IEEE Trans. Ind. Inf., 14, 7, 3271-3280 (2018)
[16] D. Eswaran, C. Faloutsos, Sedanspot: Detecting anomalies in edge streams, in: ICDM, 2018.
[17] D. Eswaran, C. Faloutsos, S. Guha, N. Mishra, Spotlight: Detecting anomalies in streaming graphs, in: KDD, 2018.
[18] Zhu, X.; Goldberg, A. B., Introduction to Semi-Supervised Learning (2009), Morgan & Claypool Publishers: Morgan & Claypool Publishers Synthesis Lectures on Artificial Intelligence and Machine Learning · Zbl 1209.68435
[19] Li, Y.; Ye, J., Learning adversarial networks for semi-supervised text classification via policy gradient (2018), KDD: KDD in
[20] R.A.R. Ashfaq, X. Wang, J.Z. Huang, H. Abbas, Y. He, Fuzziness based semi-supervised learning approach for intrusion detection system, Inf. Sci. 378.
[21] G. Pang, C. Yan, C. Shen, A. van den Hengel, X. Bai, Self-trained deep ordinal regression for end-to-end video anomaly detection, in: CVPR, 2020.
[22] Akcay, S.; Abarghouei, A. A.; Breckon, T. P., Ganomaly:Semi-supervised anomaly detection via adversarial training (2018), ACCV: ACCV in
[23] D. Li, D. Chen, J. Goh, S. Ng, Anomaly detection with generative adversarial networks for multivariate time series, arXiv preprintabs/1809.04758.
[24] H. Zenati, M. Romain, C. Foo, B. Lecouat, V. Chandrasekhar, Adversarially learned anomaly detection, in: ICDM, 2018.
[25] H. Zenati, C.S. Foo, B. Lecouat, G. Manek, V.R. Chandrasekhar, Efficient gan-based anomaly detection, arXiv preprintarXiv:1802.06222.
[26] I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A.C. Courville, Y. Bengio, Generative adversarial nets, in: NIPS, 2014.
[27] J. Donahue, P. Krähenbühl, T. Darrell, Adversarial feature learning, arXiv preprint.
[28] Hochreiter, S.; Schmidhuber, J., Long short-term memory, Neural Comput., 9, 8, 1735-1780 (1997)
[29] Dubhashi, D. P.; Panconesi, A., Concentration of Measure for the Analysis of Randomized Algorithms (2009), Cambridge University Press · Zbl 1213.60006
[30] G. Pang, L. Cao, L. Chen, H. Liu, Learning representations of ultrahigh-dimensional data for random distance-based outlier detection, in: KDD, 2018.
[31] Clark, D. D.; Landau, S., The problem isn’t attribution: It’s multi-stage attacks, in, (Proceedings of the Re-Architecting the Internet Workshop (2010))
[32] Zhang, Y.; Luo, X.; Luo, H., A multi-step attack-correlation method with privacy protection, J. Commun. Inf. Networks, 1, 4, 133-142 (2016)
[33] Inokuchi, M.; Ohta, Y.; Kinoshita, S.; Yagyu, T.; Stan, O.; Bitton, R.; Elovici, Y.; Shabtai, A., Design procedure of knowledge base for practical attack graph generation, (AsiaCCS (2019))
[34] Atwood, J.; Towsley, D., Diffusion-convolutional neural networks, (NIPS (2016)), 1993-2001
[35] Wu, L.; Sun, P.; Fu, Y.; Hong, R.; Wang, X.; Wang, M., A neural influence diffusion model for social recommendation, (SIGIR (2019))
[36] Z. Yang, D. Yang, C. Dyer, X. He, A.J. Smola, E.H. Hovy, Hierarchical attention networks for document classification, in: NAANL, 2016.
[37] H. Zhang, I.J. Goodfellow, D.N. Metaxas, A. Odena, Self-attention generative adversarial networks, in: ICML, 2019.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.