Document Zbl 1437.62234

On Bayesian new edge prediction and anomaly detection in computer networks. (English) Zbl 1437.62234

Ann. Appl. Stat. 13, No. 4, 2586-2610 (2019).

Summary: Monitoring computer network traffic for anomalous behaviour presents an important security challenge. Arrivals of new edges in a network graph represent connections between a client and server pair not previously observed, and in rare cases these might suggest the presence of intruders or malicious implants. We propose a Bayesian model and anomaly detection method for simultaneously characterising existing network structure and modelling likely new edge formation. The method is demonstrated on real computer network authentication data and successfully identifies some machines which are known to be compromised.

Cited in 6 Documents

MSC:

62H30	Classification and discrimination; cluster analysis (statistical aspects)
60G55	Point processes (e.g., Poisson, Cox, Hawkes processes)
62P25	Applications of statistics to social sciences
62H22	Probabilistic graphical models

Keywords:

computer networks; anomaly detection; new edges; Bayesian inference; network graph

Software:

PyHawkes

Cite Review PDF

Full Text: DOI Euclid

References:

[1]	Cahill, M. H., Lambert, D., Pinheiro, J. C. and Sun, D. X. (2002). Detecting fraud in the real world. In Handbook of Massive Data Sets 911-929. Kluwer Academic, Dordrecht. · Zbl 1010.68064
[2]	Cho, H., Dhillon, I. S., Guan, Y. and Sra, S. (2004). Minimum sum-squared residue co-clustering of gene expression data. In Proceedings of the Fourth SIAM International Conference on Data Mining 114-125. SIAM, Philadelphia, PA.
[3]	Cox, D. R. (1972). Regression models and life-tables. J. Roy. Statist. Soc. Ser. B 34 187-220. · Zbl 0243.62041 · doi:10.1111/j.2517-6161.1972.tb00899.x
[4]	Dhillon, I. S. (2001). Co-clustering documents and words using bipartite spectral graph partitioning. In Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 269-274. ACM, New York.
[5]	Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver & Boyd, Edinburgh. · JFM 51.0414.08
[6]	Ghahramani, Z., Griffiths, T. L. and Sollich, P. (2007). Bayesian nonparametric latent feature models. In Bayesian Statistics 8. Oxford Sci. Publ. 201-226. Oxford Univ. Press, Oxford. · Zbl 1252.62004
[7]	Hall, E. C. and Willett, R. M. (2016). Tracking dynamic point processes on networks. IEEE Trans. Inform. Theory 62 4327-4346. · Zbl 1359.94952 · doi:10.1109/TIT.2016.2568202
[8]	Heard, N. and Metelli, S. (2014). Modelling new edge formation in a computer network through Bayesian variable selection. In Joint Intelligence and Security Informatics Conference (JISIC), 2014 European 272-275. IEEE, New York.
[9]	Heard, N. and Metelli, S. (2016). Model-based clustering and new edge modelling in a large computer network. In IEEE International Conference on Intelligence and Security Informatics (ISI), 2016 91-96. IEEE, New York.
[10]	Holland, P. W., Laskey, K. B. and Leinhardt, S. (1983). Stochastic blockmodels: First steps. Soc. Netw. 5 109-137.
[11]	Kent, A. D. (2015a). Comprehensive, multi-source cyber-security events. Los Alamos National Laboratory, Washington, DC.
[12]	Kent, A. D. (2015b). Cybersecurity data sources for dynamic network research. In Dynamic Networks in Cybersecurity Imperial College Press, London.
[13]	Lee, M., Shen, H., Huang, J. Z. and Marron, J. S. (2010). Biclustering via sparse singular value decomposition. Biometrics 66 1087-1095. · Zbl 1233.62182 · doi:10.1111/j.1541-0420.2010.01392.x
[14]	Li, S., Xie, Y., Farajtabar, M., Verma, A. and Song, L. (2017). Detecting changes in dynamic events over networks. IEEE Trans. Signal Inform. Process. Netw. 3 346-359.
[15]	Linderman, S. W. and Adams, R. P. (2014). Discovering latent network structure in point process data. In Proceedings of the 31st International Conference on Machine Learning 1413-1421.
[16]	Meinshausen, N. and Bühlmann, P. (2010). Stability selection. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 417-473. · Zbl 1411.62142
[17]	Metelli, S. and Heard, N. (2019). Supplement to “On Bayesian new edge prediction and anomaly detection in computer networks.” DOI:10.1214/19-AOAS1286SUPP. · Zbl 1437.62234
[18]	Neil, J., Hash, C., Brugh, A., Fisk, M. and Storlie, C. B. (2013). Scan statistics for the online detection of locally anomalous subgraphs. Technometrics 55 403-414.
[19]	Patcha, A. and Park, J. (2007). An overview of anomaly detection techniques: Existing solutions and latest technological trends. Comput. Netw. 51 3448-3470.
[20]	Pearson, K. (1933). On a method of determining whether a sample of size n supposed to have been drawn from a parent population having a known probability integral has probably been drawn at random. Biometrika 25 379-410. · Zbl 0008.12303 · doi:10.1093/biomet/25.3-4.379
[21]	Perry, P. O. and Wolfe, P. J. (2013). Point process modelling for directed interaction networks. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 821-849. · Zbl 1411.60076 · doi:10.1111/rssb.12013
[22]	Rohe, K., Chatterjee, S. and Yu, B. (2011). Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Statist. 39 1878-1915. · Zbl 1227.62042 · doi:10.1214/11-AOS887
[23]	Rubin-Delanchy, P., Priebe, C. E., Tang, M. and Cape, J. (2017). A statistical interpretation of spectral embedding: The generalised random dot product graph. Preprint. Available at arXiv:1709.05506.
[24]	Scholz, F.-W. and Stephens, M. A. (1987). \(k\)-sample Anderson-Darling tests. J. Amer. Statist. Assoc. 82 918-924.
[25]	Sill, M., Kaiser, S., Benner, A. and Kopp-Schneider, A. (2011). Robust biclustering by sparse singular value decomposition incorporating stability selection. Bioinformatics 27 2089-2097.
[26]	Snijders, T., van de Bunt, G. and Steglich, C. (2010). Introduction to stochastic actor-based models for network dynamics. Soc. Netw. 32 44-60.
[27]	Solomon, H. and Stephens, M. (1978). Approximations to density functions using Pearson curves. J. Amer. Statist. Assoc. 73 153-160.
[28]	Sussman, D. L., Tang, M., Fishkind, D. E. and Priebe, C. E. (2012). A consistent adjacency spectral embedding for stochastic blockmodel graphs. J. Amer. Statist. Assoc. 107 1119-1128. · Zbl 1443.62188 · doi:10.1080/01621459.2012.699795
[29]	Taddy, M. A. (2010). Autoregressive mixture models for dynamic spatial Poisson processes: Application to tracking intensity of violent crime. J. Amer. Statist. Assoc. 105 1403-1417. · Zbl 1388.62379 · doi:10.1198/jasa.2010.ap09655
[30]	Turcotte, M. J., Heard, N. A. and Neil, J. (2014). Detecting localised anomalous behaviour in a computer network. In Advances in Intelligent Data Analysis XIII—13th International Symposium 321-332. Springer, Berlin.
[31]	Zammit-Mangion, A., Alan Dewar, M., Kadirkamanathan, V. and Sanguinetti, G. (2012). Point process modelling of the Afghan War Diary. Proc. Natl. Acad. Sci. USA 109 12414-12419.
[32]	Zhou, K., Zha, H. and Song, L. (2013). Learning triggering kernels for multi-dimensional Hawkes processes. In Proceedings of the 30th International Conference on Machine Learning 28 1301-1309.

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.