×

Dynamic texture analysis using networks generated by deterministic partially self-avoiding walks. (English) Zbl 07527005

Summary: Dynamic textures are sequences of images (video) with the concept of texture patterns extended to the spatiotemporal domain. This research field has attracted attention due to the range of applications in different areas of science and the emergence of a large number of multimedia datasets. Unlike the static textures, the methods for dynamic texture analysis also need to deal with the time domain, which increases the challenge for representation. Thus, it is important to obtain features that properly describe the appearance and motion properties of the dynamic texture. In this paper, we propose a new method for dynamic texture analysis based on Deterministic Partially Self-avoiding Walks (DPSWs) and network science theory. Here, each pixel of the video is considered a vertex of the network and the edges are given by the movements of the deterministic walk between the pixels. The feature vector is obtained by calculating network measures from the networks generated by the DPSWs. The modeled networks incorporate important characteristics of the DPSWs transitivity and their spatial arrangement in the video. In this paper, two different strategies are tested to apply the DPSWs in the video and capture appearance and motion characteristics. One strategy applies the DPSWs in three orthogonal planes and the other is based on spatial and temporal descriptors. We validate our proposed method in Dyntex++ and UCLA (and their variants) databases, which are two well-known dynamic texture databases. The results have demonstrated the effectiveness of the proposed approach using a small feature vector for both strategies. Also, the proposed method improved the performance when compared to the previous DPSW-based method and the network-based method.

MSC:

82-XX Statistical mechanics, structure of matter

Software:

DynTex
Full Text: DOI

References:

[1] Chetverikov, D.; Péteri, R., A brief survey of dynamic texture description and recognition, (Computer Recognition Systems (2005), Springer), 17-26
[2] Doretto, G.; Jones, E.; Soatto, S., Spatially homogeneous dynamic textures, (Computer Vision-ECCV 2004 (2004), Springer), 591-602 · Zbl 1098.68752
[3] Nelson, R. C.; Polana, R., Qualitative recognition of motion using temporal texture, CVGIP: Image Underst., 56, 1, 78-89 (1992) · Zbl 0780.68102
[4] Xu, Y.; Quan, Y.; Ling, H.; Ji, H., Dynamic texture classification using dynamic fractal analysis, (2011 International Conference on Computer Vision (2011)), 1219-1226
[5] Zhao, Y.; Zhao, J.; Dong, E.; Chen, B.; Chen, J.; Yuan, Z.; Zhang, D., Dynamic texture modeling applied on computer vision based fire recognition, (International Conference on Artificial Intelligence and Computational Intelligence (2011), Springer), 545-553
[6] Milko, S.; Samset, E.; Kadir, T., Segmentation of the liver in ultrasound: a dynamic texture approach, Int. J. Comput. Assist. Radiol. Surg., 3, 1-2, 143 (2008)
[7] Wang, S.-J.; Yan, W.-J.; Li, X.; Zhao, G.; Fu, X., Micro-expression recognition using dynamic textures on tensor independent color space, (2014 22nd International Conference on Pattern Recognition (2014), IEEE), 4678-4683
[8] Gonçalves, W. N.; Bruno, O. M., Dynamic texture analysis and segmentation using deterministic partially self-avoiding walks, Expert Syst. Appl., 40, 11, 4283-4300 (2013)
[9] Kellokumpu, V.; Zhao, G.; Pietikäinen, M., Human activity recognition using a dynamic texture based method, (BMVC, Vol. 1 (2008)), 2
[10] Zhao, G.; Pietikäinen, M., Dynamic texture recognition using volume local binary patterns, (Vidal, R.; Heyden, A.; Ma, Y., Dynamical Vision (2007), Springer Berlin Heidelberg: Springer Berlin Heidelberg Berlin, Heidelberg), 165-177
[11] Ravichandran, A.; Vidal, R., Video registration using dynamic textures, IEEE Trans. Pattern Anal. Mach. Intell., 33, 1, 158-171 (2011)
[12] Polana, R.; Nelson, R., Motion-Based Recognition, 87-124 (1997), Springer Netherlands: Springer Netherlands Dordrecht, Ch. Temporal Texture and Activity Recognition · Zbl 0885.00043
[13] Doretto, G.; Chiuso, A.; Wu, Y.; Soatto, S., Dynamic textures, Int. J. Comput. Vis., 51, 2, 91-109 (2003) · Zbl 1030.68646
[14] de Mesquita, J. J.S.; Ribas, L. C.; Bruno, O. M., Randomized neural network based signature for dynamic texture classification, Expert Syst. Appl., 135, 194-200 (2019)
[15] Qiao, Y.; Weng, L., Hidden markov model based dynamic texture classification, IEEE Signal Process. Lett., 22, 4, 509-512 (2015)
[16] Gonçalves, W. N.; Machado, B. B.; Bruno, O. M., A complex network approach for dynamic texture recognition, Neurocomputing, 153, 0, 211-220 (2015)
[17] Ribas, L. C.; Gonçalves, W. N.; Bruno, O. M., Dynamic texture analysis with diffusion in networks, Digit. Signal Process., 92, 109-126 (2019)
[18] Wildes, R. P.; Bergen, J. R., Qualitative spatiotemporal analysis using an oriented energy representation, (European Conference on Computer Vision (2000), Springer: Springer Berlin, Heidelberg), 768-784
[19] Wu, P.; Ro, Y. M.; Won, C. S.; Choi, Y., Texture descriptors in mpeg-7, (Skarbek, W., Computer Analysis of Images and Patterns (2001), Springer Berlin Heidelberg: Springer Berlin Heidelberg Berlin, Heidelberg), 21-28 · Zbl 0998.68610
[20] J. Smith, C.-Y. Lin, M. Naphade, Video texture indexing using spatio-temporal wavelets, in: Image Processing. 2002. Proceedings. 2002 International Conference on, Vol. 2, 2002, pp. II-437-II-440.
[21] Dubois, S.; Péteri, R.; Ménard, M., Pattern Recognition and Image Analysis: 4th Iberian Conference, IbPRIA 2009 Póvoa de Varzim, Portugal, June 10-12, 2009, Proceedings, 314-321 (2009), Springer Berlin Heidelberg: Springer Berlin Heidelberg Berlin, Heidelberg, Ch. A Comparison of Wavelet Based Spatio-temporal Decomposition Methods for Dynamic Texture Recognition
[22] Gonçalves, W. N.; Machado, B. B.; Bruno, O. M., Spatiotemporal gabor filters: a new method for dynamic texture recognition, (Workshop de Visão Computacional (2011)), 184-189
[23] Tiwari, D.; Tyagi, V., Dynamic texture recognition: a review, (Information Systems Design and Intelligent Applications (2016), Springer), 365-373
[24] Haralick, R.; Shanmugam, K.; Dinstein, I., Textural features for image classification, IEEE Trans. Syst. Man Cybern., SMC-3, 6, 610-621 (1973)
[25] Ojala, T.; Pietikäinen, M.; Mäenpää, T., Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell., 24, 7, 971-987 (2002)
[26] Zhao, G.; Pietikainen, M., Dynamic texture recognition using local binary patterns with an application to facial expressions, IEEE Trans. Pattern Anal. Mach. Intell., 29, 6, 915-928 (2007)
[27] Tiwari, D.; Tyagi, V., Dynamic texture recognition based on completed volume local binary pattern, Multidimens. Syst. Signal Process., 27, 2, 563-575 (2016)
[28] Tiwari, D.; Tyagi, V., A novel scheme based on local binary pattern for dynamic texture recognition, Comput. Vis. Image Underst., 150, 58-65 (2016)
[29] Tiwari, D.; Tyagi, V., Improved weber’s law based local binary pattern for dynamic texture recognition, Multimedia Tools Appl., 1-18 (2016)
[30] Tiwari, D.; Tyagi, V., Dynamic texture recognition using multiresolution edge-weighted local structure pattern, Comput. Electr. Eng., 62, 485-498 (2017)
[31] Gonçalves, W. N.; Bruno, O. M., Dynamic texture segmentation based on deterministic partially self-avoiding walks, Comput. Vis. Image Underst., 117, 9, 1163-1174 (2013)
[32] Lima, G. F.; Martinez, A. S.; Kinouchi, O., Deterministic walks in random media, Phys. Rev. Lett., 87, Article 010603 pp. (2001)
[33] Backes, A. R.; Gonçalves, W. N.; Martinez, A. S.; Bruno, O. M., Texture analysis and classification using deterministic tourist walk, Pattern Recognit., 43, 3, 685-694 (2010) · Zbl 1187.68426
[34] Backes, A. R.; Martinez, A. S.; Bruno, O. M., Texture analysis using graphs generated by deterministic partially self-avoiding walks, Pattern Recognit., 44, 8, 1684-1689 (2011)
[35] Goncalves, W. N.; Backes, A. R.; Martinez, A. S.; Bruno, O. M., Texture descriptor based on partially self-avoiding deterministic walker on networks, Expert Syst. Appl., 39, 15, 11818-11829 (2012)
[36] Campiteli, M.; Martinez, A.; Bruno, O., An image analysis methodology based on deterministic tourist walks, (Advances in Artificial Intelligence - IBERAMIA-SBIA 2006. Advances in Artificial Intelligence - IBERAMIA-SBIA 2006, Lecture Notes in Computer Science, vol. 4140 (2006), Springer Berlin Heidelberg), 159-167
[37] Gonçalves, W. N.; Martinez, A. S.; Bruno, O. M., Complex network classification using partially self-avoiding deterministic walks, Chaos, 22, 3, Article 033139 pp. (2012) · Zbl 1319.05121
[38] Newman, M. E.; Park, J., Why social networks are different from other types of networks, Phys. Rev. E, 68, 3, Article 036122 pp. (2003)
[39] Barabasi, A.-L.; Oltvai, Z. N., Network biology: understanding the cell’s functional organization, Nature Rev. Genet., 5, 2, 101-113 (2004)
[40] Bornholdt, S.; Schuster, H. G., Handbook of Graphs and Networks: From the Genome to the Internet (2006), John Wiley & Sons
[41] Holme, P., Congestion and centrality in traffic flow on complex networks, Adv. Complex Syst., 6, 02, 163-176 (2003) · Zbl 1203.90047
[42] L.C. Ribas, J.J.de M. Sá Junior, L.F.S. Scabini, O.M. Bruno, Fusion of complex networks and randomized neural networks for texture analysis, CoRR abs/1806.09170. arXiv:1806.09170.
[43] Ribas, L. C.; Neiva, M. B.; Bruno, O. M., Distance transform network for shape analysis, Inform. Sci., 470, 28-42 (2019)
[44] Ghanem, B.; Ahuja, N., Maximum margin distance learning for dynamic texture recognition, (Proceedings of the 11th European Conference on Computer Vision: Part II, ECCV’10 (2010), Springer-Verlag: Springer-Verlag Berlin, Heidelberg), 223-236
[45] Péteri, R.; Fazekas, S.; Huiskes, M. J., Dyntex: A comprehensive database of dynamic textures, Pattern Recognit. Lett., 31, 12, 1627-1632 (2010)
[46] Chan, A. B.; Vasconcelos, N., Probabilistics for the classification of auto-regressive visual processes, (Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 1 (2005), IEEE), 846-851
[47] Ravichandran, A.; Chaudhry, R.; Vidal, R., View-invariant dynamic texture recognition using a bag of dynamical systems, (2009 IEEE Conference on Computer Vision and Pattern Recognition (2009)), 1651-1657
[48] Chan, A. B.; Vasconcelos, N., Classifying video with kernel dynamic textures, (CVPR (2007)), 1-6
[49] Xu, Y.; Huang, S.; Ji, H.; Fermüller, C., Scale-space texture description on sift-like textons, Comput. Vis. Image Underst., 116, 9, 999-1013 (2012)
[50] Arashloo, S. R.; Kittler, J., Dynamic texture recognition using multiscale binarized statistical image features, IEEE Trans. Multimed., 16, 8, 2099-2109 (2014)
[51] Wang, Y.; Hu, S., Exploiting high level feature for dynamic textures recognition, Neurocomputing, 154, 217-224 (2015)
[52] Rivera, A. R.; Chae, O., Spatiotemporal directional number transitional graph for dynamic texture recognition, IEEE Trans. Pattern Anal. Mach. Intell., 37, 10, 2146-2152 (2015)
[53] Ji, H.; Yang, X.; Ling, H.; Xu, Y., Wavelet domain multifractal analysis for static and dynamic texture classification, IEEE Trans. Image Process., 22, 1, 286-299 (2013) · Zbl 1373.94187
[54] Wang, Y.; Hu, S., Chaotic features for dynamic textures recognition, Soft Comput., 20, 5, 1977-1989 (2016)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.