Fast unsupervised embedding learning with anchor-based graph. (English) Zbl 07825391
Summary: As graph technology is widely used in unsupervised dimensionality reduction, many methods automatically construct a full connection graph to learn the structure of data, and then preserve critical information on data in subspace. The construction of a full connection graph with heavy computational complexity, however, is separated from the optimization of transformation matrix. In order to solve significant computational burden, we design anchor-based graph and unify the construction of graph and optimization of transformation matrix into a framework called fast unsupervised embedding learning with anchor-based graph (FUAG) which not only can avoid the impact of noises and redundant features in original space, but also can capture local structure of data in subspace precisely. Our method additionally incorporates the discriminant information of data captured by using trace difference form. Meanwhile, it optimizes the anchor-based graph partitioning problem with Constrained Laplacian Rank in order to ensure that the number of connected components is exactly equal to the number of classes. We also impose \(\ell_0\) norm constraint on each point to avoid trivial solutions and propose an efficient iterative algorithm. Experimental results on both synthetic and real-world datasets demonstrate the promising performance of the proposed algorithm.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
References:
| [1] | Bober, Miroslaw, Mpeg-7 visual shape descriptors, IEEE Trans. Circuits Syst. Video Technol., 11, 6, 716-719 (2001) · Zbl 0998.68597 |
| [2] | Cai, Deng; He, Xiaofei; Han, Jiawei, Spectral regression: a unified subspace learning framework for content-based image retrieval, (Proceedings of the 15th ACM international conference on Multimedia (2007)), 403-412 |
| [3] | Cai, Deng; He, Xiaofei; Han, Jiawei, Locally consistent concept factorization for document clustering, IEEE Trans. Knowl. Data Eng., 23, 6, 902-913 (2010) |
| [4] | Fan, Ky, On a theorem of weyl concerning eigenvalues of linear transformations i, Proc. Natl. Acad. Sci. U.S.A., 35, 11, 652 (1949) · Zbl 0041.00602 |
| [5] | Mark Fanty, Ronald Cole, Spoken letter recognition, in: Advances in neural information processing systems, 1991, pp. 220-226. |
| [6] | Gao, Quanxue; Huang, Yunfang; Gao, Xinbo; Shen, Weiguo; Zhang, Hailin, A novel semi-supervised learning for face recognition, Neurocomputing, 152, 69-76 (2015) |
| [7] | Gao, Quanxue; Wang, Qianqian; Huang, Yunfang; Gao, Xinbo; Hong, Xin; Zhang, Hailin, Dimensionality reduction by integrating sparse representation and fisher criterion and its applications, IEEE Trans. Image Process., 24, 12, 5684-5695 (2015) · Zbl 1408.94191 |
| [8] | Georghiades, Athinodoros S.; Belhumeur, Peter N.; Kriegman, David J., From few to many: Illumination cone models for face recognition under variable lighting and pose, IEEE Trans. Pattern Anal. Mach. Intell., 23, 6, 643-660 (2001) |
| [9] | He, Ran; Cao, Jie; Song, Lingxiao; Sun, Zhenan; Tan, Tieniu, Adversarial cross-spectral face completion for nir-vis face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 42, 5, 1025-1037 (2019) |
| [10] | Xiaofei He, Deng Cai, Shuicheng Yan, Hong-Jiang Zhang, Neighborhood preserving embedding, in: Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, vol. 2, IEEE, 2005, pp. 1208-1213. |
| [11] | Xiaofei He, Partha Niyogi, Locality preserving projections, in: Advances in neural information processing systems, 2004, pp. 153-160. |
| [12] | He, Xiaofei; Yan, Shuicheng; Yuxiao, Hu.; Zhang, Hong-Jiang, Learning a locality preserving subspace for visual recognition, (Proceedings Ninth IEEE International Conference on Computer Vision, IEEE (2003)), 385-392 |
| [13] | Huang, Yi; Dong, Xu.; Nie, Feiping, Semi-supervised dimension reduction using trace ratio criterion, IEEE Trans. Neural Networks Learn. Syst., 23, 3, 519-526 (2012) |
| [14] | Hurtik, Petr; Molek, Vojtech; Perfilieva, Irina, Novel dimensionality reduction approach for unsupervised learning on small datasets, Pattern Recogn., 103, Article 107291 pp. (2020) |
| [15] | Kim, Minyoung; Pavlovic, Vladimir, Central subspace dimensionality reduction using covariance operators, IEEE Trans. Pattern Anal. Mach. Intell., 33, 4, 657-670 (2010) |
| [16] | Martin H.C. Law, Anil K. Jain, Incremental nonlinear dimensionality reduction by manifold learning, IEEE Trans. Pattern Anal. Mach. Intell. 28(3) (2006) 377-391. |
| [17] | Liu, Yang; Nie, Feiping; Gao, Quanxue; Gao, Xinbo; Han, Jungong; Shao, Ling, Flexible unsupervised feature extraction for image classification, Neural Networks, 115, 65-71 (2019) · Zbl 1434.68427 |
| [18] | Yun Liu, Rui Zhang, Feiping Nie, Xuelong Li, and Chris Ding, Supervised dimensionality reduction methods via recursive regression, IEEE Trans. Neural Networks Learn. Syst. (2019). |
| [19] | Mao, Qi; Wang, Li; Goodison, Steve; Sun, Yijun, Dimensionality reduction via graph structure learning, (Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015)), 765-774 |
| [20] | Sameer A. Nene, Shree K. Nayar, Hiroshi Murase, et al., Columbia object image library (coil-100), 1996. |
| [21] | Nie, Feiping; Dong, Xia; Li, Xuelong, Unsupervised and semisupervised projection with graph optimization, IEEE Trans. Neural Networks Learn. Syst., 32, 4, 1547-1559 (2020) |
| [22] | Nie, Feiping; Wang, Cheng-Long; Li, Xuelong, K-multiple-means: A multiple-means clustering method with specified k clusters, (Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2019)), 959-967 |
| [23] | Nie, Feiping; Wang, Xiaoqian; Deng, Cheng; Huang, Heng, Learning a structured optimal bipartite graph for co-clustering, Advances in Neural Information Processing Systems, 30, 4129-4138 (2017) |
| [24] | Feiping Nie, Xiaoqian Wang, Michael Jordan, and Heng Huang, The constrained laplacian rank algorithm for graph-based clustering, in: Proceedings of the AAAI conference on artificial intelligence, vol. 30, 2016. |
| [25] | Nie, Feiping; Wang, Zheng; Wang, Rong; Li, Xuelong, Submanifold-preserving discriminant analysis with an auto-optimized graph, IEEE Trans. Cybern., 50, 8, 3682-3695 (2019) |
| [26] | Nie, Feiping; Zhu, Wei; Li, Xuelong, Unsupervised large graph embedding, Proceedings of the AAAI Conference on Artificial Intelligence, 31, 2422-2428 (2017) |
| [27] | Pang, Yanwei; Zhou, Bo; Nie, Feiping, Simultaneously learning neighborship and projection matrix for supervised dimensionality reduction, IEEE Trans. Neural Networks Learn. Syst., 30, 9, 2779-2793 (2019) |
| [28] | Pei, Xiaobing; Chen, Chuanbo; Guan, Yue, Joint sparse representation and embedding propagation learning: A framework for graph-based semisupervised learning, IEEE Trans. Neural Networks Learn. Syst., 28, 12, 2949-2960 (2016) |
| [29] | Emanuel, F.; Petricoin, I. I.I.; Ornstein, David K.; Paweletz, Cloud P.; Ardekani, Ali; Hackett, Paul S.; Hitt, Ben A.; Velassco, Alfredo; Trucco, Christian; Wiegand, Laura; Wood, Kamillah, Serum proteomic patterns for detection of prostate cancer, J. Natl. Cancer Inst., 94, 20, 1576-1578 (2002) |
| [30] | Sim, Terence; Baker, Simon; Bsat, Maan, The cmu pose, illumination, and expression (pie) database, (Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition, IEEE (2002)), 53-58 |
| [31] | Amarnag Subramanya, Partha Pratim Talukdar, Graph-based semi-supervised learning, Synthesis Lectures Artif. Intell. Mach. Learn. 8(4) (2014) 1-125. · Zbl 1296.68002 |
| [32] | Tasoulis, Sotiris; Pavlidis, Nicos G.; Roos, Teemu, Nonlinear dimensionality reduction for clustering, Pattern Recogn., 107, Article 107508 pp. (2020) |
| [33] | Jingyu Wang, Fangyuan Xie, Feiping Nie, and Xuelong Li, Unsupervised adaptive embedding for dimensionality reduction, IEEE Trans. Neural Networks Learn. Syst. (2021). |
| [34] | Wang, Jung-Ying, Application of support vector machines in bioinformatics, National Taiwan University (2002) |
| [35] | Wang, Li; Mao, Qi, Probabilistic dimensionality reduction via structure learning, IEEE Trans. Pattern Anal. Mach. Intell., 41, 1, 205-219 (2017) |
| [36] | Wold, Svante; Esbensen, Kim; Geladi, Paul, Principal component analysis, Chemometrics Intell. Lab. Syst., 2, 1-3, 37-52 (1987) |
| [37] | Xiang, Shiming; Nie, Feiping; Zhang, Changshui; Zhang, Chunxia, Nonlinear dimensionality reduction with local spline embedding, IEEE Trans. Knowl. Data Eng., 21, 9, 1285-1298 (2008) |
| [38] | Yan, Yuanjie; Hao, Hongyan; Xu, Baile; Zhao, Jian; Shen, Furao, Image clustering via deep embedded dimensionality reduction and probability-based triplet loss, IEEE Trans. Image Process., 29, 5652-5661 (2020) |
| [39] | Yu, Jun; Tao, Dacheng; Wang, Meng, Adaptive hypergraph learning and its application in image classification, IEEE Trans. Image Process., 21, 7, 3262-3272 (2012) · Zbl 1381.62216 |
| [40] | Zhang, Xinyu; Gao, Hongbo; Li, Guopeng; Zhao, Jianhui; Huo, Jianghao; Yin, Jialun; Liu, Yuchao; Zheng, Li, Multi-view clustering based on graph-regularized nonnegative matrix factorization for object recognition, Inf. Sci., 432, 463-478 (2018) · Zbl 1436.68323 |
| [41] | Wang, Rong; Zhang, Canyu; Bian, Jintang; Wang, Zheng; Nie, Feiping; Li, Xuelong, Sparse and flexible projections for unsupervised feature selection, IEEE Trans. Knowl. Data Eng. (2022), In press |
| [42] | Zhang, Canyu; Nie, Feiping; Wang, Zheng; Wang, Rong; Li, Xuelong, Fast local representation learning via adaptive anchor graph for image retrieval, Inf. Sci., 578, 870-886 (2021) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
