×

A joint-norm distance metric 2DPCA for robust dimensionality reduction. (English) Zbl 1535.68312

Summary: Two-dimensional principal component analysis (2DPCA) is one of the most representative dimensionality reduction methods and the robustness degrades in the presence of noise. To improve the robustness, several 2DPCA methods using similar distance metrics were proposed to achieve the objective of the maximum projection distance or the minimum reconstruction error. However, these two objectives cannot be accomplished simultaneously or are constrained by the same projection matrix. To handle these problems, we propose a generalized robust 2DPCA method called 2DPCA-2-Lp, which joins 2-norm and \(l_p\)-norm metrics in the objective function. It aims to maximize the ratio of projected vectors to image row vectors and to fulfill the dual objectives of directly maximizing projection distances and indirectly minimizing reconstruction errors. 2DPCA-2-Lp measures the distances between row vectors of matrices rather than the distances between matrices, thus remarkably enhancing the robustness. Furthermore, the relationship between reconstruction errors and projection distances for different parameter values is analyzed theoretically under the joint-norm metric. Then, the closed greedy iterative algorithm is developed for obtaining optimal solutions. Finally, extensive experimental results on four datasets show that 2DPCA-2-Lp outperforms most existing 2DPCA methods in terms of reconstruction and classification performances and has better robustness against noise.

MSC:

68T09 Computational aspects of data analysis and big data
62H25 Factor analysis and principal components; correspondence analysis
65F15 Numerical computation of eigenvalues and eigenvectors of matrices
Full Text: DOI

References:

[1] Camastra, F.; Staiano, A., Intrinsic dimension estimation: advances and open problems, Inf. Sci., 328, 26-41 (2016) · Zbl 1392.62171
[2] Irfan, M.; Jiangbin, Z.; Iqbal, M.; Arif, M. H., A novel lifelong learning model based on cross domain knowledge extraction and transfer to classify underwater images, Inf. Sci., 552, 80-101 (2021)
[3] Irfan, M.; Jiangbin, Z.; Iqbal, M.; Masood, Z.; Arif, M. H., Knowledge extraction and retention based continual learning by using convolutional autoencoder-based learning classifier system, Inf. Sci., 591, 287-305 (2022)
[4] Diaz-Chito, K.; Ferri, F. J.; Hernandez-Sabate, A., An overview of incremental feature extraction methods based on linear subspaces, Knowl.-Based Syst., 145, 219-235 (2018)
[5] Tenenbaum, J.; de Silva, V.; Langford, J., A global geometric framework for nonlinear dimensionality reduction, Science, 290, 5500, 2319-2323 (2000)
[6] Jolliffe, I. T.; Cadima, J., Principal component analysis: a review and recent developments, Philos. Trans. R. Soc. A, Math. Phys. Eng. Sci., 374, 2065, Article 20150202 pp. (2016) · Zbl 1353.62067
[7] Belhumeur, P. N.; Hespanha, J. P.; Kriegman, D. J., Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., 19, 7, 711-720 (1997)
[8] Hyvärinen, A.; Oja, E., Independent component analysis: algorithms and applications, Neural Netw., 13, 4, 411-430 (2000)
[9] Yang, J.; Zhang, D.; Frangi, A. F.; Jing, Y., Two-dimensional PCA: a new approach to appearance-based face representation and recognition, IEEE Trans. Pattern Anal. Mach. Intell., 26, 1, 131-137 (2004)
[10] Qian, J.; Zhu, S.; Wong, W. K.; Zhang, H.; Lai, Z.; Yang, J., Dual robust regression for pattern classification, Inf. Sci., 546, 1014-1029 (2021) · Zbl 1475.62190
[11] Nguyen, B.; Morell, C.; De Baets, B., Distance metric learning for ordinal classification based on triplet constraints, Knowl.-Based Syst., 142, 17-28 (2018)
[12] Li, B.; Fan, Z.; Zhang, X.; Huang, D., Robust dimensionality reduction via feature space to feature space distance metric learning, Neural Netw., 112, 1-14 (2019) · Zbl 1435.68270
[13] Kwak, N., Principal component analysis based on L1-norm maximization, IEEE Trans. Pattern Anal. Mach. Intell., 30, 9, 1672-1680 (2008)
[14] Nie, F.; Huang, H.; Ding, C.; Luo, D.; Wang, H., Robust principal component analysis with non-greedy L1-norm maximization, (Proc. 22nd IJCAI (2011)), 1433-1438
[15] Li, X.; Pang, Y.; Yuan, Y., L1-norm-based 2DPCA, IEEE Trans. Syst. Man Cybern., Part B, Cybern., 40, 4, 1170-1175 (2010)
[16] Wang, R.; Nie, F.; Yang, X.; Gao, F.; Yao, M., Robust 2DPCA with non-greedy \(\ell_1\) -norm maximization for image analysis, IEEE Trans. Cybern., 45, 5, 1108-1112 (2015)
[17] Meng, D.; Zhao, Q.; Xu, Z., Improve robustness of sparse PCA by L-1-norm maximization, Pattern Recognit., 45, 1, 487-497 (2012) · Zbl 1225.68202
[18] Wang, H.; Wang, J., 2DPCA with L1-norm for simultaneously robust and sparse modelling, Neural Netw., 46, 190-198 (2013) · Zbl 1296.68150
[19] Li, B. N.; Yu, Q.; Wang, R.; Xiang, K.; Wang, M.; Li, X., Block principal component analysis with nongreedy \(\ell_1\) -norm maximization, IEEE Trans. Cybern., 46, 11, 2543-2547 (2016)
[20] Wang, H., Block principal component analysis with L1-norm for image analysis, Pattern Recognit. Lett., 33, 5, 537-542 (2012)
[21] Kwak, N., Principal component analysis by \(L_p\)-norm maximization, IEEE Trans. Cybern., 44, 5, 594-609 (2014)
[22] Liang, Z.; Xia, S.; Zhou, Y.; Zhang, L.; Li, Y., Feature extraction based on Lp-norm generalized principal component analysis, Pattern Recognit. Lett., 34, 9, 1037-1045 (2013)
[23] Wang, J., Generalized 2-D principal component analysis by Lp-norm for image analysis, IEEE Trans. Cybern., 46, 3, 792-803 (2016)
[24] Hunter, D.; Lange, K., A tutorial on MM algorithms, Am. Stat., 58, 1, 30-37 (2004)
[25] Wang, Q.; Gao, Q.; Gao, X.; Nie, F., l(2, p)-norm based PCA for image recognition, IEEE Trans. Image Process., 27, 3, 1336-1346 (2018) · Zbl 1409.94616
[26] Bi, P.; Xu, J.; Du, X.; Li, J.; Chen, G., l(2, p)-norm sequential bilateral 2DPCA: a novel robust technology for underwater image classification and representation, Neural Comput. Appl., 32, 22, SI, 17027-17041 (2020)
[27] Zhou, G.; Xu, G.; Hao, J.; Chen, S.; Xu, J.; Zheng, X., Generalized centered 2-D principal component analysis, IEEE Trans. Cybern., 51, 3, 1666-1677 (2021)
[28] Li, T.; Li, M.; Gao, Q.; Xie, D., F-norm distance metric based robust 2DPCA and face recognition, Neural Netw., 94, 204-211 (2017) · Zbl 1429.68231
[29] Ding, C.; Zhou, D.; He, X.; Zha, H., R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization, (Proc. Int. Conf. Mach. Learn. (2006)), 281-288
[30] Gao, Q.; Ma, L.; Liu, Y.; Gao, X.; Nie, F., Angle 2DPCA: a new formulation for 2DPCA, IEEE Trans. Cybern., 48, 5, 1672-1678 (2018)
[31] Huang, P.; Ye, Q.; Zhang, F.; Yang, G.; Zhu, W.; Yang, Z., Double \(L_{2 , p}\)-norm based PCA for feature extraction, Inf. Sci., 573, 345-359 (2021) · Zbl 1531.68079
[32] Bi, P.; Deng, Y.; Du, X., A robust optimal mean cosine angle 2DPCA for image feature extraction, Neural Comput. Appl., 34, 22, SI, 20117-20134 (2022)
[33] Wang, X.; Shi, L.; Liu, J.; Zhang, M., Cosine 2DPCA with weighted projection maximization, IEEE Trans. Neural Netw. Learn. Syst. Online (2022)
[34] Wang, Y.; Li, Q., Robust 2DPCA with F-norm minimization, IEEE Access, 7, 68083-68090 (2019)
[35] Gao, Q.; Xu, S.; Chen, F.; Ding, C.; Gao, X.; Li, Y., \( R_1-2\)-DPCA and face recognition, IEEE Trans. Cybern., 49, 4, 1212-1223 (2019)
[36] Nie, F.; Tian, L.; Huang, H.; Ding, C., Non-greedy L21-norm maximization for principal component analysis, IEEE Trans. Image Process., 30, 5277-5286 (2021)
[37] Shi, X.; Nie, F.; Lai, Z.; Guo, Z., Robust principal component analysis via optimal mean by joint \(\ell_{2 , 1}\) and Schatten p-norms minimization, Neurocomputing, 283, 205-213 (2018)
[38] Kong, X.; Song, Y.; Liu, J.; Zheng, C.; Yuan, S.; Wang, J.; Dai, L., Joint Lp-norm and L-2,L-1-norm constrained graph laplacian PCA for robust tumor sample clustering and gene network module discovery, Front. Genet., 12, Article 621317 pp. (2021)
[39] Fu, L.; Li, Z.; Ye, Q.; Yin, H.; Liu, Q.; Chen, X.; Fan, X.; Yang, W.; Yang, G., Learning robust discriminant subspace based on joint L-2,L-p and L-2,L-s-norm distance metrics, IEEE Trans. Neural Netw. Learn. Syst., 33, 1, 130-144 (2022)
[40] Dong, W.; Wu, X., Robust low rank subspace segmentation via joint-norm minimization, Neural Process. Lett., 48, 1, 299-312 (2018)
[41] Zhang, Z.; Li, F.; Zhao, M.; Zhang, L.; Yan, S., Robust neighborhood preserving projection by nuclear/L2, 1-norm regularization for image feature extraction, IEEE Trans. Image Process., 26, 4, 1607-1622 (2017) · Zbl 1409.94782
[42] Shi, X.; Yang, Y.; Guo, Z.; Lai, Z., Face recognition by sparse discriminant analysis via joint L2,L1-norm minimization, Pattern Recognit., 47, 7, 2447-2453 (2014)
[43] Yang, Z.; Ye, Q.; Chen, Q.; Ma, X.; Fu, L.; Yang, G.; Yan, H.; Liu, F., Frobust discriminant feature selection via joint L-2,L-1-norm distance minimization and maximization, Knowl.-Based Syst., 207, Article 106090 pp. (2020)
[44] Boyd, S.; Vandenberghe, L., Convex Optimization (2004), Cambridge University Press · Zbl 1058.90049
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.