×

Graph optimization for dimensionality reduction with sparsity constraints. (English) Zbl 1227.68097

Summary: Graph-based dimensionality reduction (DR) methods play an increasingly important role in many machine learning and pattern recognition applications. In this paper, we propose a novel graph-based learning scheme to conduct graph optimization for dimensionality reduction with sparsity constraints (GODRSC). Different from most of graph-based DR methods where graphs are generally constructed in advance, GODRSC aims to simultaneously seek a graph and a projection matrix preserving such a graph in one unified framework, resulting in an automatically updated graph. Moreover, by applying an \(l_{1}\) regularizer, a sparse graph is achieved, which models the “locality” structure of data and contains natural discriminating information. Finally, extensive experiments on several publicly available UCI and face databases verify the feasibility and effectiveness of the proposed method.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
68T10 Pattern recognition, speech recognition

Software:

SLEP
Full Text: DOI

References:

[1] Yan, S. C.; Xu, D.; Zhang, B. Y.; Zhang, H. J.; Yang, Q.; Lin, S., Graph embedding and extensions: a general framework for dimensionality reduction, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1, 40-51 (2007)
[2] Tenenbaum, J. B.; Silva, V.; Langford, J., A global geometric framework for nonlinear dimensionality reduction, Science, 290, 5500, 2319-2323 (2000)
[3] Roweis, S. T.; Saul, L. K., Nonlinear dimensionality reduction by locally linear embedding, Science, 290, 5500, 2323-2326 (2000)
[4] Belkin, M.; Niyogi, P., Laplacian eigenmaps for dimensionality reduction and data representation, Neural Computation, 15, 6, 1373-1396 (2003) · Zbl 1085.68119
[5] X.F. He, P. Niyogi, Locality preserving projections, in: Neural Information Processing Systems (NIPS), 2003.; X.F. He, P. Niyogi, Locality preserving projections, in: Neural Information Processing Systems (NIPS), 2003.
[6] W. Liu, S.-F. Chang, Robust multi-class transductive learning with graphs, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.; W. Liu, S.-F. Chang, Robust multi-class transductive learning with graphs, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
[7] T. Jebara, J. Wang, S. Chang, Graph construction and b-matching for semi-supervised learning, in: International Conference on Machine Learning (ICML), 2009.; T. Jebara, J. Wang, S. Chang, Graph construction and b-matching for semi-supervised learning, in: International Conference on Machine Learning (ICML), 2009.
[8] C. Cortes, M. Mohri, On transductive regression, in: Neural Information Processing Systems (NIPS), 2007.; C. Cortes, M. Mohri, On transductive regression, in: Neural Information Processing Systems (NIPS), 2007.
[9] M. Maier, U. Luxburg, Influence of graph construction on graph-based clustering measures, in: Neural Information Processing Systems (NIPS), 2008.; M. Maier, U. Luxburg, Influence of graph construction on graph-based clustering measures, in: Neural Information Processing Systems (NIPS), 2008.
[10] X. Zhu, Semi-supervised Learning Literature Survey, Technical Report, 2008.; X. Zhu, Semi-supervised Learning Literature Survey, Technical Report, 2008.
[11] Zhang, L.; Qiao, L.; Chen, S., Graph-optimized locality preserving projections, Pattern Recognition, 43, 6, 1993-2002 (2010) · Zbl 1191.68611
[12] Qiao, L. S.; Chen, S. C.; Tan, X. Y., Sparsity preserving projections with applications to face recognition, Pattern Recognition, 43, 1, 331-341 (2010) · Zbl 1186.68421
[13] Wright, J.; Yang, A. Y.; Ganesh, A.; Sastry, S. S.; Ma, Y., Robust face recognition via sparse representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 2, 210-227 (2009)
[14] S. Yan, H. Wang, Semi-supervised learning by sparse representation, in: SIAM International Conference on Data Mining (SDM), 2009.; S. Yan, H. Wang, Semi-supervised learning by sparse representation, in: SIAM International Conference on Data Mining (SDM), 2009.
[15] X.F. He, D. Cai, S.C. Yan, H.J. Zhang, Neighborhood preserving embedding, in: IEEE International Conference on Computer Vision (ICCV), 2005.; X.F. He, D. Cai, S.C. Yan, H.J. Zhang, Neighborhood preserving embedding, in: IEEE International Conference on Computer Vision (ICCV), 2005.
[16] H. Wang, S.C. Yan, D. Xu, X.O. Tang, T. Huang, Trace ratio vs. ratio trace for dimensionality reduction, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007.; H. Wang, S.C. Yan, D. Xu, X.O. Tang, T. Huang, Trace ratio vs. ratio trace for dimensionality reduction, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007.
[17] Guo, Y.; Li, S.; Yang, J.; Shu, T.; Wu, L., A generalized Foley-Sammon transform based on generalized fisher discrimimant criterion and its application to face recognition, Pattern Recognition Letters, 24, 1-3, 147-158 (2003) · Zbl 1055.68092
[18] Jia, Y.; Nie, F.; Zhang, C., Trace ratio problem revisited, IEEE Transactions on Neural Networks, 20, 4, 729-735 (2009)
[19] Hesterberg, T.; Choi, N. H.; Meier, L.; Fraley, C., Least angle and l1 penalized regression: a review, Statistics Surveys, 2, 61-93 (2008) · Zbl 1189.62070
[20] Liu, J.; Ye, J.; Jin, R., Sparse learning with euclidean projection onto l1 ball, Journal of Machine Learning Research (2009)
[21] J. Liu, S. Ji, J. Ye, SLEP: sparse learning with efficient projections, 2009.; J. Liu, S. Ji, J. Ye, SLEP: sparse learning with efficient projections, 2009.
[22] Nie, F.; Xiang, S.; Jia, Y.; Zhang, C., Semi-supervised orthogonal discriminant analysis via label propagation, Pattern Recognition, 42, 2615-2627 (2009) · Zbl 1175.68338
[23] Tseng, P., Convergence of a block coordinate descent method for nondifferentiable minimization, Journal of Optimization Theory and Applications, 109, 3, 475-494 (2001) · Zbl 1006.65062
[24] Qiao, L. S.; Chen, S. C.; Tan, X. Y., Sparsity preserving discriminant analysis for single training image face recognition, Pattern Recognition Letters, 31, 5, 422-429 (2010)
[25] Martinez, A. M.; Kak, A. C., PCA versus LDA, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 2, 228-233 (2001)
[26] Lee, K. C.; Ho, J.; Kriegman, D. J., Acquiring linear subspaces for face recognition under variable lighting, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 5, 684-698 (2005)
[27] Cai, D.; He, X. F.; Han, J. W.; Zhang, H. J., Orthogonal laplacianfaces for face recognition, IEEE Transactions on Image Processing, 15, 11, 3608-3614 (2006)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.