×

Low-rank nonnegative matrix factorization on Stiefel manifold. (English) Zbl 1462.62773

Summary: Low rank is an important but ill-posed problem in the development of nonnegative matrix factorization (NMF) algorithms because the essential information is often encoded in a low-rank intrinsic data matrix, whereas noise and outliers are contained in a residue matrix. Most existing NMF approaches achieve low rank by directly specifying the dimensions of the factor matrices. A few others impose the low rank constraint on the factor matrix and use the alternating direction method of multipliers to solve the optimization problem. In contrast to previous approaches, this paper proposes a novel method for low-rank nonnegative matrix factorization on a Stiefel manifold (LNMFS), which utilizes the low rank structure of intrinsic data and transforms it into a Frobenius norm of the latent factors. To obtain orthogonal factors as distinct patterns, we further impose orthogonality constraints by assuming that the basis matrix lies on a Stiefel manifold. In addition, to improve the robustness of the data in a manifold structure, we incorporate the graph smoothness of the coefficient matrix. Finally, we develop an efficient alternative iterative algorithm to solve the optimization problem and provide proof of its convergence. Extensive experiments on real-world datasets demonstrate the superiority of the proposed method compared with other representative NMF-based algorithms.

MSC:

62R30 Statistics on manifolds
62H12 Estimation in multivariate analysis
15A23 Factorization of matrices

Software:

BrainNet Viewer
Full Text: DOI

References:

[1] Absil, P. A.; Mahony, R.; Sepulchre, R., Optimization Algorithms on Matrix Manifolds (2009), Princeton University Press · Zbl 1147.65043
[2] Bagga, A.; Baldwin, B., Entity-based cross-document coreferencing using the vector space model, Meeting of the Association for Computational Linguistics and International Conference on Computational Linguistics, 79-85 (1998)
[3] Berry, M. W.; Browne, M.; Langville, A. N.; Pauca, V. P.; Plemmons, R. J., Algorithms and applications for approximate nonnegative matrix factorization, Comput. Stat. Data Anal., 52, 1, 155-173 (2007) · Zbl 1452.90298
[4] Brouwer, T.; Frellsen, J.; Lio, P., Comparative study of inference methods for bayesian nonnegative matrix factorisation, Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18-22, 2017, Proceedings, Part I, Skopje, Macedonia, 513-529 (2017)
[5] Busovaca, E.; Zimmerman, M. E.; Meier, I. B.; Griffith, E. Y.; Grieve, S. M.; Korgaonkar, M. S.; Williams, L. M.; Brickman, A. M., Is the alzheimer’s disease cortical thickness signature a biological marker for memory?, Brain Imaging Behav., 10, 2, 1-7 (2016)
[6] Cabral, R.; Torre, F. D.L.; Costeira, J.a. P.; Bernardino, A., Unifying nuclear norm and bilinear factorization approaches for low-rank matrix decomposition, Proceedings of the 2013 IEEE International Conference on Computer Vision. Proceedings of the 2013 IEEE International Conference on Computer Vision, ICCV ’13, 2488-2495 (2013), IEEE Computer Society: IEEE Computer Society Washington, DC, USA
[7] Cai, D.; He, X.; Han, J.; Huang, T. S., Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., 33, 8, 1548-1560 (2011)
[8] Choi, S., Algorithms for orthogonal nonnegative matrix factorization, IEEE International Joint Conference on Neural Networks, 1828-1832 (2008)
[9] Comon, P.; Jutten, C., Handbook of Blind Source Separation: independent Component Analysis and Separation (2010), Academic Press
[10] URL http://jmlr.org/papers/v16/cunningham15a.html. · Zbl 1351.62123
[11] Dahnke, R.; Yotter, R. A.; Gaser, C., Cortical thickness and central surface estimation, Neuroimage, 65, 1, 336-348 (2013)
[12] Ding, C.; Li, T.; Peng, W.; Park, H., Orthogonal nonnegative matrix t-factorizations for clustering, Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’06, 126-135 (2006)
[13] Edelman, A.; Arias, T. A.; Smith, S. T., The geometry of algorithms with orthogonality constraints (1998), Soc. Ind. Appl. Math. · Zbl 0928.65050
[14] Eggert, J.; Korner, E., Sparse coding and nmf, IEEE International Joint Conference on Neural Networks, 2004. Proceedings, 2529-2533 vol.4 (2004)
[15] Févotte, C.; Dobigeon, N., Nonlinear hyperspectral unmixing with robust nonnegative matrix factorization, IEEE Trans. Image Process., 24, 12, 4810-4819 (2015) · Zbl 1408.94183
[16] Févotte, C.; Idier, J., Algorithms for nonnegative matrix factorization with the β-divergence, Neural Comput., 23, 9, 2421-2456 (2011) · Zbl 1231.65072
[17] Hoyer, P. O., Non-negative matrix factorization with sparseness constraints, J. Mach. Learn. Res., 5, 1, 1457-1469 (2004) · Zbl 1222.68218
[18] S0020025516306193
[19] Jolliffe, I. T.; Cadima, J., Principal component analysis: a review and recent developments, Philos. Trans. A Math. Phys. Eng. Sci., 374, 2065, 20150202 (2016) · Zbl 1353.62067
[20] Kong, D.; Huang, H.; Huang, H., Robust nonnegative matrix factorization using l21-norm, ACM International Conference on Information and Knowledge Management, 673-682 (2011)
[21] Lathauwer, L. D.; Moor, B. D.; Vandewalle, J., A Multilinear Singular Value Decomposition (2000), Soc. Ind. Appl. Math. · Zbl 0962.15005
[22] Lee, D. D.; Seung, H. S., Learning the parts of objects by non-negative matrix factorization, Nature, 401, 6755, 788 (1999) · Zbl 1369.68285
[23] Li, T.; Ding, C., The relationships among various nonnegative matrix factorization methods for clustering, International Conference on Data Mining, 362-371 (2007)
[24] Li, X.; Cui, G.; Dong, Y., Graph regularized non-negative low-rank matrix factorization for image clustering., IEEE Trans. Cybern., 47, 11, 3840-3853 (2017)
[25] Li, Z.; Wu, X.; Peng, H., Nonnegative matrix factorization on orthogonal subspace, Pattern Recognit. Lett., 31, 9, 905-911 (2010)
[26] Macovski, A., Noise of mri, Magnet. Reson. Med., 36, 3, 494-497 (1996)
[27] Manning, C. D.; Raghavan, P.; Schütze, H., Introduction to Information Retrieval (2008), Cambridge University Press · Zbl 1160.68008
[28] Pan, J.; Ng, M. K., Orthogonal nonnegative matrix factorization by sparsity and nuclear norm optimization, SIAM J. Matrix Anal. Appl., 39, 2, 856-875 (2018) · Zbl 1391.65111
[29] S2352872918300150
[30] Rijsbergen, C. J.V., Foundation of evaluation, J. Document., 30, 4, 365-373 (1974)
[31] Schmidt, M. N.; Larsen, J.; Hsiao, F. T., Wind noise reduction using non-negative sparse coding, Machine Learning for Signal Processing, 2007 IEEE Workshop on, 431-436 (2007)
[32] SL, D.; GW, V. H.; MD, C.; A., P., Parcellation of human temporal polar cortex: a combined analysis of multiple cytoarchitectonic, chemoarchitectonic, and pathological markers, J. Compar. Neurol., 514, 6, 595-623 (2010)
[33] Srebro, N.; Shraibman, A., Rank, trace-norm and max-norm, Conference On Learning Theory, 545-560 (2005) · Zbl 1137.68563
[34] Strehl, A.; Strehl, E.; Ghosh, J.; Mooney, R., Impact of similarity measures on web-page clustering, In Workshop on Artificial Intelligence for Web Search (AAAI 2000, 58-64 (2000), AAAI
[35] Sun, D. L.; Mazumder, R., Non-negative matrix completion for bandwidth extension: a convex optimization approach, IEEE International Workshop on Machine Learning for Signal Processing, 1-10 (2013)
[36] Tzourio-Mazoyer, N.; Landeau, B.; Papathanassiou, D.; Crivello, F.; Etard, O.; Delcroix, N.; Mazoyer, B.; Joliot, M., Automated anatomical labeling of activations in spm using a macroscopic anatomical parcellation of the mni mri single-subject brain, Neuroimage, 15, 1, 273-289 (2002)
[37] Vandereycken, B., Low-rank matrix completion by riemannian optimization, SIAM J. Optim., 23, 2, 1214-1236 (2013) · Zbl 1277.15021
[38] Wang, K.; Liang, M.; Wang, L.; Tian, L.; Zhang, X.; Li, K.; Jiang, T., Altered functional connectivity in early alzheimer’s disease: a resting-state fmri study, Hum. Brain Mapp., 28, 10, 967 (2007)
[39] Wang, Y. X.; Zhang, Y. J., Nonnegative matrix factorization: a comprehensive review, IEEE Trans. Knowl. Data Eng., 25, 6, 1336-1353 (2013)
[40] Weller, J.; Budson, A., Current understanding of alzheimer’s disease diagnosis and treatment, F1000research, 7, 1161 (2018)
[41] Wright, J.; Ganesh, A.; Rao, S.; Peng, Y. G.; Ma, Y., Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization., International Conference on Neural Information Processing Systems, 2080-2088 (2009)
[42] Wu, W.; Kwong, S.; Yu, Z.; Jia, Y.; Wei, G., Nonnegative matrix factorization with mixed hypergraph regularization for community detection, Inf. Sci., 435, 263-281 (2018) · Zbl 1440.68246
[43] Xia, M.; Wang, J.; Yong, H., Brainnet viewer: a network visualization tool for human brain connectomics, Plos One, 8, 7, e68910 (2013)
[44] Yang, J.; Yang, S.; Fu, Y.; Li, X.; Huang, T., Non-negative graph embedding, Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 1-8 (2008)
[45] Zhang, K.; Zhang, S.; Liu, J.; Wang, J.; Zhang, J., Greedy orthogonal pivoting algorithm for non-negative matrix factorization, Proceedings of the 36th International Conference on Machine Learning, 7493-7501 (2019)
[46] Zhang, L.; Chen, Z.; Zheng, M.; Xiaofei, H. E., Robust non-negative matrix factorization, Front. Electr. Electron. Eng. China, 6, 2, 192-200 (2011)
[47] Zhang, T.; Fang, B.; Tang, Y. Y.; He, G.; Wen, J., Topology preserving non-negative matrix factorization for face recognition, IEEE Trans. Image Process. A Public. IEEE Signal Process. Soc., 17, 4, 574-584 (2008)
[48] Zhang, W. E.; Tan, M.; Sheng, Q.; Yao, L.; Shi, Q., Efficient orthogonal non-negative matrix factorization over stiefel manifold, Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 1743-1752 (2016)
[49] Zhang, X.; Gao, H.; Li, G.; Zhao, J.; Li, Z., Multi-view clustering based on graph-regularized nonnegative matrix factorization for object recognition, Inf. Sci., 432, 463-478 (2018) · Zbl 1436.68323
[50] Zhao, R.; Tan, V. Y.F., Online nonnegative matrix factorization with outliers, Proceeding of IEEE International Conference on Acoustics, Speech and Signal Processing, 2662-2666 (2016)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.