Rank-optimized logistic matrix regression toward improved matrix data classification. (English) Zbl 1471.62429
Summary: While existing logistic regression suffers from overfitting and often fails in considering structural information, we propose a novel matrix-based logistic regression to overcome the weakness. In the proposed method, 2D matrices are directly used to learn two groups of parameter vectors along each dimension without vectorization, which allows the proposed method to fully exploit the underlying structural information embedded inside the 2D matrices. Further, we add a joint \(\ell_{2,1}\)-norm on two parameter matrices, which are organized by aligning each group of parameter vectors in columns. This added co-regularization term has two roles – enhancing the effect of regularization and optimizing the rank during the learning process. With our proposed fast iterative solution, we carried out extensive experiments. The results show that in comparison to both the traditional tensor-based methods and the vector-based regression methods, our proposed solution achieves better performance for matrix data classifications.
MSC:
| 62J12 | Generalized linear models (logistic models) |
| 62H30 | Classification and discrimination; cluster analysis (statistical aspects) |
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
References:
| [1] | Bootkrajang, J., & Kabán, A. (2014). Learning kernel logistic regression in the presence of class label noise. Pattern Recognition, 47(11), 3641-3655. , · Zbl 1373.68314 |
| [2] | Cai, X., Nie, F., Huang, H., & Ding, C. (2011). Multi-class l2, 1-norm support vector machine. In Proceedings of the 2011 IEEE 11th International Conference on Data Mining (pp. 91-100). Piscataway, NJ: IEEE. , |
| [3] | Cao, X., Wei, X., Han, Y., Yang, Y., & Lin, D. (2013). Robust tensor clustering with non-greedy maximization. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Palo Alto, CA: AAAI. |
| [4] | Cheng, G., Han, J., Guo, L., Liu, Z., Bu, S., & Ren, J. (2015). Effective and efficient midlevel visual elements-oriented land-use classification using VHR remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 53(8), 4238-4249. , |
| [5] | Duller, A., Guller, G., France, I., & Lamb, H. (1999). A pollen image database for evaluation of automated identification systems. Quaternary Newsletter, 89, 4-9. |
| [6] | Fang, L., Li, S., Duan, W., Ren, J., & Benediktsson, J. A. (2015). Classification of hyperspectral images by exploiting spectral-spatial information of superpixel via multiple kernels. IEEE Transactions on Geoscience and Remote Sensing, 53(12), 6663-6674. , |
| [7] | Genkin, A., Lewis, D. D., & Madigan, D. (2007). Large-scale Bayesian logistic regression for text categorization. Technometrics, 49(3), 291-304. , |
| [8] | Gönen, M., & Alpaydın, E. (2011). Multiple kernel learning algorithms. Journal of Machine Learning Research, 12, 2211-2268. · Zbl 1280.68167 |
| [9] | Guo, W., Kotsia, I., & Patras, I. (2012). Tensor learning for regression. IEEE Transactions on Image Processing, 21(2), 816-827. , · Zbl 1373.62308 |
| [10] | Han, J., Zhang, D., Hu, X., Guo, L., Ren, J., & Wu, F. (2015). Background prior-based salient object detection via deep reconstruction residual. IEEE Transactions on Circuits and Systems for Video Technology, 25(8), 1309-1321. , |
| [11] | Han, Y., Yang, Y., & Zhou, X. (2013). Co-regularized ensemble for feature selection. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence.Palo Alto, CA: AAAI. |
| [12] | He, X., Cai, D., & Niyogi, P. (2005). Tensor subspace analysis. In Y. Weiss, B. Schölkopf, & J. Platt (Eds.), Advances in neural information processing systems, 18 (pp. 499-506). Cambridge, MA: MIT Press. |
| [13] | Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67. , · Zbl 0202.17205 |
| [14] | Hou, C., Nie, F., Yi, D., & Wu, Y. (2013). Efficient image classification via multiple rank regression. IEEE Transactions on Image Processing, 22(1), 340-352. , · Zbl 1373.94166 |
| [15] | Hou, C., Nie, F., Zhang, C., Yi, D., & Wu, Y. (2014). Multiple rank multi-linear SVM for matrix data classification. Pattern Recognition, 47(1), 454-469. , · Zbl 1326.68245 |
| [16] | Huang, D., Cabral, R., & De la Torre, F. (2016). Robust regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 363-375. , |
| [17] | Hung, H., & Wang, C.-C. (2013). Matrix variate logistic regression model with application to EEG data. Biostatistics, 14(1), 189-202. , |
| [18] | Kauppi, J.-P., Hahne, J., Müller, K.-R., & Hyvärinen, A. (2015). Three-way analysis of spectrospatial electromyography data: Classification and interpretation. PloS One, 10(6), e0127231. , |
| [19] | Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review, 51(3), 455-500. , · Zbl 1173.65029 |
| [20] | Komarek, P. (2004). Logistic regression for data mining and high-dimensional classification (CMU-RI-TR-04-3Y). Pittsburgh, PA: Robotics Institute, Carnegie Mellon University. |
| [21] | Kotsia, I., & Patras, I. (2011). Support Tucker machines. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (pp. 633-640). Piscataway, NJ: IEEE. , |
| [22] | Ma, Z., Yang, Y., Nie, F., & Sebe, N. (2013). Thinking of images as what they are: Compound matrix regression for image classification. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Palo Alto, CA: AAAI. |
| [23] | Naseem, I., Togneri, R., & Bennamoun, M. (2010). Linear regression for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 2106-2112. , |
| [24] | Pang, Y., Li, X., & Yuan, Y. (2010). Robust tensor analysis with l1-norm. IEEE Transactions on Circuits and Systems for Video Technology, 20(2), 172-178. , |
| [25] | Qiao, T., Ren, J., Wang, Z., Zabalza, J., Sun, M., Zhao, H., …, Marshall, S. (2017). Effective denoising and classification of hyperspectral images using curvelet transform and singular spectrum analysis. IEEE Transactions on Geoscience and Remote Sensing, 55(1), 119-133. , |
| [26] | Ren, J. (2012). ANN vs. SVM: Which one performs better in classification of MCCS in mammogram imaging? Knowledge-Based Systems, 26, 144-153. , |
| [27] | Saberian, M. J., Masnadi-Shirazi, H., & Vasconcelos, N. (2011). Taylorboost: First and second-order boosting algorithms with explicit margin control. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2929-2934). Piscataway, NJ: IEEE. , |
| [28] | Shi, J. V., Xu, Y., & Baraniuk, R. G. (2014). Sparse bilinear logistic regression. arXiv:1404.4104. |
| [29] | Tan, X., Li, Y., Liu, J., & Jiang, L. (2010). Face liveness detection from a single image with sparse low rank bilinear discriminative model. In Proceedings of the European Conference on Computer Vision (pp. 504-517). Berlin: Springer. , |
| [30] | Tan, X., Zhang, Y., Tang, S., Shao, J., Wu, F., & Zhuang, Y. (2012). Logistic tensor regression for classification. Proceedings of the International Conference on Intelligent Science and Intelligent Data Engineering (pp. 573-581). Berlin: Springer. |
| [31] | Tao, D., Li, X., Hu, W., Maybank, S., & Wu, X. (2005). Supervised tensor learning. In Proceedings of the Fifth IEEE International Conference on Data Mining (pp. 8-16). Piscataway, NJ: IEEE. |
| [32] | Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 267-288. , · Zbl 0850.62538 |
| [33] | Wang, C., He, X., Bu, J., Chen, Z., Chen, C., & Guan, Z. (2011). Image representation using Laplacian regularized nonnegative tensor factorization. Pattern Recognition, 44(10), 2516-2526. , · Zbl 1218.68134 |
| [34] | Yang, J., Zhang, D., Frangi, A. F., & Yang, J.-Y. (2004). Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE transactions on Pattern Analysis and Machine Intelligence, 26(1), 131-137. , |
| [35] | Zabalza, J., Ren, J., Zheng, J., Han, J., Zhao, H., Li, S., & Marshall, S. (2015). Novel two-dimensional singular spectrum analysis for effective feature extraction and data classification in hyperspectral imaging. IEEE Transactions on Geoscience and Remote Sensing, 53(8), 4418-4433. , |
| [36] | Zhang, L., Yang, M., & Feng, X. (2011). Sparse representation or collaborative representation: Which helps face recognition? In Proceedings of the 2011 International Conference on Computer Vision (pp. 471-478). Piscataway, NJ: IEEE. , |
| [37] | Zhang, Y., Ren, J., & Jiang, J. (2015). Combining MLC and SVM classifiers for learning based decision making: Analysis and evaluations. Computational Intelligence and Neuroscience, 2015, art. 44. , |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
