×

A latent variable model for two-dimensional canonical correlation analysis and the variational inference. (English) Zbl 1490.62136

Summary: The probabilistic dimension reduction has been and is a major concern. Probabilistic models provide a better interpretability of the dimension reduction methods and present a framework for their further extensions. In pattern recognition problems, data that have a matrix or tensor structure is initially transformed into a vector format. This eliminates the internal structure of the data. The available perspective is to maintain the internal structure of each data while reducing the dimensionality, which can reduce the small sample size problem. Canonical correlation analysis is one of the most important techniques in dimension reduction in multi-view data. A two-dimensional canonical correlation analysis as an extension of canonical correlation analysis has been proposed to preserve the matrix structure of the data. Here, a new probabilistic framework for two-dimensional canonical correlation analysis is proposed, where the matrix-variate distributions are applied to model the relation between the latent matrix and the two-view matrix-variate observed data. These distributions, specific to the matrix data, can provide better understanding of two-dimensional canonical correlation analysis and pave the way for further extensions. In general, there does not exist any analytical maximum likelihood solution for this model; therefore, here the two approaches, one based on the expectation maximization and other on variational expected maximization, are proposed for learning the model parameters. The synthetic data are applied to evaluate the convergence and quality of the mapping of these algorithms. The functionalities of these methods and their counterparts are compared on the real face datasets.

MSC:

62H20 Measures of association (correlation, canonical correlation, etc.)

References:

[1] Bach FR, Jordan MI (2005) A probabilistic interpretation of canonical correlation analysis. Technical report, Department of Statistics, University of California, Berkeley
[2] Bilenko, N.; Gallant, J., Pyrcca: rgularized kernel canonical correlation analysis in python and its applications to neuroimaging, Front Neuroinform (2015)
[3] Bishop CM (1999) Bayesian pca. In: Proceedings of the 1998 conference on advances in neural information processing systems II. MIT Press, Cambridge, pp 382-388
[4] Chen, T.; Martin, E.; Montague, G., Robust probabilistic pca with missing data and contribution analysis for outlier detection, Comput Stat Data Anal, 53, 10, 3706-3716 (2009) · Zbl 1453.62067
[5] Damianou A, Lawrence N, Ek C (2016) Multi-view learning as a nonparametric nonlinear inter-battery factor analysis. arXiv:1511.04839
[6] Georghiades, A.; Belhumeur, P.; Kriegman, D., From few to many: illumination cone models for face recognition under variable lighting and pose, IEEE Trans Pattern Anal Mach Intell, 23, 6, 643-660 (2001)
[7] Gupta, A.; Nagar, D., Matrix variate distributions. PMS series (1999), Reading: Addison-Wesley Longman, Limited, Reading
[8] Hotelling, H., Relations between two sets of variates, Biometrika, 28, 3-4, 321-377 (1936) · Zbl 0015.40705
[9] Jordan, MI; Ghahramani, Z.; Jaakkola, TS; Saul, LK, An introduction to variational methods for graphical models, Mach Learn, 37, 2, 183-233 (1999) · Zbl 0945.68164
[10] Ju, F.; Sun, Y.; Gao, J.; Hu, Y.; Yin, B., Image outlier detection and feature extraction via l1-norm-based 2d probabilistic pca, IEEE Trans Image Process, 24, 12, 4834-4846 (2015) · Zbl 1408.94294
[11] Ju F, Sun Y, Gao J, Liu S, Hu Y, Yin B (2016) Mixture of bilateral-projection two-dimensional probabilistic principal component analysis. In: The IEEE conference on computer vision and pattern recognition (CVPR)
[12] Ju, F.; Sun, Y.; Gao, J.; Hu, Y.; Yin, B., Vectorial dimension reduction for tensors based on bayesian inference, IEEE Trans Neural Netw Learn Syst, 29, 10, 4579-4592 (2018)
[13] Klami, A.; Virtanen, S.; Kaski, S., Bayesian canonical correlation analysis, J Mach Learn Res, 14, 1, 965-1003 (2013) · Zbl 1320.62134
[14] Lawrence, N., Probabilistic non-linear principal component analysis with gaussian process latent variable models, J Mach Learn Res, 6, 1783-1816 (2005) · Zbl 1222.68247
[15] Lee, SH; Choi, S., Two-dimensional canonical correlation analysis, IEEE Signal Process Lett, 14, 10, 735-738 (2007)
[16] Li S, Yi D, Lei Z, Liao S (2013) The casia nir-vis 2.0 face database. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 348-353
[17] Michaeli T, Wang W, Livescu K (2016) Nonparametric canonical correlation analysis. arXiv:1511.04839
[18] Rasiwasia N, Mahajan D, Mahadevan V, Aggarwal G (2014) Cluster canonical correlation analysis. In: Proceedings of the seventeenth international conference on artificial intelligence and statistics, vol 33, pp 823-831
[19] Safayani, M.; Ahmadi, SH; Afrabandpey, H.; Mirzaei, A., An EM based probabilistic two-dimensional cca with application to face recognition, Appl Intell, 48, 3, 755-770 (2018)
[20] Sarvestani, RR; Boostani, R., FF-SKPCCA: kernel probabilistic canonical correlation analysis, Appl Intell, 46, 438-454 (2016)
[21] Sun, N.; Ji, Z.; Cr, Z.; Zhao, L., Two-dimensional canonical correlation analysis and its application in small sample size face recognition, Neural Comput Appl, 19, 3, 377-382 (2010)
[22] Tipping, ME; Bishop, CM, Mixtures of probabilistic principal component analyzers, Neural Comput, 11, 2, 443-482 (1999)
[23] Titsias M, Lawrence ND (2010) Bayesian Gaussian process latent variable model. In: Proceedings of the 13th international conference on artificial intelligence and statistics, Sardinia, Italy, Proceedings of machine learning research, vol 9, pp 844-851
[24] Viinikanoja J, Klami A, Kaski S (2010) Variational Bayesian mixture of robust cca models. In: Proceedings of the 2010 European conference on machine learning and knowledge discovery in databases: Part III, , ECML PKDD’10. Springer, Berlin, pp 370-385
[25] Wang, C., Variational Bayesian approach to canonical correlation analysis, IEEE Trans Neural Netw, 18, 3, 905-910 (2007)
[26] Williams VV (2012) Multiplying matrices faster than coppersmith-winograd. In: Proceedings of the forty-fourth annual ACM symposium on theory of computing, ACM, New York, NY, USA, STOC ��12, pp 887-898 · Zbl 1286.65056
[27] Zhao, J., Efficient model selection for mixtures of probabilistic pca via hierarchical BIC, IEEE Trans Cybern, 44, 10, 1871-1883 (2014)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.