×
Huang, Su-Yun; Lee, Mei-Hsien; Hsiao, Chuhsing Kate

Nonlinear measures of association with kernel canonical correlation analysis and applications. (English) Zbl 1160.62059

J. Stat. Plann. Inference 139, No. 7, 2162-2174 (2009).
Summary: Measures of association between two sets of random variables have long been of interest to statisticians. The classical canonical correlation analysis (LCCA) can characterize, but also is limited to, linear association. This article introduces a nonlinear and nonparametric kernel method for association studies and proposes a new independence test for two sets of variables. This nonlinear kernel canonical correlation analysis (KCCA) can also be applied to nonlinear discriminant analysis. Implementation issues are discussed. We place the implementation of KCCA in the framework of classical LCCA via a sequence of independent systems in the kernel associated Hilbert spaces. Such a placement provides an easy way to carry out the KCCA. Numerical experiments and comparison with other nonparametric methods are presented.

Cited in 3 Documents

MSC:

62H20 Measures of association (correlation, canonical correlation, etc.)
62G10 Nonparametric hypothesis testing
62H30 Classification and discrimination; cluster analysis (statistical aspects)
46N30 Applications of functional analysis in probability theory and statistics
62G99 Nonparametric inference

Keywords:

association measure; canonical correlation analysis; dimension reduction; kernel method; multivariate analysis; reproducing kernel; reproducing kernel Hilbert space; test of independence

Software:

fda (R); RSVM; ElemStatLearn
Cite Review PDF
Full Text: DOI Link

References:

[1] Akaho, S., 2001. A kernel method for canonical correlation analysis. In: International Meeting of Psychometric Society (IMPS2001).; Akaho, S., 2001. A kernel method for canonical correlation analysis. In: International Meeting of Psychometric Society (IMPS2001). · Zbl 1001.68659
[2] Anderson, T. W., An Introduction to Multivariate Statistical Analysis (2003), Wiley: Wiley New York · Zbl 0083.14601
[3] Asuncion, A., Newman, D.J., 2007. UCI repository of smachine learning databases http://www.ics.uci.edu/∼;mlearn/MLRepository.html; Asuncion, A., Newman, D.J., 2007. UCI repository of smachine learning databases http://www.ics.uci.edu/∼;mlearn/MLRepository.html
[4] Bach, F. R.; Jordan, M. I., Kernel independent component analysis, J. Mach. Learning Res., 3, 1-48 (2002) · Zbl 1088.68689
[5] Bartlett, M. S., Multivariate analysis, J. Roy. Statist. Soc., 9, Suppl., 176-197 (1947) · Zbl 0030.40403
[6] Bartlett, M. S., The general canonical correlation distribution, Ann. Math. Statist., 18, 1-17 (1947) · Zbl 0032.29503
[7] Berlinet, A.; Thomas-Agnan, C., Reproducing Kernel Hilbert Spaces in Probability and Statistics (2004), Kluwer: Kluwer Boston · Zbl 1145.62002
[8] Cortes, C.; Vapnik, V., Support vector networks, Mach. Learning, 20, 273-279 (1995) · Zbl 0831.68098
[9] Cristianini, N.; Shawe-Taylor, J., An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods (2000), Cambridge University Press: Cambridge University Press Cambridge, UK
[10] Dauxois, J.; Nkiet, G. M., Canonical analysis of two Euclidean subspaces and its applications, Linear Algebra Appl., 264, 355-388 (1997) · Zbl 0895.15002
[11] Dauxois, J.; Nkiet, G. M., Nonlinear canonical analysis and independence tests, Ann. Statist., 26, 1254-1278 (1998) · Zbl 0934.62061
[12] Dauxois, J.; Nkiet, G. M., Measure of association for Hilbert subspaces and some applications, J. Multivariate Anal., 82, 263-298 (2002) · Zbl 1098.62538
[13] Dauxois, J.; Romain, Y.; Viguier, S., Comparison of two factor subspaces, J. Multivariate Anal., 44, 160-178 (1993) · Zbl 0766.62035
[14] Dauxois, J.; Nkiet, G. M.; Romain, Y., Canonical analysis relative to a closed subspace, Linear Algebra Appl., 388, 119-145 (2004) · Zbl 1103.62051
[15] Eubank, R., Hsing, T., 2006. Canonical correlation for stochastic processes, preprint http://www.stat.osu.edu/hsing/papers/CCpaper-rev1.pdf; Eubank, R., Hsing, T., 2006. Canonical correlation for stochastic processes, preprint http://www.stat.osu.edu/hsing/papers/CCpaper-rev1.pdf · Zbl 1145.62048
[16] Gretton, A., Herbrich, R., Smola, A., 2003. The kernel mutual information. Technical Report, MPI for Biological Cybernetics, Tuebingen, Germany.; Gretton, A., Herbrich, R., Smola, A., 2003. The kernel mutual information. Technical Report, MPI for Biological Cybernetics, Tuebingen, Germany. · Zbl 1222.68208
[17] Gretton, A.; Herbrich, R.; Smola, A.; Bousquet, O.; Schölkopf, B., Kernel methods for measuring independence, J. Mach. Learning Res., 6, 2075-2129 (2005) · Zbl 1222.68208
[18] Hardoon, D. R.; Szedmak, S.; Shawe-Taylor, J., Canonical correlation analysis: an overview with application to learning methods, Neural Comput., 16, 2639-2664 (2004) · Zbl 1062.68134
[19] Hastie, T.; Tibshirani, R.; Friedman, J., The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2001), Springer: Springer New York · Zbl 0973.62007
[20] Hotelling, H., Relations between two sets of variates, Biometrika, 28, 321-377 (1936) · Zbl 0015.40705
[21] Hsing, T.; Liu, L. Y.; Brun, M.; Dougherty, E. R., The coefficient of intrinsic dependence, Pattern Recognition, 38, 623-636 (2005)
[22] Huang, C. M.; Lee, Y. J.; Lin, D. K.J.; Huang, S. Y., Model selection for support vector machines via uniform design. A special issue on machine learning and robust data mining, Comput. Statist. Data Anal., 52, 335-346 (2007) · Zbl 1452.62073
[23] Huang, S.Y., Hwand, C.R., 2006. Kernel Fisher discriminant analysis in Gaussian reproducing kernel Hilbert spaces—theory. Technical Report, Institute of Statistical Science, Academia Sinica http://www.stat.sinica.edu.tw/syhuang/; Huang, S.Y., Hwand, C.R., 2006. Kernel Fisher discriminant analysis in Gaussian reproducing kernel Hilbert spaces—theory. Technical Report, Institute of Statistical Science, Academia Sinica http://www.stat.sinica.edu.tw/syhuang/
[24] Jensen, D. R.; Mayer, L. S., Some variational results and their applications in multiple inference, Ann. Statist., 5, 922-931 (1977) · Zbl 0368.62007
[25] Kuss, M., Graepel, T., 2003. The geometry of kernel canonical correlation analysis. Technical Report, Max Planck Institute for Biological Cybernetics, Germany.; Kuss, M., Graepel, T., 2003. The geometry of kernel canonical correlation analysis. Technical Report, Max Planck Institute for Biological Cybernetics, Germany.
[26] Lee, Y. J.; Huang, S. Y., Reduced support vector machines: a statistical theory, IEEE Trans. Neural Networks, 18, 1-13 (2007)
[27] Lee, Y.J., Mangasarian, O.L., 2001. RSVM: reduced support vector machines. In: Proceeding of the First International Conference on Data Mining, SIAM.; Lee, Y.J., Mangasarian, O.L., 2001. RSVM: reduced support vector machines. In: Proceeding of the First International Conference on Data Mining, SIAM.
[28] Ramsay, J. O.; Silverman, B. W., Functional Data Analysis (2005), Springer: Springer New York · Zbl 0882.62002
[29] Schölkopf, B.; Smola, A., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2002), MIT Press: MIT Press Cambridge, MA
[30] Smola, A., Schölkopf, B., 2000. Sparse greedy matrix approximation for machine learning. In: Proceedings of the 17th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, CA, pp. 911-918.; Smola, A., Schölkopf, B., 2000. Sparse greedy matrix approximation for machine learning. In: Proceedings of the 17th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, CA, pp. 911-918.
[31] Snelson, E.; Ghahramani, Z., Sparse Gaussian processes using pseudo-inputs, (Weiss, Y.; Schölkopf, B.; Platt, J., Advances in Neural Information Processing Systems, vol. 18 (2005), MIT Press: MIT Press Cambridge, MA)
[32] Vapnik, V. N., Statistical Learning Theory (1998), Wiley: Wiley New York · Zbl 0934.62009
[33] Wang, J.; Neskovic, P.; Cooper, L. N., Training data selection for support vector machines, (Wang, L.; Chen, K.; Ong, Y.-S., Advances in Natural Computation: Proceedings, Part I, First International Conference. Lecture Notes in Computer Science, vol. 3610 (2005), Springer: Springer Berlin), 554-564
[34] Williams, C. K.I.; Seeger, M., Using the Nyström method to speed up kernel machines, (Leen, T. K.; Dietterich, T. G.; Tresp, V., Advances in Neural Information Processing Systems, vol. 13 (2001), MIT Press: MIT Press Cambridge, MA), 682-688
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.