Explicit connections between longitudinal data analysis and kernel machines. (English) Zbl 1326.62140
Summary: Two areas of research – longitudinal data analysis and kernel machines – have large, but mostly distinct, literatures. This article shows explicitly that both fields have much in common with each other. In particular, many popular longitudinal data fitting procedures are special types of kernel machines. These connections have the potential to provide fruitful cross-fertilization between longitudinal data analytic and kernel machine methodology.
MSC:
| 62H30 | Classification and discrimination; cluster analysis (statistical aspects) |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 62J12 | Generalized linear models (logistic models) |
Keywords:
best linear unbiased prediction; classification; generalized linear mixed models; machine learning; linear mixed models; reproducing kernel Hilbert spaces; penalized likelihood; support vector machinesReferences:
| [1] | Aronszajn, N. (1950). Theory of reproducing kernels., Transactions of the American Mathematical Society , 68 , 337-404. · Zbl 0037.20701 · doi:10.2307/1990404 |
| [2] | Bachrach, L.K., Hastie, T., Wang, M.-C., Narasimhan, B. and Marcus, R. (1999). Bone mineral acquisition in healthy Asian, Hispanic, Black and Caucasian youth. A longitudinal study., Journal of Clinical Endocrinology and Metabolism , 84 , 4702-12. |
| [3] | Breslow, N.E. and Clayton, D.G. (1993). Approximate inference in generalized linear mixed models., Journal of the American Statistical Association , 88 , 9-25. · Zbl 0775.62195 · doi:10.2307/2290687 |
| [4] | Cristianini, N. and Shawe-Taylor, J. (2000)., An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods . Cambridge: Cambridge University Press. · Zbl 0994.68074 |
| [5] | Cressie, N. (1993)., Statistics for Spatial Data. New York: John Wiley & Sons. · Zbl 0799.62002 |
| [6] | Diggle, P.J., Heagerty, P., Liang, K.-L. and Zeger, S. (2002)., Analysis of Longitudinal Data (Second Edition) . Oxford: Oxford University Press. · Zbl 1031.62002 |
| [7] | Evgeniou, T., Pontil, M. and Poggio, T. (2000). Regularization networks and support vector machines., Advances in Computational Mathematics , 13 , 1-50. · Zbl 0939.68098 · doi:10.1023/A:1018946025316 |
| [8] | Fitzmaurice, G., Davidian, M., Verbeke, G. and Molenberghs, G. (Eds.) (2008)., Longitudinal Data Analysis: A Handbook of Modern Statistical Methods. Boca Raton, Florida: Chapman & Hall/CRC. |
| [9] | Fitzmaurice, G.M., Laird, N.M. and Ware, J.H. (2004)., Applied Longitudinal Analysis . New York: John Wiley & Sons. · Zbl 1057.62052 |
| [10] | Gianola, D., Fernando, R.L. and Stella, A. (2006). Genomic-assisted prediction of genetic value with semiparametric procedures., Genetics , 173 , 1761-1776. |
| [11] | Green, P.J. and Silverman, B.W. (1994)., Nonparametric Regression and Generalized Linear Models. London: Chapman & Hall. · Zbl 0832.62032 |
| [12] | Harville, D.A. (1977). Maximum likelihood approaches to variance component estimation and to related problems., Journal of the American Statistical Association , 72 , 320-338. · Zbl 0373.62040 · doi:10.2307/2286796 |
| [13] | Hastie, T.J. and Tibshirani, R.J. (1990)., Generalized Additive Models . London: Chapman & Hall. · Zbl 0747.62061 |
| [14] | Huber P. (1981)., Robust Statistics. Chichester: John Wiley & Sons. · Zbl 0536.62025 |
| [15] | James, G.M. and Hastie, T.J. (2001). Functional linear discriminant analysis for irregularly sampled curves., Journal of the Royal Statistical Society, Series B , 63 , 533-550. · Zbl 0989.62036 · doi:10.1111/1467-9868.00297 |
| [16] | Kimeldorf, G. and Wahba, G. (1971). Some results on Tchebycheffian spline functions., Journal of Mathematical Analysis and Its Application , 33 , 82-95. · Zbl 0201.39702 · doi:10.1016/0022-247X(71)90184-3 |
| [17] | Laird, N.M. and Ware, J.H. (1982). Random-effects models for longitudinal data., Biometrics , 38 , 963-974. · Zbl 0512.62107 · doi:10.2307/2529876 |
| [18] | Liu, D., Lin, X. and Ghosh, D. (2007). Semiparametric regression of multidimensional genetic pathway data: least-squares kernel machines and linear mixed models., Biometrics , 63 , 1079-1088. · Zbl 1274.62825 · doi:10.1111/j.1541-0420.2007.00799.x |
| [19] | McCulloch, C.E., Searle, S.R. and Neuhaus, J.M. (2008)., Generalized, Linear, and Mixed Models, Second Edition . New York: John Wiley & Sons. · Zbl 1165.62050 |
| [20] | Moguerza, J.M. and Muñoz, A. (2006). Support vector machines with applications (with discussion)., Statistical Science , 21 , 322-362. · Zbl 1246.68185 · doi:10.1214/088342306000000493 |
| [21] | Müller, H.-G. (2005). Functional modelling and classification of longitudinal data., Scandinavian Journal of Statistics , 32 , 223-240. · Zbl 1089.62072 · doi:10.1111/j.1467-9469.2005.00429.x |
| [22] | Ormerod, J.T. and Wand, M.P. (2006). LowRankQP 1.0. R package., http://cran.r-project.org |
| [23] | Ormerod, J.T., Wand, M.P. and Koch, I. (2008). Penalised spline support vector classifiers: computational issues., Computational Statistics , 23 , 623-641. · Zbl 1223.62105 · doi:10.1007/s00180-007-0102-8 |
| [24] | Pearce, N.D. and Wand, M.P. (2006). Penalized splines and reproducing kernel methods., The American Statistician , 60 , 233-240. · doi:10.1198/000313006X124541 |
| [25] | Rasmussen, C.E. and Williams, K.I. (2006)., Gaussian Processes for Machine Learning , The MIT Press. · Zbl 1177.68165 |
| [26] | Robinson, G.K. (1991). That BLUP is a good thing: the estimation of random effects., Statistical Science , 6 , 15-51. · Zbl 0955.62500 · doi:10.1214/ss/1177011926 |
| [27] | Rudin, W. (1991)., Functional Analysis, Second Edition . New York: McGraw Hill. · Zbl 0867.46001 |
| [28] | Ruppert, D., Wand, M. P. and Carroll, R.J. (2003)., Semiparametric Regression . New York: Cambridge University Press. · Zbl 1038.62042 |
| [29] | Schölkopf, B. and Smola, A.J. (2002)., Learning with Kernels. Cambridge USA: MIT Press. · Zbl 1019.68094 |
| [30] | Shen, X., Tseng, G.C., Zhang, X. and Wong, W. (2003). On, \psi -learning. Journal of the American Statistical Association , 98 , 724-734. · Zbl 1052.62095 · doi:10.1198/016214503000000639 |
| [31] | Simmons, G.F. (1983)., Introduction to Topology and Modern Analysis . Melbourne USA: Krieger. · Zbl 0495.46001 |
| [32] | Stein, M.L. (1999)., Interpolation of Spatial Data: Some Theory for Kriging . New York: Springer. · Zbl 0924.62100 |
| [33] | Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B. and Vandewall, J. (2002), Least squares support vector machines. Singapore: World Scientific Publishing Company. · Zbl 1017.93004 |
| [34] | Tarantola, A. (2005)., Inverse Problem Theory . Philadelphia: Society of Industrial and Applied Mathematics. · Zbl 1074.65013 |
| [35] | Verbeke, G. and Molenberghs, G. (2000)., Linear Mixed Models for Longitudinal Data. New York: Springer-Verlag. · Zbl 0956.62055 |
| [36] | Wahba, G. (1990)., Spline Models for Observational Data. Philadelphia: Society of Industrial and Applied Mathematics. · Zbl 0813.62001 |
| [37] | Wahba, G. (1999). Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV. In, Advances in Kernel Methods : Support Vector Learning (eds. B. Scholkopf, C. Burges and A. Smola) pp. 69-88. Cambridge USA: MIT Press. |
| [38] | Wand, M.P. and Ormerod, J.T. (2008). On O’Sullivan penalised splines and semiparametric regression., Australian and New Zealand Journal of Statistics , 50 , 179-198. · Zbl 1146.62030 |
| [39] | Wang, J. and Shen, X. (2007). Large margin semi-supervised learning., Journal of Machine Learning Research , 8 , 1867-1891. · Zbl 1222.68329 |
| [40] | Zhu, J. and Hastie, T. (2005). Kernel logistic regression and the import vector machine., Journal of Computational and Graphical Statistics , 14 , 185-205. · doi:10.1198/106186005X25619 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
