×

Mixtures of restricted skew-\(t\) factor analyzers with common factor loadings. (English) Zbl 1474.62209

Summary: Mixtures of common \(t\) factor analyzers (MCtFA) have been shown its effectiveness in robustifying mixtures of common factor analyzers (MCFA) when handling model-based clustering of the high-dimensional data with heavy tails. However, the MCtFA model may still suffer from a lack of robustness against observations whose distributions are highly asymmetric. This paper presents a further robust extension of the MCFA and MCtFA models, called the mixture of common restricted skew-\(t\) factor analyzers (MCrstFA), by assuming a restricted multivariate skew-\(t\) distribution for the common factors. The MCrstFA model can be used to accommodate severely non-normal (skewed and leptokurtic) random phenomena while preserving its parsimony in factor-analytic representation and performing graphical visualization in low-dimensional plots. A computationally feasible expectation conditional maximization either algorithm is developed to carry out maximum likelihood estimation. The numbers of factors and mixture components are simultaneously determined based on common likelihood penalized criteria. The usefulness of our proposed model is illustrated with simulated and real datasets, and experimental results signify its superiority over some existing competitors.

MSC:

62H25 Factor analysis and principal components; correspondence analysis
62H30 Classification and discrimination; cluster analysis (statistical aspects)
Full Text: DOI

References:

[1] Aitken AC (1926) On Bernoulli’s numerical solution of algebraic equations. Proc R Soc Edinb 46:289-305 · JFM 52.0098.05 · doi:10.1017/S0370164600022070
[2] Arellano-Valle RB, Genton MG (2005) On fundamental skew distributions. J Multivar Anal 96:93-116 · Zbl 1073.62049 · doi:10.1016/j.jmva.2004.10.002
[3] Azzalini A (2014) The skew-normal and related families. IMS monographs series. Cambridge University Press, Cambridge · Zbl 1338.62007
[4] Azzalini A, Browne RP, Genton MG, McNicholas PD (2016) On nomenclature for, and the relative merits of, two formulations of skew distributions. Stat Probab Lett 110:201-206 · Zbl 1376.60024 · doi:10.1016/j.spl.2015.12.008
[5] Baek J, McLachlan GJ (2011) Mixtures of common \[t\] t-factor analyzers for clustering high-dimensional microarray data. Bioinformatics 27:1269-1276 · doi:10.1093/bioinformatics/btr112
[6] Baek J, McLachlan GJ, Flack LK (2010) Mixtures of factor analyzers with common factor loadings: applications to the clustering and visualization of high-dimensional data. IEEE Trans Pattern Anal Mach Intell 32:1-13 · doi:10.1109/TPAMI.2010.8
[7] Barndorff-Nielsen O, Shephard N (2001) Non-Gaussian Ornstein-Uhlenbeck-based models and some of their uses in financial economics. J Roy Stat Soc Ser B 63:167-241 · Zbl 0983.60028 · doi:10.1111/1467-9868.00282
[8] Beal MJ (2003) Variational algorithms for approximate Bayesian inference. Ph.D. thesis, The University of London, London, UK
[9] Biernacki C, Celeux G, Govaert G (2000) Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans Pattern Anal Mach Intell 22:719-725 · doi:10.1109/34.865189
[10] Cabral CR, Lachos VH, Prates MO (2012) Multivariate mixture modeling using skew-normal independent distributions. Comput Stat Data Anal 56:126-142 · Zbl 1239.62058 · doi:10.1016/j.csda.2011.06.026
[11] Castro LM, Costa DR, Prates MO, Lachos VH (2015) Likelihood-based inference for Tobit confirmatory factor analysis using the multivariate Student-\[t\] t distribution. Stat Comput 25:1163-1183 · Zbl 1331.62294 · doi:10.1007/s11222-014-9502-0
[12] Chen X, Cheung ST, So S, Fan ST, Barry C, Higgins J, Lai KM, Ji J, Dudoit S, Ng IO, Van De Rijn M, Botstein D, Brown PO (2002) Gene expression patterns in human liver cancers. Mol Biol Cell 13:1929-1939 · doi:10.1091/mbc.02-02-0023
[13] Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J R Stat Soc B 9:1-38 · Zbl 0364.62022
[14] Ghahramani, Z.; Beal, M.; Solla, S. (ed.); Leen, T. (ed.); Muller, K-R (ed.), Variational inference for Bayesian mixture of factor analysers (2000), Cambridge
[15] Ghahramani Z, Hinton GE (1997) The EM algorithm for factor analyzers. Technical Report No. CRG-TR-96-1, The University of Toronto, Toronto
[16] Hartigan JA, Wong MA (1979) Algorithm AS 136: a K-means clustering algorithm. J R Stat Soc C 28:100-108 · Zbl 0447.62062
[17] Hubert LJ, Arabie P (1985) Comparing partitions. J Classif 2:193-218 · Zbl 0587.62128 · doi:10.1007/BF01908075
[18] Jordan MI, Ghahramani Z, Jaakkola TS, Saul LK (1999) An introduction to variational methods for graphical models. Mach Learn 37:183-233 · Zbl 0945.68164 · doi:10.1023/A:1007665907178
[19] Lachos VH, Morenoa EJL, Chen K, Cabralc CRB (2017) Finite mixture modeling of censored data using the multivariate Student-\[t\] t distribution. J Multivar Anal 159:151-167 · Zbl 1397.62221 · doi:10.1016/j.jmva.2017.05.005
[20] Lee SX, McLachlan GJ (2014) Finite mixtures of multivariate skew \[t\] t-distributions: some recent and new results. Stat Comp 24:181-202 · Zbl 1325.62107 · doi:10.1007/s11222-012-9362-4
[21] Lee SX, McLachlan GJ (2016) Finite mixtures of canonical fundamental skew \[t\] t-distributions: the unication of the restricted and unrestricted skew \[t\] t-mixture models. Stat Comp 26:573-589 · Zbl 1420.60020 · doi:10.1007/s11222-015-9545-x
[22] Lee YW, Poon SH (2011) Systemic and systematic factors for loan portfolio loss distribution. Econometrics and applied economics workshops, pp 1-61. School of Social Science, University of Manchester
[23] Lee WL, Chen YC, Hsieh KS (2003) Ultrasonic liver tissues classification by fractal feature vector based on M-band wavelet transform. IEEE Trans Med Imaging 22:382-392 · doi:10.1109/TMI.2003.809593
[24] Lin TI (2014) Learning from incomplete data via parameterized \[t\] t mixture models through eigenvalue decomposition. Comput Stat Data Anal 71:183-195 · Zbl 1471.62120 · doi:10.1016/j.csda.2013.02.020
[25] Lin TI, Wu PH, McLachlan GJ, Lee SX (2015) A robust factor analysis model using the restricted skew-\[t\] t distribution. TEST 24:510-531 · Zbl 1327.62344 · doi:10.1007/s11749-014-0422-2
[26] Lin TI, McLachlan GJ, Lee SX (2016) Extending mixtures of factor models using the restricted multivariate skew-normal distribution. J Multivar Anal 143:398-413 · Zbl 1328.62378 · doi:10.1016/j.jmva.2015.09.025
[27] Lin TI, Wang WL, McLachlan GJ, Lee SX (2018) Robust mixtures of factor analysis models using the restricted multivariate skew-\[t\] t distribution. Stat Model 28:50-72 · Zbl 07289498 · doi:10.1177/1471082X17718119
[28] Liu C, Rubin DB (1994) The ECME algorithm: a simple extension of EM and ECM with faster monotone convergence. Biometrika 81:33-648 · Zbl 0812.62028
[29] McLachlan GJ, Basford KE (1988) Mixture models: inference and application to clustering. Marcel Dekker, New York · Zbl 0697.62050
[30] McLachlan GJ, Krishnan T (2008) The EM algorithm and extensions, 2nd edn. Wiley, New York · Zbl 1165.62019 · doi:10.1002/9780470191613
[31] McLachlan GJ, Peel D (2000) Finite mixture models. Wiley, New York · Zbl 0963.62061 · doi:10.1002/0471721182
[32] McNicholas PD, Murphy TB (2008) Parsimonious Gaussian mixture models. Stat Comp 18:285-296 · doi:10.1007/s11222-008-9056-0
[33] McNicholas PD, Murphy TB, McDaid AF, Frost D (2010) Serial and parallel implementations of model-based clustering via parsimonious Gaussian mixture models. Comput Stat Data Anal 54:711-723 · Zbl 1464.62131 · doi:10.1016/j.csda.2009.02.011
[34] Meng XL, Rubin DB (1993) Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80:267-278 · Zbl 0778.62022 · doi:10.1093/biomet/80.2.267
[35] Murray PM, Browne RP, McNicholas PD (2014a) Mixtures of skew-\[t\] t factor analyzers. Comput Stat Data Anal 77:326-335 · Zbl 1506.62132 · doi:10.1016/j.csda.2014.03.012
[36] Murray PM, McNicholas PD, Browne RP (2014b) Mixtures of common skew-\[t\] t factor analyzers. Stat 3:68-82 · Zbl 1506.62132 · doi:10.1002/sta4.43
[37] Murray PM, Browne RP, McNicholas PD (2017a) A mixture of SDB skew-\[t\] t factor analyzers. Econom Stat 3:160-168 · doi:10.1016/j.ecosta.2017.05.001
[38] Murray PM, Browne RP, McNicholas PD (2017b) Hidden truncation hyperbolic distributions, finite mixtures thereof, and their application for clustering. J Multivar Anal 161:141-156 · Zbl 1403.62028 · doi:10.1016/j.jmva.2017.07.008
[39] Ouyang M, Welsh W, Georgopoulos P (2004) Gaussian mixture clustering and imputation of microarray data. Bioinformatics 20:917-923 · doi:10.1093/bioinformatics/bth007
[40] Prates MO, Cabral CR, Lachos VH (2013) mixsmsn: fitting finite mixture of scale mixture of skew-normal distributions. J Stat Soft 54:1-20 · doi:10.18637/jss.v054.i12
[41] Pyne S, Hu X, Wang K, Rossin E, Lin TI, Maier LM, Baecher-Allan C, McLachlan GJ, Tamayo P, Hafler DA, De Jager PL, Mesirov JP (2009) Automated high-dimensional flow cytometric data analysis. Proc Natl Acad Sci USA 106:8519-8524 · doi:10.1073/pnas.0903028106
[42] Sahu SK, Dey DK, Branco MD (2003) A new class of multivariate skew distributions with application to Bayesian regression models. Can J Stat 31:129-150 · Zbl 1039.62047 · doi:10.2307/3316064
[43] Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461-464 · Zbl 0379.62005 · doi:10.1214/aos/1176344136
[44] Subedi S, McNicholas PD (2014) Variational Bayes approximations for clustering via mixtures of normal inverse Gaussian distributions. Adv Data Anal Classif 8:167-193 · Zbl 1459.62122 · doi:10.1007/s11634-014-0165-7
[45] Teschendorff A, Wang Y, Barbosa-Morais N, Brenton J, Caldas C (2005) A variational Bayesian mixture modelling framework for cluster analysis of gene-expression data. Bioinformatics 21:3025-3033 · doi:10.1093/bioinformatics/bti466
[46] Tortora C, McNicholas P, Browne R (2016) A mixture of generalized hyperbolic factor analyzers. Adv Data Anal Classif 10:423-440 · Zbl 1414.62278 · doi:10.1007/s11634-015-0204-z
[47] Ueda N, Nakano R, Ghahramani Z, Hinton GE (2000) SMEM algorithm for mixture models. Neural Comput 12:2109-2128 · doi:10.1162/089976600300015088
[48] Wang WL (2013) Mixtures of common factor analyzers for high-dimensional data with missing information. J Multivar Anal 117:120-133 · Zbl 1277.62162 · doi:10.1016/j.jmva.2013.02.003
[49] Wang WL (2015) Mixtures of common \[t\] t-factor analyzers for modeling high-dimensional data with missing values. Comput Stat Data Anal 83:223-235 · Zbl 1507.62179 · doi:10.1016/j.csda.2014.10.007
[50] Wang WL, Lin TI (2016) Maximum likelihood inference for the multivariate t mixture model. J Multivar Anal 149:54-64 · Zbl 1341.62138 · doi:10.1016/j.jmva.2016.03.009
[51] Wang WL, Lin TI (2017) Flexible clustering via extended mixtures of common \[t\] t-factor analyzers. AStA Adv Stat Anal 101:227-252 · Zbl 1443.62177 · doi:10.1007/s10182-016-0281-0
[52] Wang K, McLachlan GJ, Ng SK, Peel D (2009) EMMIX-skew: EM algorithm for mixture of multivariate skew normal/\[t\] t distributions. R package version 1.0-12
[53] Wang WL, Castro LM, Lin TI (2017a) Automated learning of \[t\] t factor analysis models with complete and incomplete data. J Multivar Anal 161:157-171 · Zbl 1403.62108 · doi:10.1016/j.jmva.2017.07.009
[54] Wang WL, Liu M, Lin TI (2017b) Robust skew-\[t\] t factor analysis models for handling missing data. Stat Methods Appl 26:649-672 · Zbl 1441.62161 · doi:10.1007/s10260-017-0388-9
[55] Waterhouse, S.; MacKay, D.; Robinson, T.; Touretzky, DS (ed.); Mozer, MC (ed.); Hasselmo, ME (ed.), Bayesian methods for mixture of experts, No. 8 (1996), Cambridge
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.