×

Dynamic and robust Bayesian graphical models. (English) Zbl 1499.62023

Summary: Gaussian graphical models are widely popular for studying the conditional dependence among random variables. By encoding conditional dependence as an undirected graph, Gaussian graphical models provide interpretable representations and insightful visualizations of the relationships among variables. However, time series data present additional challenges: the graphs can evolve over time – with changes occurring at unknown time points – and the data often exhibit heavy-tailed characteristics. To address these challenges, we propose dynamic and robust Bayesian graphical models that employ state-of-the-art hidden Markov models (HMMs) to introduce dynamics in the graph and heavy-tailed multivariate \(t\)-distributions for model robustness. The HMM latent states are linked both temporally and hierarchically for greater information sharing across time and between states. The proposed methods are computationally efficient and demonstrate excellent graph estimation on simulated data with substantial improvements over non-robust graphical models. We demonstrate our proposed approach on human hand gesture tracking data, and discover edges and dynamics with well explained practical meanings.

MSC:

62-08 Computational methods for problems pertaining to statistics
62F15 Bayesian inference
62H12 Estimation in multivariate analysis
62M05 Markov processes: estimation; hidden Markov models
62P10 Applications of statistics to biology and medical sciences; meta analysis

Software:

BayesDA; glasso; loggle
Full Text: DOI

References:

[1] Cheok, MJ; Omar, Z.; Jaward, MH, A review of hand gesture and sign language recognition techniques, Int. J. Mach. Learn. Cybern., 10, 1, 131-153 (2019) · doi:10.1007/s13042-017-0705-5
[2] Cremaschi, A., Argiento, R., Shoemaker, K., Peterson, C., Vannucci, M.: Hierarchical normalized completely random measures for robust graphical modeling. Bayesian Anal. 14(4), 1271 (2019) · Zbl 1435.62121
[3] Danaher, P.; Wang, P.; Witten, DM, The joint graphical lasso for inverse covariance estimation across multiple classes, J. R. Stat. Soc. Ser. B Stat. Methodol., 76, 2, 373 (2014) · Zbl 07555455 · doi:10.1111/rssb.12033
[4] Fan, J.; Feng, Y.; Wu, Y., Network exploration via the adaptive Lasso and SCAD penalties, Ann. Appl. Stat., 3, 2, 521 (2009) · Zbl 1166.62040 · doi:10.1214/08-AOAS215
[5] Finegold, M.; Drton, M., Robust graphical modeling of gene networks using classical and alternative \(t\)-distributions, Ann. Appl. Stat., 5, 2, 1057-1080 (2011) · Zbl 1232.62083 · doi:10.1214/10-AOAS410
[6] Finegold, M.; Drton, M., Robust Bayesian graphical modeling using Dirichlet \(t \)-distributions, Bayesian Anal., 9, 3, 521-550 (2014) · Zbl 1327.62143 · doi:10.1214/13-BA856
[7] Friedman, J.; Hastie, T.; Tibshirani, R., Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9, 3, 432-441 (2008) · Zbl 1143.62076 · doi:10.1093/biostatistics/kxm045
[8] Gelman, A.; Carlin, JB; Stern, HS; Rubin, DB, Bayesian Data Analysis (1995), London: Chapman and Hall/CRC, London · Zbl 1039.62018 · doi:10.1201/9780429258411
[9] Geweke, J., et al.: Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments, vol. 196. Federal Reserve Bank of Minneapolis, Research Department Minneapolis (1991)
[10] Gibberd, AJ; Nelson, JD, Regularized estimation of piecewise constant Gaussian graphical models: the group-fused graphical lasso, J. Comput. Graph. Stat., 26, 3, 623-634 (2017) · doi:10.1080/10618600.2017.1302340
[11] Gottard, A.; Pacillo, S., Robust concentration graph model selection, Comput. Stat. Data Anal., 54, 12, 3070-3079 (2010) · Zbl 1284.62199 · doi:10.1016/j.csda.2008.11.021
[12] Han, T.X., Ning, H., Huang, T.S.: Efficient nonparametric belief propagation with application to articulated body tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 214-221. IEEE (2006)
[13] Ishwaran, H.; James, LF, Gibbs sampling methods for stick-breaking priors, J. Am. Stat. Assoc., 96, 453, 161-173 (2001) · Zbl 1014.62006 · doi:10.1198/016214501750332758
[14] Jasra, A.; Holmes, CC; Stephens, DA, Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling, Stat. Sci., 20, 1, 50-67 (2005) · Zbl 1100.62032 · doi:10.1214/088342305000000016
[15] Kendon, A., Gesture: Visible Action as Utterance (2004), Cambridge: Cambridge University Press, Cambridge · doi:10.1017/CBO9780511807572
[16] Kim, D.; Song, J.; Kim, D., Simultaneous gesture segmentation and recognition based on forward spotting accumulative HMMs, Pattern Recognit., 40, 11, 3012-3026 (2007) · Zbl 1118.68625 · doi:10.1016/j.patcog.2007.02.010
[17] Kolar, M.; Song, L.; Ahmed, A.; Xing, EP, Estimating time-varying networks, Ann. Appl. Stat., 4, 1, 94-123 (2010) · Zbl 1189.62142 · doi:10.1214/09-AOAS308
[18] Kotz, S.; Nadarajah, S., Multivariate \(t\)-Distributions and Their Applications (2004), Cambridge: Cambridge University Press, Cambridge · Zbl 1100.62059 · doi:10.1017/CBO9780511550683
[19] Lenkoski, A.; Dobra, A., Computational aspects related to inference in Gaussian graphical models with the G-Wishart prior, J. Comput. Graph. Stat., 20, 1, 140-157 (2011) · doi:10.1198/jcgs.2010.08181
[20] Liu, X.; Daniels, MJ, A new algorithm for simulating a correlation matrix based on parameter expansion and reparameterization, J. Comput. Graph. Stat., 15, 4, 897-914 (2006) · doi:10.1198/106186006X160681
[21] Liu, Y., Wichura, M.J., Drton, M.: Rejection sampling for an extended Gamma distribution. Submitted (2012)
[22] Lun, R.; Zhao, W., A survey of applications and human motion recognition with microsoft kinect, Int. J. Pattern Recognit. Artif. Intell., 29, 5, 1555008 (2015) · doi:10.1142/S0218001415550083
[23] Madeo, R.C., Lima, C.A., Peres, S.M.: Gesture unit segmentation using support vector machines: segmenting gestures from rest positions. In: Proceedings of the 28th Annual ACM Symposium on Applied Computing, pp. 46-52 (2013)
[24] Meinshausen, N.; Bühlmann, P., High-dimensional graphs and variable selection with the lasso, Ann. Stat., 34, 3, 1436-1462 (2006) · Zbl 1113.62082 · doi:10.1214/009053606000000281
[25] Mitra, S., Acharya, T.: Gesture recognition: a survey. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 37(3), 311-324 (2007)
[26] Miyamura, M.; Kano, Y., Robust Gaussian graphical modeling, J. Multivar. Anal., 97, 7, 1525-1550 (2006) · Zbl 1093.62038 · doi:10.1016/j.jmva.2006.02.006
[27] Neal, RM, Markov chain sampling methods for Dirichlet process mixture models, J. Comput. Graph. Stat., 9, 2, 249-265 (2000)
[28] Ni, Y., Baladandayuthapani, V., Vannucci, M., Stingo, F.C.: Bayesian graphical models for modern biological applications. In: Statistical Methods and Applications, pp. 1-29 (2021) · Zbl 1515.62112
[29] Osborne, N.; Peterson, CB; Vannucci, M., Latent network estimation and variable selection for compositional data via variational EM, J. Comput. Graph. Stat., 31, 1, 163-175 (2022) · Zbl 07546467 · doi:10.1080/10618600.2021.1935971
[30] Peng, J.; Wang, P.; Zhou, N.; Zhu, J., Partial correlation estimation by joint sparse regression models, J. Am. Stat. Assoc., 104, 486, 735-746 (2009) · Zbl 1388.62046 · doi:10.1198/jasa.2009.0126
[31] Peterson, CB; Osborne, N.; Stingo, FC; Bourgeat, P.; Doecke, JD; Vannucci, M., Bayesian modeling of multiple structural connectivity networks during the progression of Alzheimer’s disease, Biometrics, 76, 4, 1120-1132 (2020) · Zbl 1520.62306 · doi:10.1111/biom.13235
[32] Pinheiro, JC; Liu, C.; Wu, YN, Efficient algorithms for robust estimation in linear mixed-effects models using the multivariate \(t\) distribution, J. Comput. Graph. Stat., 10, 2, 249-276 (2001) · doi:10.1198/10618600152628059
[33] Rodríguez, CE; Walker, SG, Label switching in Bayesian mixture models: deterministic relabeling strategies, J. Comput. Graph. Stat., 23, 1, 25-45 (2014) · doi:10.1080/10618600.2012.735624
[34] Roverato, A., Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models, Scand. J. Stat., 29, 3, 391-411 (2002) · Zbl 1036.62027 · doi:10.1111/1467-9469.00297
[35] Scott, SL, Bayesian methods for hidden Markov models: recursive computing in the 21st century, J. Am. Stat. Assoc., 97, 457, 337-351 (2002) · Zbl 1073.65503 · doi:10.1198/016214502753479464
[36] Shaddox, E.; Stingo, FC; Peterson, CB; Jacobson, S.; Cruickshank-Quinn, C.; Kechris, K.; Bowler, R.; Vannucci, M., A Bayesian approach for learning gene networks underlying disease severity in COPD, Stat. Biosci., 10, 1, 59-85 (2018) · doi:10.1007/s12561-016-9176-6
[37] Sigal, L.; Isard, M.; Haussecker, H.; Black, MJ, Loose-limbed people: estimating 3d human pose and motion using non-parametric belief propagation, Int. J. Comput. Vis., 98, 1, 15-48 (2012) · Zbl 1254.68283 · doi:10.1007/s11263-011-0493-4
[38] Song, Y.; Goncalves, L.; Perona, P., Unsupervised learning of human motion, IEEE Trans. Pattern Anal. Mach. Intell., 25, 7, 814-827 (2003) · doi:10.1109/TPAMI.2003.1206511
[39] Sperrin, M.; Jaki, T.; Wit, E., Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models, Stat. Comput., 20, 3, 357-366 (2010) · doi:10.1007/s11222-009-9129-8
[40] Sun, H.; Li, H., Robust Gaussian graphical modeling via \(\ell_1\) penalization, Biometrics, 68, 4, 1197-1206 (2012) · Zbl 1259.62102 · doi:10.1111/j.1541-0420.2012.01785.x
[41] Tan, LS; Jasra, A.; De Iorio, M.; Ebbels, TM, Bayesian inference for multiple Gaussian graphical models with application to metabolic association networks, Ann. App. Stat., 11, 4, 2222-2251 (2017) · Zbl 1383.62294
[42] Vinciotti, V.; Hashem, H., Robust methods for inferring sparse network structures, Comput. Stat. Data Anal., 67, 84-94 (2013) · Zbl 1471.62201 · doi:10.1016/j.csda.2013.05.004
[43] Wang, H., Bayesian graphical lasso models and efficient posterior computation, Bayesian Anal., 7, 4, 867-886 (2012) · Zbl 1330.62041 · doi:10.1214/12-BA729
[44] Wang, H., Scaling it up: stochastic search structure learning in graphical models, Bayesian Anal., 10, 2, 351-377 (2015) · Zbl 1335.62068 · doi:10.1214/14-BA916
[45] Warnick, R.; Guindani, M.; Erhardt, E.; Allen, E.; Calhoun, V.; Vannucci, M., A Bayesian approach for estimating dynamic functional network connectivity in fMRI data, J. Am. Stat. Assoc., 113, 521, 134-151 (2018) · Zbl 1398.62350 · doi:10.1080/01621459.2017.1379404
[46] Xu, R., Wu, J., Yue, X., Li, Y.: Online structural change-point detection of high-dimensional streaming data via dynamic sparse subspace learning. In: Technometrics, pp. 1-14 (2022)
[47] Yang, E., Lozano, A.C.: Robust Gaussian graphical modeling with the trimmed graphical lasso. In: Advances in Neural Information Processing Systems, pp. 2602-2610 (2015)
[48] Yang, J., Peng, J.: Estimating time-varying graphical models. J. Comput. Graph. Stat. 29(1), 191-202 (2020) · Zbl 07499282
[49] Zhang, C.; Yan, H.; Lee, S.; Shi, J., Dynamic multivariate functional data modeling via sparse subspace learning, Technometrics, 63, 3, 370-383 (2021) · doi:10.1080/00401706.2020.1800516
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.