On the global and componentwise rates of convergence of the EM algorithm. (English) Zbl 0818.65153
For incomplete-data problems in statistics, the EM algorithm is a very popular one for finding maximum-likelihood estimates and posterior modes. It is simple to apply and to implement, and it is stable. Each iteration of EM consists of an expectation step and a maximization step.
After recalling that the order of the EM algorithm is generally linear, the authors focus on the rate of convergence for linear iterations. By using the diagonalizability theorem, they describe how and when the componentwise rates differ and their relationships with the global rate. This study is nicely enlightened by an example of a standard contaminated normal model which illustrates that these phenomena are not necessarily pathological and can occur in useful statistical models.
After recalling that the order of the EM algorithm is generally linear, the authors focus on the rate of convergence for linear iterations. By using the diagonalizability theorem, they describe how and when the componentwise rates differ and their relationships with the global rate. This study is nicely enlightened by an example of a standard contaminated normal model which illustrates that these phenomena are not necessarily pathological and can occur in useful statistical models.
Reviewer: M.Bernadou (Le Chesnay)
MSC:
| 65C99 | Probabilistic methods, stochastic differential equations |
| 62H12 | Estimation in multivariate analysis |
| 62M15 | Inference from stochastic processes and spectral analysis |
References:
| [1] | Boyles, R. A., On the convergence of the EM algorithm, J. Roy. Statist. Soc. Ser. B, 45, 47-50 (1983) · Zbl 0508.62030 |
| [2] | Cox, D. R.; Hinkley, D. V., Theoretical Statistics (1974), Wiley: Wiley New York · Zbl 0334.62003 |
| [3] | Dempster, A. P.; Laird, N. M.; Rubin, D. B., Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion), J. Roy. Statist. Soc. Ser. B, 39, 1-38 (1977) · Zbl 0364.62022 |
| [4] | Faddeev, D. K.; Faddeeva, V. N., Computational Methods of Linear Algebra (1963), Freeman: Freeman San Francisco · Zbl 0112.07503 |
| [5] | Healy, M. J.R.; Westmacott, M., Missing values in experiments analyzed on automatic computers, Appl. Statist., 5, 203-206 (1956) |
| [6] | Little, R. J.A.; Rubin, D. B., Statistical Analysis with Missing Data (1987), Wiley: Wiley New York · Zbl 0665.62004 |
| [7] | Meng, X. L.; Pedlow, S., EM: A bibliographic review with missing articles, Proc. Statist. Comput. Sect. Amer. Statist. (1992), to appear. |
| [8] | Meng, X. L.; Rubin, D. B., Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm, J. Amer. Statist. Assoc., 86, 899-909 (1991) |
| [9] | Meng, X. L.; Rubin, D. B., Recent extensions to the EM algorithm (with discussion), (Bernardo, J. M.; Berger, J. O.; Dawid, A. P.; Smith, A. F.M., Bayesian Statistics 4 (1992), Oxford U.P: Oxford U.P Oxford), 307-320 |
| [10] | Meng, X. L.; Rubin, D. B., Maximum likelihood estimation via the ECM algorithm: A general framework, Biometrika, 80, 267-278 (1993) · Zbl 0778.62022 |
| [11] | Orchard, T.; Woodbury, M. A., A missing information principle: Theory and application, Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 697-715 (1972) · Zbl 0263.62023 |
| [12] | Searle, S. R., Matrix Algebra Useful for Statistics (1982), Wiley: Wiley New York · Zbl 0555.62002 |
| [13] | Wu, C. F.J., On the convergence properties of the EM algorithm, Ann. Statist., 11, 95-103 (1983) · Zbl 0517.62035 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
