Quasi-Monte Carlo for highly structured generalised response models. (English) Zbl 1234.62110
Summary: Highly structured generalised response models, such as generalised linear mixed models and generalised linear models for time series regression, have become an indispensable vehicle for data analysis and inference in many areas of application. However, their use in practice is hindered by high-dimensional intractable integrals. Quasi-Monte Carlo (QMC) is a dynamic research area in the general problem of high-dimensional numerical integration, although its potential for statistical applications is yet to be fully explored. We survey recent research in \(QMC\), particularly lattice rules, and report on their application to highly structured generalised response models. New challenges for QMC are identified and new methodologies are developed. QMC methods are seen to provide significant improvements compared with ordinary Monte Carlo methods.
MSC:
| 62J12 | Generalized linear models (logistic models) |
| 65C05 | Monte Carlo methods |
| 62M10 | Time series, auto-correlation, regression, etc. in statistics (GARCH) |
| 62G08 | Nonparametric regression and quantile regression |
Keywords:
Generalised linear mixed models; High-dimensional integration; Lattice rules; Longitudinal data analysis; Maximum likelihood; Semiparametric regression; Serial dependence; Time series regressionReferences:
| [1] | P. J. Acklam, ”An algorithm for computing the inverse normal cumulative distribution function,” http://home.online.no/jacklam/notes/invnorm/ , 2007. |
| [2] | E. Al-Eid, and J. Pan, ”Estimation in generalized linear mixed models using SNTO approximation.” In A. R. Francis, K. M. Matawie, A. Oshlack, and G. K. Smyth (eds.), Proceedings of the 20th International Workshop on Statistical Modelling, pp. 77–84, Sydney, Australia, 2005. |
| [3] | E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK Users’ Guide, Third Edition, SIAM: Philadelphia, 1999. |
| [4] | D. Bernat, W. T. M. Dunsmuir, and A. C. Wagenaar, ”Effects of lowering the BAC limit to .08 on fatal traffic crashes in 19 states,” Accident Analysis and Prevention vol. 36 pp. 1089–1097, 2004. · doi:10.1016/j.aap.2004.04.001 |
| [5] | J. G. Booth, J. P. Hobert, and W. Jank, ”A survey of Monte Carlo algorithms for maximizing the likelihood of a two-stage hierarchical model,” Statistical Modelling vol. 1 pp. 333–349, 2001. · Zbl 1102.62019 · doi:10.1191/147108201128249 |
| [6] | P. Bratley, and B. L. Fox, ”Algorithm 659: Implementing Sobol’s quasirandom sequence generator,” ACM Transactions on Mathematical Software vol. 14 pp. 88–100, 1988. · Zbl 0642.65003 · doi:10.1145/42288.214372 |
| [7] | N. E. Breslow, and D. G. Clayton, ”Approximate inference in generalized linear mixed models,” Journal of the American Statistical Association vol. 88 pp. 9–25, 1993. · Zbl 0775.62195 · doi:10.2307/2290687 |
| [8] | BUGS Project, ”BUGS: Bayesian Inference Using Gibbs Sampling,” http://www.mrc-bsu.cam.ac.uk/bugs , 2007. |
| [9] | R. E. Caflisch, W. Morokoff, and A. Owen, ”Valuation of mortgage backed securities using Brownian bridges to reduce effective dimension,” Journal of Computational Finance vol. 1 pp. 27–46, 1997. |
| [10] | B. S. Caffo, W. Jank, and G. L. Jones, ”Ascent-based Monte Carlo expectation-maximization,” Journal of the Royal Statistical Society Series B vol. 67 pp. 235–251, 2005. · Zbl 1075.65011 · doi:10.1111/j.1467-9868.2005.00499.x |
| [11] | D. Clayton, ”Generalized linear mixed models.” In W. R. Gilks, S. Richardson, and D. J. Spiegelhalter (eds.), Markov Chain Monte Carlo in Practice, pp. 275–301, Chapman & Hall: London, 1996. · Zbl 0841.62059 |
| [12] | R. Cools, F. Y. Kuo, and D. Nuyens, ”Constructing embedded lattice rules for multivariate integration,” SIAM Journal on Scientific Computing vol. 28, pp. 2162–2188, 2006. · Zbl 1126.65002 · doi:10.1137/06065074X |
| [13] | C. Crainiceanu, D. Ruppert, and M. P. Wand, ”Bayesian analysis for penalised spline regression using WinBUGS,” Journal of Statistical Software vol. 14(14), 2005. · Zbl 1068.62021 |
| [14] | R. A. Davis, W. T. M. Dunsmuir, and Y. Wang, ”Modelling time series of count data.” In S. Ghosh (ed.), Asymptotics, Nonparametrics and Time Series, pp. 63–114, Marcel-Dekker: New York, 1999. · Zbl 1069.62540 |
| [15] | R. A. Davis, W. T. M. Dunsmuir, and Y. Wang, ”On autocorrelation in a Poisson regression model,” Biometrika vol. 87 pp. 491–505, 2000. · Zbl 0956.62075 · doi:10.1093/biomet/87.3.491 |
| [16] | R. A. Davis, and G. Rodriguez-Yam, ”Estimation for state-space models based on a likelihood approximation,” Statistica Sinica vol. 15 pp. 81–406, 2005. · Zbl 1070.62070 |
| [17] | J. Dick, F. Pillichshammer, and B. J. Waterhouse, ”The construction of good extensible rank-1 lattices,” Mathematics of Computation, (in press), 2007. · Zbl 1131.11051 |
| [18] | P. Diggle, K.-L. Liang, and S. Zeger, Analysis of Longitudinal Data, Oxford University Press: Oxford, 1995. · Zbl 1031.62002 |
| [19] | P. Diggle, P. Heagerty, K.-L. Liang, and S. Zeger, Analysis of Longitudinal Data, Second Edition, Oxford University Press: Oxford, 2002. · Zbl 1031.62002 |
| [20] | M. Evans, and T. Swartz, ”Methods for approximating integrals in statistics with special emphasis on Bayesian integration problems,” Statistical Science vol. 10 pp. 254–272, 1995. · Zbl 0955.62553 · doi:10.1214/ss/1177009938 |
| [21] | R. Fletcher, Practical Methods of Optimisation, Second Edition, Wiley: Chichester, 1987. · Zbl 0905.65002 |
| [22] | J. Gonzalez, F. Tuerlinckx, P. De Boeck, and R. Cools, ”Numerical integration in logistic-normal models,” Computational Statistics and Data Analysis vol. 51 pp. 1535–1548, 2006. · Zbl 1157.65336 · doi:10.1016/j.csda.2006.05.003 |
| [23] | L. C. Gurrin, K. J. Scurrah, and M. L. Hazelton, ”Tutorial in biostatistics: Spline smoothing with linear mixed models,” Statistics in Medicine vol. 24 pp. 3361–3381, 2005. · doi:10.1002/sim.2193 |
| [24] | F. J. Hickernell, and H. S. Hong, ”Quasi-Monte Carlo methods and their randomisations.” In R. Chan, Y.-K. Kwok, D. Yao, and Q. Zhang (eds.), Applied Probability, AMS/IP Studies in Advanced Mathematics, vol. 26, pp. 59–77, American Mathematical Society: Providence, 2002. · Zbl 1007.65003 |
| [25] | F. J. Hickernell, C. Lemieux, and A. B. Owen, ”Control variates for quasi-monte carlo,” Statistical Science vol. 20 pp. 1–31, 2005. · Zbl 1100.65006 · doi:10.1214/088342304000000468 |
| [26] | W. Jank, ”Quasi-Monte Carlo sampling to improve the efficiency of Monte Carlo EM,” Computational Statistics and Data Analysis vol. 48 pp. 685–701, 2005. · Zbl 1429.62021 · doi:10.1016/j.csda.2004.03.019 |
| [27] | S. Joe, and F. Y. Kuo, ”Remark on Algorithm 659: Implementing Sobol’s quasirandom sequence generator,” ACM Transactions on Mathematical Software vol. 29 pp. 49–57, 2003. · Zbl 1070.65501 · doi:10.1145/641876.641879 |
| [28] | A. Y. C. Kuk, ”Laplace importance sampling for generalized linear mixed models,” Journal of Statistical Computation and Simulation vol. 63 pp. 143–158, 1999. · Zbl 0956.62052 · doi:10.1080/00949659908548522 |
| [29] | F. Y. Kuo, ”Component-by-component constructions achieve the optimal rate of convergence for multivariate integration in weighted Korobov and Sobolev spaces,” Journal of Complexity vol. 19 pp. 301–320, 2003. · Zbl 1027.41031 · doi:10.1016/S0885-064X(03)00006-2 |
| [30] | F. Y. Kuo, G. W. Wasilkowski, and B. J. Waterhouse, ”Randomly-shifted lattice rules for unbounded integrands,” Journal of Complexity vol. 22 pp. 630–651, 2006. · Zbl 1112.65003 · doi:10.1016/j.jco.2006.04.006 |
| [31] | F. Y. Kuo, and I. H. Sloan, ”Lifting the curse of dimensionality,” Notices of the American Mathematical Society vol. 52 pp. 1320–1328, 2005. · Zbl 1080.65020 |
| [32] | P. L’Ecuyer, and C. Lemieux, ”Recent advances in randomized quasi-Monte Carlo methods.” In M. Dror, P. L’Ecuyer, and F. Szidarovszki (eds.), Modeling Uncertainty: An Examination of Stochastic Theory, Methods, and Applications, pp. 419–474, Kluwer Dordrecht, 2002. |
| [33] | R. Liu, and A. Owen, ”Estimating mean dimensionality of analysis of variance decompositions,” Journal of the American Statistical Association vol. 101 pp. 712–721, 2006. · Zbl 1119.62343 · doi:10.1198/016214505000001410 |
| [34] | C. E. McCulloch, and S. R. Searle, Generalized, Linear, and Mixed Models, Wiley: New York, 2000. · Zbl 0964.62061 |
| [35] | H. Niederreiter, Random Number Generation and Quasi-Monte Carlo Methods, SIAM: Philadelphia, 1992. · Zbl 0761.65002 |
| [36] | H. Niederreiter, ”Construction of (t,m,s)-nets and (t,s)-sequences,” Finite Fields and Their Applications vol. 11, pp. 578–600, 2005. · Zbl 1087.11051 · doi:10.1016/j.ffa.2005.01.001 |
| [37] | J. Nocedal, and S. J. Wright, Numerical Optimization, Springer, 1999. · Zbl 0930.65067 |
| [38] | D. Nuyens, and R. Cools, ”Fast algorithms for component-by-component construction of rank-1 lattice rules in shift-invariant reproducing kernel Hilbert spaces,” Mathematics of Computation vol. 75 pp. 903–920, 2006. · Zbl 1094.65004 · doi:10.1090/S0025-5718-06-01785-6 |
| [39] | A. R. Owen, and S. D. Tribble, ”A quasi-Monte Carlo Metropolis algorithm,” Proceedings of the National Academy of Sciences vol. 102 pp. 8844–8849, 2005. · Zbl 1135.65001 · doi:10.1073/pnas.0409596102 |
| [40] | J.-X. Pan, and R. Thompson, ”Quasi-Monte Carlo EM algorithm for estimation in generalized linear mixed models.” In R. Payne, and P. Green (eds.), Proceedings in Computational Statistics, pp. 419–424, Physical-Verlag, 1998. · Zbl 0951.62059 |
| [41] | J. Pan, and R. Thompson, ”Quasi-Monte Carlo estimation in generalized linear mixed models.” In A. Biggeri, E. Dreassi, C. Lagazio, M. Marchi (eds.), Proceedings of the 19th International Workshop on Statistical Modelling, pp. 239–243, Firenze University Press: FLorence, 2004. |
| [42] | J. Pan, and R. Thompson, ”Quasi-Monte Carlo approximation for estimation in generalized linear mixed models,” Computational Statistics and Data Analysis vol. 51 pp. 5765–5775, 2007. · Zbl 1445.62196 · doi:10.1016/j.csda.2006.10.003 |
| [43] | S. H. Paskov, and J. F. Traub, ”Faster valuation of financial derivatives,” Journal of Portfolio Management vol. 22 pp. 113–120, 1995. · doi:10.3905/jpm.1995.409541 |
| [44] | W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C, Second Edition, Cambridge University Press, 1995. · Zbl 0778.65003 |
| [45] | D. Ruppert, M. P. Wand, and R. J. Carroll, Semiparametric Regression, Cambridge University Press: New York, 2003. · Zbl 1038.62042 |
| [46] | SAS Institute, Inc, http://www.sas.com , 2007. |
| [47] | A. Skrondal, and S. Rabe-Hesketh, Generalized Latent Variable Modelling: Multilevel, Longitudinal and Structural Equation Models, Chapman and Hall: Boca Raton, Florida, 2004. · Zbl 1097.62001 |
| [48] | I. H. Sloan, and S. Joe, Lattice Methods for Multiple Integration, Oxford University Press: Oxford, 1994. · Zbl 0855.65013 |
| [49] | I. H. Sloan, F. Y. Kuo, and S. Joe, ”Constructing randomly shifted lattice rules in weighted Sobolev spaces,” SIAM Journal on Numerical Analysis vol. 40 pp. 1650–1665, 2002. · Zbl 1037.65005 · doi:10.1137/S0036142901393942 |
| [50] | I. H. Sloan, X. Wang, and H. Woźniakowski, ”Finite-order weights imply tractability of multivariate integration,” Journal of Complexity vol. 20 pp. 46–74, 2004. · Zbl 1067.65006 · doi:10.1016/j.jco.2003.11.003 |
| [51] | I. H. Sloan, and H. Woźniakowski, ”When are quasi-Monte Carlo algorithms efficient for high dimensional integrals?” Journal of Complexity vol. 14 pp. 1–33, 1998. · Zbl 1032.65011 · doi:10.1006/jcom.1997.0463 |
| [52] | M. P. Wand, ”Smoothing and mixed models,” Computational Statistics vol. 18 pp. 223–249, 2003. · Zbl 1050.62049 |
| [53] | X. Wang, and K. T. Fang, ”Effective dimensions and quasi-Monte Carlo integration,” Journal of Complexity vol. 19 pp. 101–124, 2003. · Zbl 1021.65002 · doi:10.1016/S0885-064X(03)00003-7 |
| [54] | R. Wolfinger, and M. O’Connell, ”Generalized linear mixed models: a pseudo-likelihood approach,” Journal of Statistical Computation and Simulation vol. 48 pp. 233–243, 1993. · Zbl 0833.62067 · doi:10.1080/00949659308811554 |
| [55] | Y. Zhao, J. Staudenmayer, B. A. Coull, and M. P. Wand, ”General design Bayesian generalized linear mixed models,” Statistical Science vol. 21 pp. 35–51, 2006. · Zbl 1129.62063 · doi:10.1214/088342306000000015 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
