Particle methods for stochastic differential equation mixed effects models. (English) Zbl 1480.62053
Summary: Parameter inference for stochastic differential equation mixed effects models (SDEMEMs) is challenging. Analytical solutions for these models are rarely available, which means that the likelihood is also intractable. In this case, exact inference (up to the discretisation of the stochastic differential equation) is possible using particle MCMC methods. Although the exact posterior is targeted by these methods, a naive implementation for SDEMEMs can be highly inefficient. Our article develops three extensions to the naive approach which exploit specific aspects of SDEMEMs and other advances such as correlated pseudo-marginal methods. We compare these methods on simulated data and data from a tumour xenography study on mice.
MSC:
| 62F15 | Bayesian inference |
| 60H10 | Stochastic ordinary differential equations (aspects of stochastic analysis) |
| 62P10 | Applications of statistics to biology and medical sciences; meta analysis |
Software:
MsdeParEstReferences:
| [1] | Andrieu, C., Doucet, A., and Holenstein, R. (2010). “Particle Markov chain Monte Carlo methods.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3): 269-342. · Zbl 1184.65001 · doi:10.1111/j.1467-9868.2009.00736.x |
| [2] | Andrieu, C. and Roberts, G. O. (2009). “The pseudo-marginal approach for efficient Monte Carlo computations.” The Annals of Statistics, 37(2): 697-725. · Zbl 1185.60083 · doi:10.1214/07-AOS574 |
| [3] | Ansley, C. F., Kohn, R., and Wong, C.-M. (1993). “Nonparametric spline regression with prior information.” Biometrika, 80(1): 75-88. · Zbl 0771.62027 · doi:10.1093/biomet/80.1.75 |
| [4] | Botha, I., Kohn, R., and Drovandi, C. (2020). “Supplementary Material for “Particle Methods for Stochastic Differential Equation Mixed Effects Models”.” Bayesian Analysis. · Zbl 1480.62053 · doi:10.1214/20-BA1216SUPP |
| [5] | Carpenter, J., Clifford, P., and Fearnhead, P. (1999). “Improved particle filter for nonlinear problems.” IEE Proceedings - Radar, Sonar and Navigation, 146(1): 2-7. |
| [6] | Dahlin, J., Lindsten, F., Kronander, J., and Schön, T. B. (2015). “Accelerating pseudo-marginal Metropolis-Hastings by correlating auxiliary variables.” arXiv preprint arXiv:1511.05483. |
| [7] | Del Moral, P., Doucet, A., and Jasra, A. (2006). “Sequential Monte Carlo samplers.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(3): 411-436. · Zbl 1105.62034 · doi:10.1111/j.1467-9868.2006.00553.x |
| [8] | Delattre, M., Genon-Catalot, V., and Samson, A. (2013). “Maximum likelihood estimation for stochastic differential equations with random effects.” Scandinavian Journal of Statistics, 40(2): 322-343. · Zbl 1328.62148 · doi:10.1111/j.1467-9469.2012.00813.x |
| [9] | Deligiannidis, G., Doucet, A., and Pitt, M. K. (2018). “The correlated pseudomarginal method.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(5): 839-870. · Zbl 1407.62074 · doi:10.1111/rssb.12280 |
| [10] | Donnet, S., Foulley, J.-L., and Samson, A. (2010). “Bayesian analysis of growth curves using mixed models defined by stochastic differential equations.” Biometrics, 66(3): 733-741. · Zbl 1203.62187 · doi:10.1111/j.1541-0420.2009.01342.x |
| [11] | Donnet, S. and Samson, A. (2013a). “A review on estimation of stochastic differential equations for pharmacokinetic/pharmacodynamic models.” Advanced Drug Delivery Reviews, 65(7): 929-939. |
| [12] | Donnet, S. and Samson, A. (2013b). “Using PMCMC in EM algorithm for stochastic mixed models: theoretical and practical issues.” Journal de la Société Française de Statistique, 155(1): 49-72. · Zbl 1316.60115 · doi:10.1007/s10955-014-0946-6 |
| [13] | Doucet, A., Godsill, S., and Andrieu, C. (2000). “On sequential Monte Carlo sampling methods for Bayesian filtering.” Statistics and Computing, 10(3): 197-208. doi: https://doi.org/10.1023/A:1008935410038. |
| [14] | Doucet, A. and Johansen, A. M. (2009). “A tutorial on particle filtering and smoothing: Fifteen years later.” Handbook of Nonlinear Filtering, 12(656-704): 3. · Zbl 1513.60043 |
| [15] | Doucet, A., Pitt, M., Deligiannidis, G., and Kohn, R. (2015). “Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator.” Biometrika, 102(2): 295-313. · Zbl 1452.62055 · doi:10.1093/biomet/asu075 |
| [16] | Duan, J. A., Gelfand, A. E., Sirmans, C., et al. (2009). “Modeling space-time data using stochastic differential equations.” Bayesian Analysis, 4(4): 733-758. · Zbl 1330.86023 · doi:10.1214/09-BA427 |
| [17] | Durham, G. B. and Gallant, A. R. (2002). “Numerical techniques for maximum likelihood estimation of continuous-time diffusion processes.” Journal of Business & Economic Statistics, 20(3): 297-338. · doi:10.1198/073500102288618397 |
| [18] | Friel, N., Mira, A., and Oates, C. J. (2016). “Exploiting multi-core architectures for reduced-variance estimation with intractable likelihoods.” Bayesian Analysis, 11(1): 215-245. · Zbl 1357.62112 · doi:10.1214/15-BA948 |
| [19] | Gerber, M., Chopin, N., and Whiteley, N. (2019). “Negative association, ordering and convergence of resampling methods.” The Annals of Statistics, 47(4): 2236-2260. · Zbl 1429.62154 · doi:10.1214/18-AOS1746 |
| [20] | Golightly, A. and Wilkinson, D. J. (2008). “Bayesian inference for nonlinear multivariate diffusion models observed with error.” Computational Statistics & Data Analysis, 52(3): 1674-1693. · Zbl 1452.62603 · doi:10.1016/j.csda.2007.05.019 |
| [21] | Gordon, N. J., Salmond, D. J., and Smith, A. F. M. (1993). “Novel approach to nonlinear/non-Gaussian Bayesian state estimation.” IEE Proceedings F - Radar and Signal Processing, 140(2): 107-113. |
| [22] | Gunawan, D., Carter, C., and Kohn, R. (2018a). “Efficiently Combining Pseudo Marginal and Particle Gibbs Sampling.” arXiv preprint arXiv:1804.04359. |
| [23] | Gunawan, D., Tran, M. N., Suzuki, K., Dick, J., and Kohn, R. (2016). “Computationally Efficient Bayesian Estimation of High Dimensional Copulas with Discrete and Mixed Margins.” arXiv preprint arXiv:1608.06174. · Zbl 1430.62101 · doi:10.1007/s11222-018-9846-y |
| [24] | Gunawan, D., Tran, M.-N., Suzuki, K., Dick, J., and Kohn, R. (2018b). “Computationally efficient Bayesian estimation of high-dimensional Archimedean copulas with discrete and mixed margins.” Statistics and Computing, 29(5): 933-946. · Zbl 1430.62101 · doi:10.1007/s11222-018-9846-y |
| [25] | Kitagawa, G. (1996). “Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models.” Journal of Computational and Graphical Statistics, 5(1): 1-25. · doi:10.2307/1390750 |
| [26] | Leander, J., Almquist, J., Ahlström, C., Gabrielsson, J., and Jirstrand, M. (2015). “Mixed effects modeling using stochastic differential equations: illustrated by pharmacokinetic data of nicotinic acid in obese Zucker rats.” The AAPS Journal, 17(3): 586-596. |
| [27] | L’Ecuyer, P. (2016). “Randomized quasi-Monte Carlo: an introduction for practitioners.” In 12th International conference on Monte Carlo and quasi-Monte Carlo methods in scientific computing (MCQMC 2016). · Zbl 1407.65013 · doi:10.1007/978-3-319-91436-7_2 |
| [28] | Lindsten, F. and Schön, T. B. (2012). “On the use of backward simulation in the particle Gibbs sampler.” In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, 3845-3848. IEEE. |
| [29] | Mira, A., Solgi, R., and Imparato, D. (2013). “Zero variance Markov chain Monte Carlo for Bayesian estimators.” Statistics and Computing, 23(5): 653-662. · Zbl 1322.62212 · doi:10.1007/s11222-012-9344-6 |
| [30] | Øksendal, B. (2013). Stochastic differential equations: an introduction with applications. Springer Science & Business Media. · doi:10.1007/978-3-662-03185-8 |
| [31] | Oravecz, Z., Tuerlinckx, F., and Vandekerckhove, J. (2011). “A hierarchical latent stochastic differential equation model for affective dynamics.” Psychological Methods, 16(4): 468. · Zbl 1272.62132 · doi:10.1007/s11336-008-9106-8 |
| [32] | Papaspiliopoulos, O., Roberts, G. O., and Sköld, M. (2007). “A General Framework for the Parametrization of Hierarchical Models.” Statistical Science, 22(1): 59-73. URL http://www.jstor.org/stable/27645805. · Zbl 1246.62195 · doi:10.1214/088342307000000014 |
| [33] | Picchini, U., De Gaetano, A., and Ditlevsen, S. (2010). “Stochastic differential mixed-effects models.” Scandinavian Journal of Statistics, 37(1): 67-90. · Zbl 1224.62041 · doi:10.1111/j.1467-9469.2009.00665.x |
| [34] | Picchini, U. and Ditlevsen, S. (2011). “Practical estimation of high dimensional stochastic differential mixed-effects models.” Computational Statistics & Data Analysis, 55(3): 1426-1444. · Zbl 1328.65014 · doi:10.1016/j.csda.2010.10.003 |
| [35] | Picchini, U. and Forman, J. L. (2019). “Bayesian inference for stochastic differential equation mixed effects models of a tumour xenography study.” Journal of the Royal Statistical Society: Series C (Applied Statistics). |
| [36] | Pitt, M. K., dos Santos Silva, R., Giordani, P., and Kohn, R. (2012). “On some properties of Markov chain Monte Carlo simulation methods based on the particle filter.” Journal of Econometrics, 171(2): 134-151. · Zbl 1443.62499 · doi:10.1016/j.jeconom.2012.06.004 |
| [37] | Price, L. F., Drovandi, C. C., Lee, A., and Nott, D. J. (2018). “Bayesian Synthetic Likelihood.” Journal of Computational and Graphical Statistics, 27(1): 1-11. · Zbl 07498962 · doi:10.1080/10618600.2017.1302882 |
| [38] | Sherlock, C., Thiery, A. H., Roberts, G. O., and Rosenthal, J. S. (2015). “On the efficiency of pseudo-marginal random walk Metropolis algorithms.” The Annals of Statistics, 43(1): 238-275. · Zbl 1326.65015 · doi:10.1214/14-AOS1278 |
| [39] | South, L., Oates, C. J., Mira, A., and Drovandi, C. C. (2019). “Regularised Zero-Variance Control Variates for High-Dimensional Variance Reduction.” arXiv preprint arXiv:1811.05073. |
| [40] | Stramer, O. and Bognar, M. (2011). “Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach.” Bayesian Analysis, 6(2): 231-258. · Zbl 1330.60092 · doi:10.1214/11-BA608 |
| [41] | Tran, M.-N., Kohn, R., Quiroz, M., and Villani, M. (2016). “The block pseudo-marginal sampler.” arXiv preprint arXiv:1603.02485. |
| [42] | Tran, M.-N., Scharth, M., Pitt, M. K., and Kohn, R. (2013). “Importance sampling squared for Bayesian inference in latent variable models.” arXiv preprint arXiv:1309.3339. |
| [43] | Vats, D., Flegal, J. M., and Jones, G. L. (2015). “Multivariate output analysis for Markov chain Monte Carlo.” arXiv preprint arXiv:1512.07713. · Zbl 1434.62100 |
| [44] | Whitaker, G. A., Golightly, A., Boys, R. J., and Sherlock, C. (2017a). “Bayesian inference for diffusion-driven mixed-effects models.” Bayesian Analysis, 12(2): 435-463. · Zbl 1384.62109 · doi:10.1214/16-BA1009 |
| [45] | Whitaker, G. A., Golightly, A., Boys, R. J., and Sherlock, C. (2017b). “Improved bridge constructs for stochastic differential equations.” Statistics and Computing, 27(4): 885-900. · Zbl 1384.65011 · doi:10.1007/s11222-016-9660-3 |
| [46] | Whiteley, N. (2010). “Discussion on particle Markov chain Monte Carlo methods.” Journal of the Royal Statistical Society: Series B, 72(3): 306-307. |
| [47] | Wiqvist, S., Golightly, A., Mclean, T. A., and Picchini, U. (2019). “Efficient inference for stochastic differential mixed-effects models using correlated particle pseudo-marginal algorithms.” arXiv preprint arXiv:1907.09851. · Zbl 1510.62079 |
| [48] | Wood, S. N. (2010). “Statistical inference for noisy nonlinear ecological dynamic systems.” Nature, 466(7310): 1102 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
