×

On predictive inference for intractable models via approximate Bayesian computation. (English) Zbl 1516.62016

Summary: Approximate Bayesian computation (ABC) is commonly used for parameter estimation and model comparison for intractable simulator-based statistical models whose likelihood function cannot be evaluated. In this paper we instead investigate the feasibility of ABC as a generic approximate method for predictive inference, in particular, for computing the posterior predictive distribution of future observations or missing data of interest. We consider three complementary ABC approaches for this goal, each based on different assumptions regarding which predictive density of the intractable model can be sampled from. The case where only simulation from the joint density of the observed and future data given the model parameters can be used for inference is given particular attention and it is shown that the ideal summary statistic in this setting is minimal predictive sufficient instead of merely minimal sufficient (in the ordinary sense). An ABC prediction approach that takes advantage of a certain latent variable representation is also investigated. We additionally show how common ABC sampling algorithms can be used in the predictive settings considered. Our main results are first illustrated by using simple time-series models that facilitate analytical treatment, and later by using two common intractable dynamic models.

MSC:

62-08 Computational methods for problems pertaining to statistics

Software:

brlm; BayesDA; epiABC

References:

[1] Andrieu, C.; Doucet, A.; Holenstein, R., Particle Markov chain Monte Carlo methods, J. R. Stat. Soc. Ser. B, 72, 3, 269-342 (2010) · Zbl 1411.65020
[2] Barber, S.; Voss, J.; Webster, M., The rate of convergence for approximate Bayesian computation, Electron. J. Stat., 9, 1, 80-105 (2015) · Zbl 1307.62063
[3] Beaumont, MA; Cornuet, J-M; Marin, J-M; Robert, CP, Adaptive approximate Bayesian computation, Biometrika, 96, 4, 983-990 (2009) · Zbl 1437.62393
[4] Beaumont, MA; Zhang, W.; Balding, DJ, Approximate Bayesian computation in population genetics, Genetics, 162, 4, 2025-2035 (2002)
[5] Bernardo, JM; Smith, AFM, Bayesian Theory (1994), Hoboken: Wiley, Hoboken
[6] Bernton, E.; Jacob, PE; Gerber, M.; Robert, CP, Approximate Bayesian computation with the Wasserstein distance, J. R. Stat. Soc. Ser. B, 81, 2, 235-269 (2019) · Zbl 1420.62022
[7] Biau, G.; Cérou, F.; Guyader, A., New insights into approximate Bayesian computation, Ann. Inst. Henri Poincaré Probab. Stat., 51, 1, 376-403 (2015) · Zbl 1307.62012
[8] Bjørnstad, JF, On the generalization of the likelihood function and the likelihood principle, J. Am. Stat. Assoc., 91, 434, 791-806 (1996) · Zbl 0871.62006
[9] Blum, MGB, Approximate Bayesian computation: a nonparametric perspective, J. Am. Stat. Assoc., 105, 491, 1178-1187 (2010) · Zbl 1390.62052
[10] Buckwar, E.; Tamborrino, M.; Tubikanec, I., Spectral density-based and measure-preserving abc for partially observed diffusion processes: an illustration on Hamiltonian SDEs, Statist. Comput., 30, 3, 627-648 (2020) · Zbl 1505.62081
[11] Bürkner, P-C; Gabry, J.; Vehtari, A., Approximate leave-future-out cross-validation for Bayesian time series models, J. Stat. Comput. Simul., 90, 14, 2499-2523 (2020) · Zbl 07480189
[12] Calvet, LE; Czellar, V., Accurate methods for approximate Bayesian computation filtering, J. Financ. Economet., 13, 4, 798-838 (2014)
[13] Canale, A.; Ruggiero, M., Bayesian nonparametric forecasting of monotonic functional time series, Electron. J. Stat., 10, 2, 3265-3286 (2016) · Zbl 1357.62278
[14] Del Moral, P.; Doucet, A.; Jasra, A., An adaptive sequential Monte Carlo method for approximate Bayesian computation, Stat. Comput., 22, 5, 1009-1020 (2012) · Zbl 1252.65025
[15] Drovandi, C., Nott, D.J., Frazier, D.T.: Improving the accuracy of marginal approximations in likelihood-free inference via localisation. Available at https://arxiv.org/abs/2207.06655 (2022)
[16] Fasiolo, M.; Pya, N.; Wood, SN, A comparison of inferential methods for highly nonlinear state space models in ecology and epidemiology, Stat. Sci., 31, 1, 96-118 (2016) · Zbl 1442.62349
[17] Fearnhead, P.; Prangle, D., Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation, J. R. Stat. Soc. Ser. B Stat. Methodol., 74, 3, 419-474 (2012) · Zbl 1411.62057
[18] Frazier, DT; Maneesoonthorn, W.; Martin, GM; McCabe, BPM, Approximate Bayesian forecasting, Int. J. Forecast., 35, 2, 521-539 (2019)
[19] Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian Data Analysis. Chapman & Hall/CRC Texts in Statistical Science, third edition (2013)
[20] Gelman, A.; Meng, X-L; Stern, H., Posterior predictive assessment of model fitness via realized discrepancies, Stat. Sin., 6, 4, 733-760 (1996) · Zbl 0859.62028
[21] Gillespie, DT, Exact stochastic simulation of coupled chemical reactions, J. Phys. Chem., 81, 25, 2340-2361 (1977)
[22] Golightly, A.; Sherlock, C., Efficient sampling of conditioned Markov jump processes, Stat. Comput., 29, 5, 1149-1163 (2019) · Zbl 1430.62032
[23] Golightly, A.; Wilkinson, DJ, Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo, Interface Focus, 1, 6, 807-820 (2011)
[24] Grazian, C.; Fan, Y., A review of Approximate Bayesian Computation methods via density estimation: inference for simulator-models, WIREs Comput. Stat., 12, 4 (2020) · Zbl 07909794
[25] Hainy, M., Drovandi, C.C., McGree, J.M.:Likelihood-free extensions for Bayesian sequentially designed experiments. In Kunert, J., Müller, C.H., Atkinson, A.C., editors, mODa 11: Advances in Model-Oriented Design and Analysis, 153-161 (2016)
[26] Heggland, K.; Frigessi, A., Estimating functions in indirect inference, J. R. Stat. Soc. Ser. B, 66, 2, 447-462 (2004) · Zbl 1062.62098
[27] Jasra, A., Approximate Bayesian computation for a class of time series models, Int. Stat. Rev., 83, 3, 405-435 (2015) · Zbl 07763454
[28] Jasra, A.; Singh, S.; Martin, J.; McCoy, E., Filtering via approximate Bayesian computation, Stat. Comput., 22, 1223-1237 (2012) · Zbl 1252.62093
[29] Jiang, B., Wu, T.-W., Wong, W.: Approximate Bayesian computation with Kullback-Leibler divergence as data discrepancy. In: Proceedings of the Twenty-First international conference on artificial intelligence and statistics, 1711-1721 (2018)
[30] Järvenpää, M.; Sater, MRA; Lagoudas, GK; Blainey, PC; Miller, LG; McKinnell, JA; Huang, SS; Grad, YH; Marttinen, P., A Bayesian model of acquisition and clearance of bacterial colonization incorporating within-host variation, PLoS Comput. Biol., 15, 4, 1-25 (2019)
[31] Järvenpää, M., Vehtari, A., Marttinen, P.: Batch simulations and uncertainty quantification in Gaussian process surrogate approximate Bayesian computation. In: Proceedings of the 36th conference on uncertainty in artificial intelligence (UAI), 779-788 (2020)
[32] Kleinegesse, S.; Drovandi, C.; Gutmann, MU, Sequential Bayesian experimental design for implicit models via mutual information, Bayesian Anal., 16, 3, 773-802 (2021) · Zbl 07807541
[33] Krüger, F.; Lerch, S.; Thorarinsdottir, T.; Gneiting, T., Predictive inference based on Markov chain Monte Carlo output, Int. Stat. Rev., 89, 2, 274-301 (2021) · Zbl 07777543
[34] Kypraios, T.; Neal, P.; Prangle, D., A tutorial introduction to Bayesian inference for stochastic epidemic models using approximate Bayesian computation, Math. Biosci., 287, 42-53 (2017) · Zbl 1377.92091
[35] Lauritzen, SL, Sufficiency, prediction and extreme models, Scand. J. Stat., 1, 3, 128-134 (1974) · Zbl 0297.62068
[36] Lewis, JR; MacEachern, SN; Lee, Y., Bayesian restricted likelihood methods: conditioning on insufficient statistics in Bayesian regression (with discussion), Bayesian Anal., 16, 4, 1393-2854 (2021) · Zbl 07808156
[37] Lintusaari, J.; Gutmann, MU; Dutta, R.; Kaski, S.; Corander, J., Fundamentals and recent developments in approximate Bayesian computation, Syst. Biol., 66, 1, e66-e82 (2017)
[38] Marin, JM; Pudlo, P.; Robert, CP; Ryder, RJ, Approximate Bayesian computational methods, Stat. Comput., 22, 6, 1167-1180 (2012) · Zbl 1252.62022
[39] Marjoram, P.; Molitor, J.; Plagnol, V.; Tavare, S., Markov chain Monte Carlo without likelihoods, Proc. Natl. Acad. Sci. USA, 100, 26, 15324-8 (2003)
[40] Martin, GM; McCabe, BPM; Frazier, DT; Maneesoonthorn, W.; Robert, CP, Auxiliary likelihood-based approximate Bayesian computation in state space models, J. Comput. Graph. Stat., 28, 3, 508-522 (2019) · Zbl 07499073
[41] Martin, JS; Jasra, A.; Singh, SS; Whiteley, N.; Del Moral, P.; McCoy, E., Approximate Bayesian computation for smoothing, Stoch. Anal. Appl., 32, 3, 397-420 (2014) · Zbl 1429.62368
[42] McKinley, T.; Cook, AR; Deardon, R., Inference in epidemic models without likelihoods, Int. J. Biostat. (2009) · doi:10.2202/1557-4679.1171/html
[43] Numminen, E.; Cheng, L.; Gyllenberg, M.; Corander, J., Estimating the transmission dynamics of streptococcus pneumoniae from strain prevalence data, Biometrics, 69, 3, 748-757 (2013) · Zbl 1418.92186
[44] O’Hagan, A., Forster, J.: Advanced Theory of Statistics. Arnold, London, UK, second edition, Bayesian inference (2004) · Zbl 1058.62002
[45] Papamakarios, G., Murray, I.: Fast e-free inference of simulation models with Bayesian conditional density estimation. In Advances in Neural Information Processing Systems 29 (2016)
[46] Papamakarios, G., Sterratt, D., Murray, I.: Sequential neural likelihood: fast likelihood-free inference with autoregressive flows. In: Proceedings of the 22nd international conference on artificial intelligence and statistics, 837-848 (2019)
[47] Pesonen, H., Simola, U., Köhn-Luque, A., Vuollekoski, H., Lai, X., Frigessi, A., Kaski, S., Frazier, D.T., Maneesoonthorn, W., Martin, G. M., Corander, J.: ABC of the Future. Int. Stat. Rev. (2022). doi:10.1111/insr.12522 · Zbl 07779067
[48] Picchini, U., Inference for SDE models via approximate Bayesian computation, J. Comput. Graph. Stat., 23, 4, 1080-1100 (2014)
[49] Prangle, D., Adapting the ABC distance function, Bayesian Anal., 12, 1, 289-309 (2017) · Zbl 1384.62098
[50] Price, LF; Drovandi, CC; Lee, A.; Nott, DJ, Bayesian synthetic likelihood, J. Comput. Graph. Stat., 27, 1, 1-11 (2018) · Zbl 07498962
[51] Pritchard, JK; Seielstad, MT; Perez-Lezaun, A.; Feldman, MW, Population growth of human Y chromosomes: a study of Y chromosome microsatellites, Mol. Biol. Evol., 16, 12, 1791-1798 (1999)
[52] Ratmann, O.; Andrieu, C.; Wiuf, C.; Richardson, S., Model criticism based on likelihood-free inference, with an application to protein network evolution, Proc. Natl. Acad. Sci. USA, 106, 26, 10576-10581 (2009)
[53] Schervish, MJ, Theory of Statistics (1995), Berlin: Springer, Berlin · Zbl 0834.62002
[54] Shestopaloff, A.Y., Neil, R.M.: On Bayesian inference for the M/G/1 queue with efficient MCMC sampling. Available at https://arxiv.org/abs/1401.5548 (2014)
[55] Simola, U.; Cisewski-Kehe, J.; Gutmann, MU; Corander, J., Adaptive approximate Bayesian computation tolerance selection, Bayesian Anal., 16, 2, 397-423 (2021) · Zbl 1480.62165
[56] Sisson, S.; Fan, Y.; Beaumont, M., Handbook of Approximate Bayesian Computation (2019), New York: Chapman and Hall/CRC, New York · Zbl 1416.62005
[57] Sisson, SA; Fan, Y.; Tanaka, MM, Sequential Monte Carlo without likelihoods, Proc. Natl. Acad. Sci. USA, 104, 6, 1760-5 (2007) · Zbl 1160.65005
[58] Skibinsky, M., Adequate subfields and sufficiency, Ann. Math. Stat., 38, 1, 155-161 (1967) · Zbl 0155.25701
[59] Stein, EM; Shakarchi, R., Real Analysis: Measure Theory, Integration, and Hilbert Spaces (2005), Princeton: Princeton University Press, Princeton · Zbl 1081.28001
[60] Tancredi, A., Approximate Bayesian inference for discretely observed continuous-time multi-state models, Biometrics, 75, 3, 966-977 (2019) · Zbl 1436.62641
[61] Tavaré, S.; Balding, DJ; Griffiths, RC; Donnelly, P., Inferring coalescence times from DNA sequence data, Genetics, 145, 2, 505-518 (1997)
[62] Thomas, O.; Dutta, R.; Corander, J.; Kaski, S.; Gutmann, MU, Likelihood-free inference by ratio estimation, Bayesian Anal., 17, 1, 1-31 (2022) · Zbl 07809841
[63] Toni, T.; Welch, D.; Strelkowa, N.; Ipsen, A.; Stumpf, MPH, Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems, J. R. Soc. Interface, 6, 31, 187-202 (2009)
[64] Vankov, E.R., Guindani, M., Ensor, K.B.: Filtering and estimation for a class of stochastic volatility models with intractable likelihoods. Bayesian Anal. 14(1), 29-52 (2019) · Zbl 1409.62184
[65] Vehtari, A., Ojanen, J.: A survey of Bayesian predictive methods for model assessment, selection and comparison. Stat. Surveys 6, 142-228 (2012) · Zbl 1302.62011
[66] Warne, DJ; Baker, RE; Simpson, MJ, Simulation and inference algorithms for stochastic biochemical reaction networks: from basic concepts to state-of-the-art, J. R. Soc. Interface, 16, 151, 20180943 (2019)
[67] Warne, DJ; Baker, RE; Simpson, MJ, A practical guide to pseudo-marginal methods for computational inference in systems biology, J. Theor. Biol., 496, 110255 (2020)
[68] Wilkinson, DJ, Stochastic Modelling for Systems Biology (2019), Boca Raton: Chapman & Hall/CRC, Boca Raton · Zbl 1403.92003
[69] Wilkinson, RD, Approximate Bayesian computation (ABC) gives exact results under the assumption of model error, Stat. Appl. Genet. Mol. Biol., 12, 2, 129-141 (2013)
[70] Wood, SN, Statistical inference for noisy nonlinear ecological dynamic systems, Nature, 466, 1102-1104 (2010)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.