Adaptive semiparametric Bayesian differential equations via sequential Monte Carlo. (English) Zbl 07547636
Summary: Nonlinear differential equations (DEs) are used in a wide range of scientific problems to model complex dynamic systems. The differential equations often contain unknown parameters that are of scientific interest, which have to be estimated from noisy measurements of the dynamic system. Generally, there is no closed-form solution for nonlinear DEs, and the likelihood surface for the parameter of interest is multi-modal and very sensitive to different parameter values. We propose a Bayesian framework for nonlinear DE systems. A flexible nonparametric function is used to represent the dynamic process such that expensive numerical solvers can be avoided. A sequential Monte Carlo algorithm in the annealing framework is proposed to conduct Bayesian inference for parameters in DEs. In our numerical experiments, we use examples of ordinary differential equations and delay differential equations to demonstrate the effectiveness of the proposed algorithm. We developed an R package that is available at https://github.com/shijiaw/smcDE. Supplementary files for this article are available online.
MSC:
| 62-XX | Statistics |
Keywords:
Bayesian smoothing; B-spline; conditional effective sample size; delay differential equation; ordinary differential equationReferences:
| [1] | Ascher, U. M.; Ruuth, S. J.; Spiteri, R. J., Implicit-Explicit Runge-Kutta Methods for Time-Dependent Partial Differential Equations, Applied Numerical Mathematics, 25, 151-167 (1997) · Zbl 0896.65061 |
| [2] | Barber, D.; Wang, Y., Gaussian Processes for Bayesian Estimation in Ordinary Differential Equations, 1485-1493 (2014) |
| [3] | Berezansky, L.; Braverman, E.; Idels, L., “Nicholson’s Blowflies Differential Equations Revisited: Main Results and Open Problems, Applied Mathematical Modelling, 34, 1405-1417 (2010) · Zbl 1193.34149 · doi:10.1016/j.apm.2009.08.027 |
| [4] | Berry, S. M.; Carroll, R. J.; Ruppert, D., “Bayesian Smoothing and Regression Splines for Measurement Error Problems, Journal of the American Statistical Association, 97, 160-169 (2002) · Zbl 1073.62524 · doi:10.1198/016214502753479301 |
| [5] | Bhaumik, P.; Ghosal, S., “Bayesian Two-Step Estimation In Differential Equation Models, Electronic Journal of Statistics, 9, 3124-3154 (2015) · Zbl 1330.62273 · doi:10.1214/15-EJS1099 |
| [6] | Bhaumik, P.; Ghosal, S., “Efficient Bayesian Estimation and Uncertainty Quantification in Ordinary Differential Equation Models, Bernoulli, 23, 3537-3570 (2017) · Zbl 1459.62048 · doi:10.3150/16-BEJ856 |
| [7] | Bulirsch, R.; Stoer, J., “Numerical Treatment of Ordinary Differential Equations by Extrapolation Methods, Numerische Mathematik, 8, 1-13 (1966) · Zbl 0135.37901 · doi:10.1007/BF02165234 |
| [8] | Burden, R. L.; Faires, J. D.; Reynolds, A. C., Numerical Analysis (2011), Boston, MA: Brooks/Cole, Cengage Learning, Boston, MA |
| [9] | Butcher, J. C., Numerical Methods for Ordinary Differential Equations (2016), Chichester, West Sussex: Wiley, Chichester, West Sussex · Zbl 1354.65004 |
| [10] | Calderhead, B.; Girolami, M.; Lawrence, N. D.; Bengio, Yoshua; Schuurmans, Dale; Lafferty, John; Williams, Chris; Culotta, Aron, Advances in Neural Information Processing Systems, Accelerating Bayesian Inference Over Nonlinear Differential Equations With Gaussian Processes, 217-224 (2009), Vancouver, BC: Curran Associates, Inc, Vancouver, BC |
| [11] | Campbell, D.; Steele, R. J., “Smooth Functional Tempering for Nonlinear Differential Equation Models, Statistics and Computing, 22, 429-443 (2012) · Zbl 1322.62011 · doi:10.1007/s11222-011-9234-3 |
| [12] | Cao, J.; Huang, J. Z.; Wu, H., “Penalized Nonlinear Least Squares Estimation of Time-Varying Parameters in Ordinary Differential Equations, Journal of Computational and Graphical Statistics, 21, 42-56 (2012) · doi:10.1198/jcgs.2011.10021 |
| [13] | Cao, J.; Wang, L.; Xu, J., “Robust Estimation for Ordinary Differential Equation Models, Biometrics, 67, 1305-1313 (2011) · Zbl 1274.62739 · doi:10.1111/j.1541-0420.2011.01577.x |
| [14] | Carpenter, J.; Clifford, P.; Fearnhead, P., “Improved Particle Filter for Nonlinear Problems, IEE Proceedings-Radar, Sonar and Navigation, 146, 2-7 (1999) · doi:10.1049/ip-rsn:19990255 |
| [15] | Chen, J.; Wu, H., “Efficient Local Estimation for Time-Varying Coefficients in Deterministic Dynamic Models With Applications to HIV-1 Dynamics, Journal of the American Statistical Association, 103, 369-384 (2008) · Zbl 1469.62365 · doi:10.1198/016214507000001382 |
| [16] | Chopin, N., “Central limit theorem for sequential Monte Carlo methods and Its Application to Bayesian Inference, The Annals of Statistics, 32, 2385-2411 (2004) · Zbl 1079.65006 · doi:10.1214/009053604000000698 |
| [17] | Dass, S. C.; Lee, J.; Lee, K.; Park, J., “Laplace Based Approximate Posterior Inference for Differential Equation Models, Statistics and Computing, 27, 679-698 (2017) · Zbl 1505.62118 · doi:10.1007/s11222-016-9647-0 |
| [18] | De Boor, C., “On Calculating With B-Splines, Journal of Approximation Theory, 6, 50-62 (1972) · Zbl 0239.41006 · doi:10.1016/0021-9045(72)90080-9 |
| [19] | Del Moral, P., Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications (2004), New York: Springer, New York · Zbl 1130.60003 |
| [20] | Del Moral, P.; Doucet, A.; Jasra, A., “Sequential Monte Carlo Samplers, Journal of the Royal Statistical Society, Series B, 68, 411-436 (2006) · Zbl 1105.62034 · doi:10.1111/j.1467-9868.2006.00553.x |
| [21] | Del Moral, P.; Doucet, A.; Jasra, A., “An Adaptive Sequential Monte Carlo Method for Approximate Bayesian Computation, Statistics and Computing, 22, 1009-1020 (2012) · Zbl 1252.65025 · doi:10.1007/s11222-011-9271-y |
| [22] | Dondelinger, F.; Husmeier, D.; Rogers, S.; Filippone, M.; Carvalho, C. M.; Ravikumar, P., Artificial Intelligence and Statistics, “ODE Parameter Inference Using Adaptive Gradient Matching With Gaussian Processes,”, 216-228 (2013), Scottsdale, Arizona: PMLR, Scottsdale, Arizona |
| [23] | Douc, R.; Cappé, O., Comparison of Resampling Schemes for Particle Filtering, Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005 (ISPA 2005), 64-69 (2005) · doi:10.1109/ISPA.2005.195385 |
| [24] | Doucet, A.; De Freitas, N.; Gordon, N.; Doucet, A.; de Freitas, N.; Gordon, N., Sequential Monte Carlo Methods in Practice, “An Introduction to Sequential Monte Carlo Methods,”, 3-14 (2001), New York, NY: Springer, New York, NY · Zbl 1056.93576 |
| [25] | Doucet, A.; Godsill, S.; Andrieu, C., “On Sequential Monte Carlo Sampling Methods for Bayesian Filtering,”, Statistics and Computing, 10, 197-208 (2000) |
| [26] | Fan, Y.; Wu, R.; Chen, M.-H.; Kuo, L.; Lewis, P. O., “Choosing Among Partition Models in Bayesian Phylogenetics,”, Molecular Biology and Evolution, 28, 523-532 (2011) |
| [27] | Hochbruck, M.; Lubich, C.; Selhofer, H., “Exponential Integrators for Large Systems of Differential Equations,”, SIAM Journal on Scientific Computing, 19, 1552-1574 (1998) · Zbl 0912.65058 |
| [28] | Hochbruck, M.; Ostermann, A., “Exponential Integrators,”, Acta Numerica, 19, 209-286 (2010) · Zbl 1242.65109 |
| [29] | Hol, J. D.; Schon, T. B.; Gustafsson, F., Nonlinear Statistical Signal Processing Workshop, 2006 IEEE, On Resampling Algorithms for Particle Filters, 79-82 (2006), IEEE |
| [30] | Jain, M. K., Numerical Solution of Differential Equations (1979), Eastern New Delhi: Wiley, Eastern New Delhi · Zbl 0409.65002 |
| [31] | Jameson, A.; Schmidt, W.; Turkel, E., Numerical Solution of the Euler Equations by Finite Volume Methods Using Runge Kutta Time Stepping Schemes, 14th Fluid and Plasma Dynamics Conference, 1259 (1981) |
| [32] | Kitagawa, G., “Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models, Journal of Computational and Graphical Statistics, 5, 1-25 (1996) |
| [33] | Lee, K.; Lee, J.; Dass, S. C., “Inference for Differential Equation Models Using Relaxation Via Dynamical Systems,”, Computational Statistics & Data Analysis, 127, 116-134 (2018) · Zbl 1469.62094 |
| [34] | Liu, J. S.; Chen, R., “Sequential Monte Carlo Methods for Dynamic Systems, Journal of the American Statistical Association, 93, 1032-1044 (1998) · Zbl 1064.65500 |
| [35] | May, R. M.; May, R. M., Theoretical Ecology: Principles and Applications, Models for Single Populations, 4-25 (1976), Philadelphia Toronto: W.B. Saunders Company, Philadelphia Toronto |
| [36] | Monk, N. A., “Oscillatory Expression of Hes1, p53, and NF-κB Driven by Transcriptional Time Delays, Current Biology, 13, 1409-1413 (2003) |
| [37] | Neal, R. M., “Annealed Importance Sampling, Statistics and Computing, 11, 125-139 (2001) |
| [38] | Nicholson, A. J., “An Outline of the Dynamics of Animal Populations, Australian Journal of Zoology, 2, 9-65 (1954) |
| [39] | Pang, T.; Yan, P.; Zhou, H. H., “Asymptotically Efficient Parameter Estimation for Ordinary Differential Equations,”, Science China Mathematics, 60, 2263-2286 (2017) · Zbl 1392.62158 |
| [40] | Poyton, A.; Varziri, M. S.; McAuley, K. B.; McLellan, P.; Ramsay, J. O., “Parameter Estimation in Continuous-Time Dynamic Models Using Principal Differential Analysis,”, Computers & Chemical Engineering, 30, 698-708 (2006) |
| [41] | Qi, X.; Zhao, H., “Asymptotic Efficiency and Finite-Sample Properties of the Generalized Profiling Estimation of Parameters in Ordinary Differential Equations, The Annals of Statistics, 38, 435-481 (2010) · Zbl 1181.62156 |
| [42] | Ramsay, J. O.; Kotz, S.; Read, C. B.; Balakrishnan, N.; Vidakovic, B.; Johnson, N. L., Encyclopedia of Statistical Sciences, 4, Functional Data Analysis, 1-8 (2006) · doi:10.1002/0471667196.ess3138 |
| [43] | Ramsay, J. O.; Hooker, G.; Campbell, D.; Cao, J., “Parameter Estimation for Differential Equations: A Generalized Smoothing Approach,”, Journal of the Royal Statistical Society, Series B, 69, 741-796 (2007) · Zbl 07555374 |
| [44] | Ramsay, J. O.; Silverman, B. W., Applied Functional Data Analysis: Methods and Case Studies (2007), New York: Springer, New York · Zbl 1011.62002 |
| [45] | Reiss, P. T.; Todd Ogden, R., “Smoothing Parameter Selection for a Class of Semiparametric Linear Models, Journal of the Royal Statistical Society, Series B, 71, 505-523 (2009) · Zbl 1248.62057 |
| [46] | Rosenzweig, M. L.; MacArthur, R. H., “Graphical Representation and Stability Conditions of Predator-Prey Interactions,”, The American Naturalist, 97, 209-223 (1963) |
| [47] | Soetaert, K.; Petzoldt, T.; Setzer, R. W., “Solving Differential Equations in R: Package Desolve, Journal of Statistical Software, 33 (2010) |
| [48] | Varah, J. M., “A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations, SIAM Journal on Scientific and Statistical Computing, 3, 28-46 (1982) · Zbl 0481.65050 |
| [49] | Wang, L.; Cao, J., “Estimating Parameters in Delay Differential Equation Models,”, Journal of Agricultural, Biological, and Environmental Statistics, 17, 68-83 (2012) · Zbl 1302.62285 |
| [50] | Wang, L.; Wang, S.; Bouchard-Côté, A., “An Annealed Sequential Monte Carlo Method for Bayesian Phylogenetics,”, Systematic Biology, 69, 155-183 (2020) |
| [51] | Wood, S. N., Generalized Additive Models: An Introduction With R (2017), Boca Raton: Chapman and Hall/CRC, Boca Raton · Zbl 1368.62004 |
| [52] | Zhang, T.; Yin, Q.; Caffo, B.; Sun, Y.; Boatman-Reich, D., “Bayesian Inference of High-Dimensional, Cluster-Structured Ordinary Differential Equation Models With Applications to Brain Connectivity Studies, The Annals of Applied Statistics, 11, 868-897 (2017) · Zbl 1391.62262 |
| [53] | Zhou, Y.; Johansen, A. M.; Aston, J. A., “Toward Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach,”, Journal of Computational and Graphical Statistics, 25, 701-726 (2016) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
