Real-time mechanistic Bayesian forecasts of COVID-19 mortality. (English) Zbl 07789359
Summary: The COVID-19 pandemic emerged in late December 2019. In the first six months of the global outbreak, the U.S. reported more cases and deaths than any other country in the world. Effective modeling of the course of the pandemic can help assist with public health resource planning, intervention efforts, and vaccine clinical trials. However, building applied forecasting models presents unique challenges during a pandemic. First, case data available to models in real time represent a nonstationary fraction of the true case incidence due to changes in available diagnostic tests and test-seeking behavior. Second, interventions varied across time and geography leading to large changes in transmissibility over the course of the pandemic. We propose a mechanistic Bayesian model that builds upon the classic compartmental susceptible-exposed-infected-recovered (SEIR) model to operationalize COVID-19 forecasting in real time. This framework includes nonparametric modeling of varying transmission rates, nonparametric modeling of case and death discrepancies due to testing and reporting issues, and a joint observation likelihood on new case counts and new deaths; it is implemented in a probabilistic programming language to automate the use of Bayesian reasoning for quantifying uncertainty in probabilistic forecasts. The model has been used to submit forecasts to the U.S. Centers for Disease Control through the COVID-19 Forecast Hub under the name MechBayes. We examine the performance relative to a baseline model as well as alternate models submitted to the forecast hub. Additionally, we include an ablation test of our extensions to the classic SEIR model. We demonstrate a significant gain in both point and probabilistic forecast scoring measures using MechBayes, when compared to a baseline model, and show that MechBayes ranks as one of the top two models out of nine which regularly submitted to the COVID-19 Forecast Hub for the duration of the pandemic, trailing only the COVID-19 Forecast Hub ensemble model of which which MechBayes is a part.
MSC:
| 62Pxx | Applications of statistics |
References:
| [1] | Abbott, S., Hellewell, J., Thompson, R. N., Sherratt, K., Gibbs, H. P., Bosse, N. I., Munday, J. D., Meakin, S., Doughty, E. L. et al. (2020). Estimating the time-varying reproduction number of SARS-CoV-2 using national and subnational case counts. Wellcome Open Res.5 112. |
| [2] | Bertozzi, A. L., Franco, E., Mohler, G., Short, M. B. and Sledge, D. (2020). The challenges of modeling and forecasting the spread of COVID-19. Proc. Natl. Acad. Sci. USA117 16732-16738. doi:10.1073/pnas.2006520117 · Zbl 1485.92115 |
| [3] | Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. Preprint. Available at arXiv:1701.02434. |
| [4] | Bokler, B. (1993). Chaos and complexity in measles models: A comparative numerical study. Math. Med. Biol.10 83-95. · Zbl 0780.92021 |
| [5] | Borchering, R. K., Viboud, C., Howerton, E., Smith, C. P., Truelove, S., Runge, M. C., Reich, N. G., Contamin, L., Levander, J. et al. (2021). Modeling of future COVID-19 cases, hospitalizations, and deaths, by vaccination rates and nonpharmaceutical intervention scenarios—United States, April-September 2021. Morb. Mort. Wkly. Rep.70 719. |
| [6] | Bradbury, J., Frostig, R., Hawkins, P., Leary, C., Maclaurin, D., Necula, G., Paszke, A., Vanderplas, J., Wanderman-milne, S. and Zhang, Q. (2018). JAX: Composable transformations of Python+NumPy programs. Available at http://github.com/google/jax. |
| [7] | Cameron, C. A. and Trivedi, P. K. (1986). Econometric models based on count data. Comparisons and applications of some estimators and tests. J. Appl. Econometrics1 29-53. |
| [8] | Catching, A., Capponi, S., Yeh, M. T., Bianco, S. and Andino, R. (2021). Examining the interplay between face mask usage, asymptomatic transmission, and social distancing on the spread of COVID-19. Sci. Rep.11 1-11. |
| [9] | Chen, R. T., Rubanova, Y., Bettencourt, J. and Duvenaud, D. K. (2018). Neural ordinary differential equations. In Advances in Neural Information Processing Systems 6571-6583. |
| [10] | Cramer, E. Y., Ray, E. L., Lopez, V. K., Bracher, J., Brennen, A., Rivadeneira, A. J. C., Gerding, A., Gneiting, T., House, K. H. et al. (2021a). Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US. medRxiv. |
| [11] | Cramer, E. Y., Huang, Y., Wang, Y., Ray, E. L., Cornell, M., Bracher, J., Brennen, A., Rivadeneira, A. J. C., Gerding, A. et al. (2021b). The United States COVID-19 forecast hub dataset. medRxiv. |
| [12] | Dong, E., Du, H. and Gardner, L. (2020). An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect. Dis.20 533-534. |
| [13] | Dormand, J. R. and Prince, P. J. (1980). A family of embedded Runge-Kutta formulae. J. Comput. Appl. Math.6 19-26. doi:10.1016/0771-050X(80)90013-3 · Zbl 0448.65045 |
| [14] | Dukic, V., Lopes, H. F. and Polson, N. G. (2012). Tracking epidemics with Google Flu Trends data and a state-space SEIR model. J. Amer. Statist. Assoc.107 1410-1426. doi:10.1080/01621459.2012.713876 · Zbl 1258.62102 |
| [15] | Flaxman, S., Mishra, S., Gandy, A., Unwin, H. J. T., Mellan, T. A., Coupland, H., Whittaker, C., Zhu, H., Berah, T. et al. (2020). Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature584 257-261. |
| [16] | Frasso, G. and Lambert, P. (2016). Bayesian inference in an extended SEIR model with nonparametric disease transmission rate: An application to the Ebola epidemic in Sierra Leone. Biostatistics17 779-792. doi:10.1093/biostatistics/kxw027 |
| [17] | Gibson, G. C., Reich, N. G. and Sheldon, D. (2023). Supplement to “Real-time mechanistic Bayesian forecasts of COVID-19 mortality.” https://doi.org/10.1214/22-AOAS1671SUPPA, https://doi.org/10.1214/22-AOAS1671SUPPB |
| [18] | Giordano, G., Blanchini, F., Bruno, R., Colaneri, P., Di Filippo, A., Di Matteo, A. and Colaneri, M. (2020). Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy. Nat. Med.26 855-860. |
| [19] | Gneiting, T., Balabdaoui, F. and Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. Ser. B. Stat. Methodol.69 243-268. doi:10.1111/j.1467-9868.2007.00587.x · Zbl 1120.62074 |
| [20] | Grinsztajn, L., Semenova, E., Margossian, C. C. and Riou, J. (2021). Bayesian workflow for disease transmission modeling in Stan. Stat. Med.40 6209-6234. doi:10.1002/sim.9164 |
| [21] | Hall, I., Gani, R., Hughes, H. and Leach, S. (2007). Real-time epidemic forecasting for pandemic influenza. Epidemiol. Infect.135 372-385. |
| [22] | Hoffman, M. D. and Gelman, A. (2014). The no-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res.15 1593-1623. · Zbl 1319.60150 |
| [23] | Hotta, L. K. (2010). Bayesian melding estimation of a stochastic SEIR model. Math. Popul. Stud.17 101-111. doi:10.1080/08898481003689528 · Zbl 1408.92031 |
| [24] | Johndrow, J., Ball, P., Gargiulo, M. and Lum, K. (2020). Estimating the number of SARS-CoV-2 infections and the impact of mitigation policies in the United States. Harv. Data Sci. Rev.. |
| [25] | Karlen, D. (2020). Characterizing the spread of COVID-19. Preprint. Available at arXiv:2007.07156. |
| [26] | Kermack, W. O. and McKendrick, A. G. (1927). A contribution to the mathematical theory of epidemics. Proc. R. Soc. Lond. Ser. A, Math. Phys. Sci.115 700-721. · JFM 53.0517.01 |
| [27] | Korolev, I. (2021). Identification and estimation of the SEIRD epidemic model for COVID-19. J. Econometrics220 63-85. doi:10.1016/j.jeconom.2020.07.038 · Zbl 1464.92248 |
| [28] | Krantz, S. G. and Rao, A. S. S. (2020). Level of under-reporting including under-diagnosis before the first peak of COVID-19 in various countries: Preliminary retrospective results based on wavelets and deterministic modeling. Infect. Control Hosp. Epidemiol.41 857-859. |
| [29] | Lau, H., Khosrawipour, T., Kocbach, P., Ichii, H., Bania, J. and Khosrawipour, V. (2021). Evaluating the massive underreporting and undertesting of COVID-19 cases in multiple global epicenters. Pulmonology27 110-115. doi:10.1016/j.pulmoe.2020.05.015 |
| [30] | Lekone, P. E. and Finkenstädt, B. F. (2006). Statistical inference in a stochastic epidemic SEIR model with control intervention: Ebola as a case study. Biometrics62 1170-1177. doi:10.1111/j.1541-0420.2006.00609.x · Zbl 1114.62120 |
| [31] | López, L. and Rodo, X. (2020). A modified SEIR model to predict the COVID-19 outbreak in Spain and Italy: Simulating control scenarios and multi-scale epidemics. Available at SSRN 3576802. |
| [32] | Lutz, C. S., Huynh, M. P., Schroeder, M., Anyatonwu, S., Dahlgren, F. S., Danyluk, G., Fernandez, D., Greene, S. K., Kipshidze, N. et al. (2019). Applying infectious disease forecasting to public health: A path forward using influenza forecasting examples. BMC Public Health19 1659. |
| [33] | Mbuvha, R. and Marwala, T. (2020). Bayesian inference of COVID-19 spreading rates in South Africa. PLoS ONE15 e0237126. doi:10.1371/journal.pone.0237126 |
| [34] | MIDAS (2020). COVID-19 parameter estimates. Available at https://github.com/midas-network/COVID-19/tree/master/parameter_estimates/2019_novel_coronavirus. |
| [35] | Myers, M. F., Rogers, D., Cox, J., Flahault, A. and Hay, S. I. (2000). Forecasting disease risk for increased epidemic preparedness in public health. Adv. Parasitol.47 309-330. |
| [36] | Neal, R. M. (2011). MCMC using Hamiltonian dynamics. In Handbook of Markov Chain Monte Carlo. Chapman & Hall/CRC Handb. Mod. Stat. Methods 113-162. CRC Press, Boca Raton, FL. · Zbl 1229.65018 |
| [37] | Ong, J. B. S., Mark, I., Chen, C., Cook, A. R., Lee, H. C., Lee, V. J., Lin, R. T. P., Tambyah, P. A. and Goh, L. G. (2010). Real-time epidemic monitoring and forecasting of H1N1-2009 using influenza-like illness from general practice and family doctor clinics in Singapore. PLoS ONE5 e10036. |
| [38] | Osthus, D., Hickmann, K. S., Caragea, P. C., Higdon, D. and Del Valle, S. Y. (2017). Forecasting seasonal influenza with a state-space SIR model. Ann. Appl. Stat.11 202-224. doi:10.1214/16-AOAS1000 · Zbl 1366.62236 |
| [39] | Pei, S., Kandula, S. and Shaman, J. (2020). Differential effects of intervention timing on COVID-19 spread in the United States. medRxiv. doi:10.1101/2020.05.15.20103655 |
| [40] | Phan, D., Pradhan, N. and Jankowiak, M. (2019). Composable effects for flexible and accelerated probabilistic programming in NumPyro. Preprint. Available at arXiv:1912.11554. |
| [41] | Prem, K., Liu, Y., Russell, T. W., Kucharski, A. J., Eggo, R. M., Davies, N., Flasche, S., Clifford, S., Pearson, C. A. et al. (2020). The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: A modelling study. Lancet Public Health5 E261-E270. |
| [42] | Rahmandad, H., Lim, T. Y. and Sterman, J. (2020). Estimating COVID-19 under-reporting across 86 nations: Implications for projections and control. Available at SSRN 3635047. |
| [43] | Ray, E. L., Wattanachit, N., Niemi, J., Kanji, A. H., House, K., Cramer, E. Y., Bracher, J., Zheng, A., Yamana, T. K. et al. (2020). Ensemble forecasts of coronavirus disease 2019 (COVID-19) in the US. medRxiv. |
| [44] | Russel, T. W., Hellewell, J., Abbot, S. et al. (2020). Using a delay-adjusted case fatality ratio to estimate under-reporting. Available at the Centre for Mathematical Modelling of Infectious Diseases Repository. |
| [45] | Russell, T. W., Hellewell, J., Jarvis, C. I., Van Zandvoort, K., Abbott, S., Ratnayake, R., Flasche, S., Eggo, R. M., Edmunds, W. J. et al. (2020). Estimating the infection and case fatality ratio for coronavirus disease (COVID-19) using age-adjusted data from the outbreak on the Diamond Princess cruise ship, February 2020. Euro Surveill.25 2000256. |
| [46] | Shaman, J. and Karspeck, A. (2012). Forecasting seasonal outbreaks of influenza. Proc. Natl. Acad. Sci. USA109 20425-20430. |
| [47] | Simonov, A., Sacher, S. K., Dubé, J.-P. H. and Biswas, S. (2020). The persuasive effect of fox news: Non-compliance with social distancing during the COVID-19 pandemic. Technical report, National Bureau of Economic Research. |
| [48] | Smirnova, A.,deCamp, L. and Chowell, G. (2019). Forecasting epidemics through nonparametric estimation of time-dependent transmission rates using the SEIR model. Bull. Math. Biol.81 4343-4365. doi:10.1007/s11538-017-0284-3 · Zbl 1430.92109 |
| [49] | Syafruddin, S. and Noorani, M. (2012). SEIR model for transmission of dengue fever in Selangor Malaysia. Int. J. Mod. Phys. Conf. Ser.9 380-389. |
| [50] | Uber Labs AI (2020). “NumPyro.” Available at https://readthedocs.org/projects/numpyro/downloads/pdf/stable/. |
| [51] | Weinberger, D. M., Chen, J., Cohen, T., Crawford, F. W., Mostashari, F., Olson, D., Pitzer, V. E., Reich, N. G., Russi, M. et al. (2020). Estimation of excess deaths associated with the COVID-19 pandemic in the United States, March to May 2020. JAMA Intern. Med.180 1336-1344. |
| [52] | Yang, H. and Lee, J. (2020). Variational Bayes method for ODE parameter estimation with application to time-varying SIR model for COVID-19 epidemic. Preprint. Available at arXiv:2011.09718. |
| [53] | Yang, Z., Zeng, Z., Wang, K., Wong, S.-S., Liang, W., Zanin, M., Liu, P., Cao, X., Gao, Z. et al. (2020). Modified SEIR and AI prediction of the epidemics trend of COVID-19 in China under public health interventions. J. Thorac. Dis.12 165. |
| [54] | Zhuang, L. and Cressie, N. (2014). Bayesian hierarchical statistical SIRS models. Stat. Methods Appl.23 601-646. doi:10.1007/s10260-014-0280-9 · Zbl 1477.62324 |
| [55] | Zhuang, L., Cressie, N., Pomeroy, L. and Janies, D. (2013). Multi-species SIR models from a dynamical Bayesian perspective. Theor. Ecol.6 457-473. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
