Simulation-based optimization of radiotherapy: agent-based modeling and reinforcement learning. (English) Zbl 1540.92067
Summary: Along with surgery and chemotherapy, radiotherapy is an effective way to treat cancer. Many cancer patients take delivery of radiation. The goal of radiotherapy is to destroy the tumor without damaging healthy tissue. Due to the complexity of the procedure, modeling and simulation can be useful for radiotherapy. In this research we propose a new approach to optimize dose calculation in radiotherapy. We consider fix schedule of irradiation and varying the fraction size during the treatment. The proposed approach contains two steps. At the first step, we develop an agent-based simulation of vascular tumor growth based on biological evidences. We consider a multi-scale model in which cellular and subcellular scales are observed. We consider heterogeneity of tumor oxygen diffusion and also the effects of cancer cells hypoxia on radiotherapy. Besides, different radiosensitivity of cells related to their cell-cycle phase is modeled. The agent-based model was implemented in NetLogo package. Based on this model, we simulate different scenarios of radiotherapy. At the second step, we propose an algorithm for the optimization of radiotherapy. Radiation dose and fractionation scheme are considered as two key elements of radiation therapy. To optimize the therapy we apply Q-learning algorithm. Finally, we combine the simulation and optimization compartments together using R-NetLogo package. By tuning the parameters of learning algorithm optimal treatment plans are achieved to cure tumor together with minimum side effects. Our research presents the power of agent-based approach combined with reinforcement learning for simulating and optimizing complex biological problems such as radiotherapy. The proposed modeling approach lets us to study different scenarios of tumor growth and radiotherapy. Furthermore, our optimization algorithm works fast and finds the best treatment plan.
MSC:
| 92C50 | Medical applications (general) |
| 92-08 | Computational methods for problems pertaining to biology |
References:
| [1] | Alfonso, J. C.L.; Jagiella, N.; Núñez, L.; Herrero, M. A.; Drasdo, D., Estimating dose painting effects in radiotherapy: a mathematical model, PLoS One, 9, e89380 (2014) |
| [2] | Antipas, V. P.; Stamatakos, G. S.; Uzunoglu, N. K.; Dionysiou, D. D.; Dale, R. G., A spatio-temporal simulation model of the response of solid tumours to radiotherapy in vivo: parametric validation concerning oxygen enhancement ratio and cell cycle duration, Phys. Med. Biol., 49, 1485 (2004) |
| [4] | Bonabeau, E., Agent-based modeling: Methods and techniques for simulating human systems, Proc. Natl. Acad. Sci., 99, 7280-7287 (2002) |
| [5] | Borkenstein, K.; Levegrün, S.; Peschke, P., Modeling and computer simulations of tumor growth and tumor response to radiotherapy, Radiat. Res., 162, 71-83 (2004) |
| [6] | Chan, T. C.-Y., Optimization under uncertainty in radiation therapy (2007), Harvard Medical School |
| [7] | Chen, J.; Sprouffske, K.; Huang, Q.; Maley, C. C., Solving the puzzle of metastasis: the evolution of cell migration in neoplasms, PLoS One, 6, e17933 (2011) |
| [8] | Chiacchio, F.; Pennisi, M.; Russo, G.; Motta, S.; Pappalardo, F., Agent-based modeling of the immune system: NetLogo, a promising framework, BioMed. Res. Int., 2014 (2014) |
| [9] | Deng, G.; Ferris, M. C., Neuro-dynamic programming for fractionated radiotherapy planning, (Optimization in Medicine (2008), Springer), 47-70 |
| [10] | Dionysiou, D. D.; Stamatakos, G. S.; Uzunoglu, N. K.; Nikita, K. S., A computer simulation of in vivo tumour growth and response to radiotherapy: New algorithms and parametric results, Comput. Biol. Med., 36, 448-464 (2006) |
| [11] | Enderling, H.; Anderson, A. R.; Chaplain, M. A., A model of breast carcinogenesis and recurrence after radiotherapy, Proc. Appl. Math. Mech., 7, 1121701-1121702 (2007) |
| [12] | Enderling, H.; Chaplain, M. A.; Hahnfeldt, P., Quantitative modeling of tumor dynamics and radiotherapy, Acta Biotheor., 58, 341-353 (2010) |
| [13] | Enderling, H.; Park, D.; Hlatky, L.; Hahnfeldt, P., The importance of spatial distribution of stemness and proliferation state in determining tumor radioresponse, Math. Model. Nat. Phenom., 4, 117-133 (2009) · Zbl 1165.92022 |
| [14] | Gao, X.; McDonald, J. T.; Hlatky, L.; Enderling, H., Acute and fractionated irradiation differentially modulate glioma stem cell division kinetics, Cancer Res., 73, 1481-1490 (2013) |
| [15] | Ghate, A., Dynamic optimization in radiotherapy, Tutor. Oper. Res., 1-14 (2011) |
| [16] | Harada, H., How can we overcome tumor hypoxia in radiation therapy?, J. Radiat. Res., 52, 545-556 (2011) |
| [17] | Harting, C.; Peschke, P.; Borkenstein, K.; Karger, C. P., Single-cell-based computer simulation of the oxygen-dependent tumour response to irradiation, Phys. Med. Biol., 52, 4775 (2007) |
| [18] | Humphrey, T. C.; Brooks, G., Cell Cycle Control: Mechanisms and Protocols (2005), Springer Science & Business Media |
| [19] | Jiménez, R. P.; Hernandez, E. O., Tumour-host dynamics under radiotherapy, Chaos Solitons Fractals, 44, 685-692 (2011) · Zbl 1402.92253 |
| [20] | Kempf, H.; Hatzikirou, H.; Bleicher, M.; Meyer-Hermann, M., In silico analysis of cell cycle synchronisation effects in radiotherapy of tumour spheroids, PLoS Comput. Biol., 9, e1003295 (2013) |
| [21] | Kim, M.; Ghate, A.; Phillips, M., A Markov decision process approach to temporal modulation of dose fractions in radiation therapy planning, Phys. Med. Biol., 54, 4455-4476 (2009) |
| [22] | Kirkby, N.; Burnet, N.; Faraday, D., Mathematical modelling of the response of tumour cells to radiotherapy, Nucl. Instrum. Methods Phys. Res., 188, 210-215 (2002) |
| [23] | Kufe, D. W.; Pollock, R. E.; Weichselbaum, R. R.; Bast, R. C.; Gansler, T. S.; Holland, J. F.; Frei, E.; Folkman, J.; Kalluri, R., Beginning of angiogenesis research, (Holland-Frei Cancer Medicine (2003)) |
| [24] | Leder, K.; Pitter, K.; LaPlant, Q.; Hambardzumyan, D.; Ross, B. D.; Chan, T. A.; Holland, E. C.; Michor, F., Mathematical modeling of PDGF-driven glioblastoma reveals optimized radiation dosing schedules, Cell, 156, 603-616 (2014) |
| [25] | Lim, J., Optimization in Radiation Treatment Planning (2002), UNIVERSITY OF WISCONSIN-MADISON |
| [26] | Moodie, E. E.; Chakraborty, B.; Kramer, M. S., Q-learning for estimating optimal dynamic treatment rules from observational data, Canad. J. Statist., 40, 629-645 (2012) · Zbl 1349.62371 |
| [27] | Motta, S.; Pappalardo, F., Mathematical modeling of biological systems, Brief. Bioinform., bbs061 (2012) |
| [28] | Murphy, S. A., Optimal dynamic treatment regimes, J. R. Stat. Soc. Ser. B., 65, 331-355 (2003) · Zbl 1065.62006 |
| [29] | O’Neil, N., An Agent Based Model of Tumor Growth and Response to Radiotherapy, Mathematical Sciences (2012), Virginia Commonwealth University |
| [30] | Orth, M.; Lauber, K.; Niyazi, M.; Friedl, A. A.; Li, M.; Maihöfer, C.; Schüttrumpf, L.; Ernst, A.; Niemöller, O. M.; Belka, C., Current concepts in clinical radiation oncology, Radiat. Environ. Biophys., 53, 1-29 (2014) |
| [31] | Pappalardo, F.; Chiacchio, F.; Motta, S., Cancer vaccines: state of the art of the computational modeling approaches, BioMed. Res. Int., 2013 (2012) |
| [32] | Powathil, G. G.; Adamson, D. J.; Chaplain, M. A., Towards predicting the response of a solid tumour to chemotherapy and radiotherapy treatments: clinical insights from a computational model, PLoS Comput. Biol., 9, e1003120 (2013) |
| [33] | Powathil, G. G.; Gordon, K. E.; Hill, L. A.; Chaplain, M. A., Modelling the effects of cell-cycle heterogeneity on the response of a solid tumour to chemotherapy: biological insights from a hybrid multiscale cellular automaton model, J. Theoret. Biol., 308, 1-19 (2012) · Zbl 1411.92146 |
| [34] | Ramakrishnan, J., Dynamic Optimization of Fractionation Schedules in Radiation Therapy (2013), Harvard Medical School |
| [36] | Stamatakos, G. S.; Dionysiou, D. D.; Zacharaki, E.; Mouravliansky, N.; Nikita, K. S.; Uzunoglu, N. K., In silico radiation oncology: combining novel simulation algorithms with current visualization techniques, Proc. IEEE, 90, 1764-1777 (2002) |
| [37] | Stamatakos, G. S.; Dionysiou, D. D.; Zacharaki, E. I.; Mouravliansky, N. A.; Nikita, K. S.; Uzunoglu, N. K., In silico radiation oncology: combining novel simulation algorithms with current visualization techniques, Proc. IEEE, 90, 1764-1777 (2002) |
| [38] | Sutton, R. S.; Barto, A. G., Reinforcement Learning: An Introduction (1998), MIT press: MIT press Cambridge |
| [39] | Thames, H. D.; Hendry, J. H., Fractionation in Radiotherapy (1987), Taylor and Franc’s: Taylor and Franc’s London |
| [40] | Thiele, J., R marries NetLogo: introduction to the RNetLogo package, J. Stat., 58, 1-41 (2014) |
| [42] | Van der Kogel, A.; Joiner, M., Basic Clinical Radiobiology (2009), Hodder Arnold Publ. |
| [43] | Vaupel, P.; Kallinowski, F.; Okunieff, P., Blood Flow, Oxygen Consumption and Tissue Oxygenation of Human Tumors, Oxygen Transport to Tissue XII, 895-905 (1990), Springer |
| [44] | Wein, L. M.; Cohen, J. E.; Wu, J. T., Dynamic optimization of a linear-quadratic model with incomplete repair and volume-dependent sensitivity and repopulation, Int. J. Radiat. Oncol.* Biol.* Phys., 47, 1073-1083 (2000) |
| [45] | Zhao, Y.; Kosorok, M. R.; Zeng, D., Reinforcement learning design for cancer clinical trials, Stat. Med., 28, 3294 (2009) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
