Bayesian learning of dose-response parameters from a cohort under response-guided dosing. (English) Zbl 1374.90437
Summary: There has been a surge of clinical interest in the idea of response-guided dosing (RGD). The goal in RGD is to tailor drug-doses to the stochastic evolution of each individual patient’s disease condition over the treatment course. The hope is that this form of individualized therapy will deliver the right dose to the right patient at the right time. Several expert panels have observed that despite the excitement surrounding RGD, quantitative, data-driven decision-making approaches that learn patients’ dose-response and incorporate this information into adaptive dosing strategies are lagging behind. This situation is particularly exacerbated in clinical trials. For instance, fixed design clinical studies for estimating the key parameter of a dose-response function might not treat trial patients optimally. Similarly, the dosing strategies employed in clinical trials for RGD often appear ad-hoc.We study the problem of finding optimal RGD policies while learning the distribution of a dose-response parameter from a cohort of patients. We provide a Bayesian stochastic dynamic programming (DP) formulation of this problem. Exact solution of Bellman’s equations for this problem is computationally intractable. We therefore present two approximate control schemes and mathematically analyze the monotonicity, stationarity, and separability structures of the resulting dosing strategies. These structures are then exploited in efficient, approximate solution of our problem. Computer simulations using the Michaelis-Menten dose-response function are included as an example wherein we study the effect of cohort size and of prior misspecification.
MSC:
| 90C90 | Applications of mathematical programming |
| 90C39 | Dynamic programming |
| 90C40 | Markov and semi-Markov decision processes |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 90C25 | Convex programming |
| 62F15 | Bayesian inference |
Keywords:
OR in medicine; dynamic programming; Markov decision processes; convex optimization; Medical treatment planningSoftware:
msmReferences:
| [1] | Abramowitz, M.; Stegun, I. A., Handbook of mathematical functions with formulas, graphs, and mathematical tables (1972), Dover: Dover New York, New York, USA · Zbl 0543.33001 |
| [2] | Acikgoz, S. U.; Diwekar, U. M., Blood glucose regulation with stochastic optimal control for insulin-dependent diabetic patients, Chemical Engineering Science, 48, 3, 1227 (2010) |
| [3] | Ahuja, V.; Birge, J., Response-adaptive designs for clinical trials: Simultaneous learning from multiple patients, European Journal of Operational Research, 248, 619-633 (2016) · Zbl 1346.90528 |
| [4] | Alagoz, O.; Hsu, H.; Schaefer, A. J.; Roberts, M. S., Markov decision processes: A tool for sequential decision making under uncertainty, Medical Decision Making, 30, 4, 474-483 (2010) |
| [5] | Alagoz, O.; Maillart, L. M.; Schaefer, A. J.; Roberts, M. S., The optimal timing of living-donor liver transplantation, Management Science, 50, 10, 1420-1430 (2004) · Zbl 1232.90354 |
| [6] | Alagoz, O.; Maillart, L. M.; Schaefer, A. J.; Roberts, M. S., Determining the acceptance of cadaveric livers using an implicit model of the waiting list, Operations Research, 55, 1, 24-36 (2007) · Zbl 1167.90570 |
| [7] | Ananworanich, J.; Gayet-Ageron, A.; Braz, M. L.; Prasithsirikul, W.; Chetchotisakd, P.; Kiertiburanakul, S., CD4-guided scheduled treatment interruptions compared with continuous therapy for patients infected with HIV-1: results of the Staccato randomised trial, The Lancet, 368, 9534, 459-465 (2006) |
| [8] | Ananworanich, J.; Siangphoe, U.; Cardiello, P.; Apateerapong, W.; Hirschel, B.; Mahanontharit, A., Highly active antiretroviral therapy (HAART) retreatment in patients on CD4-guided therapy achieved similar virologic suppression compared with patients on continuous HAART - The HIV Netherlands Australia Thailand Research Collaboration 001.4 Study, Journal of Acquired Immune Deficiency Syndromes, 39, 5, 523-529 (2005) |
| [9] | Bayard, D. S.; Milman, M. H.; Schumitzky, A., Design of dosage regimens: a multiple model stochastic control approach, International Journal of Bio-Medical Computing, 36, 103-115 (1994) |
| [10] | Beil, D. R., Analysis and comparison of multimodal cancer treatments, Mathematical Medicine and Biology, 18, 4, 343-376 (2001) · Zbl 1013.92023 |
| [11] | Beil, D. R.; Wein, L. M., Sequencing surgery, radiotherapy and chemotherapy: Insights from a mathematical analysis, Breast Cancer Research and Treatment, 74, 3, 279-286 (2002) |
| [12] | Berry, D. A., Modified two-armed bandit strategies for certain clinical trials, Journal of the American Statistical Association, 73, 362, 339-345 (1978) · Zbl 0399.62107 |
| [13] | Berry, D. A., Bayesian clinical trials, Nature Reviews Drug Discovery, 5, 27-36 (2006) |
| [14] | Berry, D. A., Adaptive clinical trials: The promise and the caution, Journal of Clinical Oncology, 29, 6, 606-609 (2011) |
| [15] | Bertsekas, D. P., Dynamic programming and optimal control, vols. 1 and 2 (2007), Athena Scientific: Athena Scientific Nashua, NH |
| [16] | Bornkamp, B.; Bretz, F.; Dette, H.; Pinheiro, J., Response-adaptive dose-finding under model uncertainty, The Annals of Applied Statistics, 5, 2B, 1611-1631 (2011) · Zbl 1223.62158 |
| [17] | Boyd, S.; Vandenberghe, L., Convex optimization (2004), Cambridge University Press: Cambridge University Press Cambridge, UK · Zbl 1058.90049 |
| [18] | Burton, M. E.; Shaw, L. M.; Schentag, J. J.; Evans, W. E., Applied pharmacokinetics & pharmacodynamics: Principles of therapeutic drug monitoring (2006), Lippincott Williams & Wilkins: Lippincott Williams & Wilkins Baltimore, Maryland, USA |
| [19] | Cardiello, P. G.; Hassink, E.; Ananworanich, J.; Srasuebkul, P.; Samor, T.; Mahanontharit, A., A prospective, randomized trial of structured treatment interruption for patients with chronic HIV type 1 infection, Clinical Infectious Diseases, 40, 4, 594-600 (2005) |
| [20] | Chhatwal, J.; Alagoz, O.; Burnside, E. S., Optimal breast biopsy decision making based on mammographic features and demographic factors, Operations Research, 58, 6, 1577-1591 (2010) · Zbl 1228.90165 |
| [21] | Chow, S.-C.; Chang, M., Adaptive design methods in clinical trials—A review, Orphanet Journal of Rare Diseases, 3, 11 (2008) |
| [22] | Danel, C.; Moh, R.; Minga, A.; Anzian, A.; Ba-Gomis, O.; Kanga, C., CD4-guided structured antiretroviral treatment interruption strategy in HIV-infected adults in West Africa (Trivacan ANRS 1269 trial): A randomised trial, The Lancet, 367, 9527, 1981-1989 (2006) |
| [23] | Davis, G. L.; Wong, J. B.; McHutchison, J. G.; Manns, M. P.; Harvey, J.; Albrecht, J., Early virologic response to treatment with peginterferon alfa-2b plus ribavirin in patients with chronic hepatitis c, Hepatology, 38, 3, 645-652 (2003) |
| [24] | Demonty, I.; Ras, R. T.; van der Knaap, H. C.M.; Duchateau, G.; Meijer, L.; Zock, P. L., Continuous dose—response relationship of the LDL-cholesterol-lowering effect of phytosterol intake, Journal of Nutrition, 139, 2, 271-284 (2009) |
| [25] | den Broeder, A. A.; Creemers, M. C.W.; van Gestel, A. M.; van Riel, P. L.C. M., Dose titration using the Disease Activity Score (DAS28) in rheumatoid arthritis patients treated with antiTNF, Rheumatology, 41, 6, 638-642 (2002) |
| [26] | Denton, B. T.; Kurt, M.; Shah, N. D.; Bryant, S. C.; Smith, S. A., Optimizing the start time of statin therapy for patients with diabetes, Medical Decision Making, 29, 3, 351-367 (2009) |
| [27] | Durez, P.; den Bosch, F. V.; Corluy, L.; Veys, E. M.; Clerck, L. D.; Peretz, A., A dose adjustment in patients with rheumatoid arthritis not optimally responding to a standard dose of infliximab of 3 mg/kg every 8 weeks can be effective: A Belgian prospective study, Rheumatology, 44, 4, 465-468 (2005) |
| [28] | Eeckhoudt, L., Risk and medical decision making (2002), Kluwer Academic: Kluwer Academic Norwell, MA, USA |
| [29] | Emilien, G.; van Meurs, W.; Maloteaux, J.-M., The dose-response relationship in Phase I clinical trials and beyond: Use, meaning, and assessment, Pharmacology & Therapeutics, 88, 1, 33-58 (2000) |
| [30] | Eussen, S. R.; Rompelberg, C. J.; Klungel, O. H.; van Eijkeren, J. C., Modelling approach to simulate reductions in LDL cholesterol levels after combined intake of statins and phytosterols/-stanols in humans, Lipids in Health and Disease, 10, 187, 1-10 (2011) |
| [31] | Expert Panel on DetectionEvaluation, and Treatment of High Blood Cholesterol in Adults, Executive summary of the third report of the National Cholesterol Education Program (NCEP) Expert Panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III), Journal of the American Medical Association, 285, 19, 2486-2497 (2001) |
| [32] | Flendrie, M.; Creemers, M. C.W.; van Riel, P. L.C. M., Titration of infliximab treatment in rheumatoid arthritis patients based on response patterns, Rheumatology (Oxford), 46, 1, 146-149 (2007) |
| [33] | Hall, E. J.; Giaccia, A. J., Radiobiology for the radiologist (2005), Lippincott Williams & Wilkins: Lippincott Williams & Wilkins Philadelphia, Pennsylvania, USA |
| [34] | Hasenclever, D.; Loeffler, M.; Diehl, V., Rationale for dose escalation of first line conventional chemotherapy in advanced Hodgkin’s disease, Annals of Oncology, 7, Supplement 4, S95-S98 (1996) |
| [35] | Helm, J. E.; Lavieri, M. S.; Oyen, M. P.V.; Stein, J. D.; Musch, D. C., Dynamic forecasting and control algorithms for glaucoma progression for clinical decision support, Operations Research, 63, 5, 979-999 (2015) · Zbl 1331.93225 |
| [36] | Hogg, R. S.; Havlir, D.; Miller, V.; Montaner, J., To stop or not to stop: That is the question, but what is the answer?, AIDS, 16, 787-789 (2002) |
| [37] | Hu, C.; Lovejoy, W. S.; Shafer, S. L., Comparison of some suboptimal control policies in medical drug therapy, Operations Research, 44, 5, 696-709 (1996) · Zbl 0879.90134 |
| [38] | Iasonos, A.; O’Quigley, J., Adaptive dose-finding studies: A review of model-guided phase I clinical trials, Journal of Clinical Oncology, 32, 12, 2505-2511 (2014) |
| [39] | Jackson, C. H., Multi-state models for panel data: The MSM package for R, Journal of Statistical Software, 38, 8, 1-29 (2011) |
| [40] | Jelliffe, R. W.; Maire, P.; Sattler, F.; Gomis, P.; Tahani, B., Adaptive control of drug dosage regimens: Basic foundations, relevant issues, and clinical examples, International Journal of Bio-Medical Computing, 36, 1-23 (1994) |
| [41] | Keenan, D. M.; Basu, R.; Liu, Y.; Basu, A.; Bock, G.; Veldhuis, J. D., Logistic model of glucose-regulated c-peptide secretion: Hysteresis pathway disruption in impaired fasting glycemia, American Journal of Physiology—Endocrinology and Metabolism, 303, 3, E397-E409 (2012) |
| [42] | Kim, M.; Ghate, A.; Phillips, M. H., A Markov decision process approach to temporal modulation of dose fractions in radiation therapy planning, Physics in Medicine and Biology, 54, 4455-4476 (2009) |
| [43] | Kim, M.; Ghate, A.; Phillips, M. H., A stochastic control formalism for dynamic biologically conformal radiation therapy, European Journal of Operational Research, 219, 3, 541-556 (2012) · Zbl 1253.93138 |
| [44] | Kirby, S.; Brain, P.; Jones, B., Fitting Emax models to clinical trial dose-response data, Pharmaceutical Statistics, 10, 2, 143-149 (2011) |
| [45] | Kotas, J.; Ghate, A., Response-guided dosing for rheumatoid arthritis, IIE Transactions on Healthcare Systems Engineering, 6, 1, 1-21 (2016) |
| [46] | Kumar, P. R., A survey of some results in stochastic adaptive control, SIAM Journal on Control and Optimization, 23, 3, 329-380 (1985) · Zbl 0571.93038 |
| [47] | Kurt, M.; Denton, B. T.; Schaefer, A. J.; Shah, N. D.; Smith, S. A., The structure of optimal statin initiation policies for patients with type 2 diabetes, IIE Transactions of Healthcare Systems Engineering, 1, 1, 49-65 (2011) |
| [48] | Laber, E. B.; Lizotte, D. J.; Qian, M.; Pelham, W. E.; Murphy, S. A., Dynamic treatment regimes: Technical challenges and applications, Electronic Journal of Statistics, 8, 1, 1225-1272 (2014) · Zbl 1298.62189 |
| [49] | Laurino, M.; Landi, A., A model predictive control strategy toward optimal structured treatment interruptions in anti-HIV therapy, IEEE Transactions on Biomedical Engineering, 57, 5, 1040-1050 (2010) |
| [50] | Lehmann, E. L., Theory of point estimation (1983), Wadsworth & Brooks/Cole: Wadsworth & Brooks/Cole Pacific Grove, California, USA · Zbl 0522.62020 |
| [51] | Lewens, T., Distinguishing treatment from research: a functional approach, Journal of Medical Ethics, 32, 7, 424-429 (2006) |
| [52] | Lori, F.; Lisziewicz, J., Structured treatment interruptions for the management of HIV infection, The Journal of the American Medical Association, 286, 23, 2981-2987 (2001) |
| [53] | Macdougall, J., Analysis of dose-response studies—Emax model, (Ting, N., Dose finding in drug development. Dose finding in drug development, Statistics for Biology and Health, chapter 9 (2006), Springer: Springer New York, NY, USA), 127-145 · Zbl 1129.62106 |
| [54] | Maggiolo, F.; Ripamonti, D.; Gregis, G.; Quinzan, G.; Callagaro, A.; Suter, F., Effect of prolonged discontinuation of successful antiretroviral therapy on CD4 t cells: A controlled, prospective study, AIDS, 18, 439-446 (2004) |
| [55] | Maillart, L. M.; Ivy, J. S.; Ransom, S.; Diehl, K., Assessing dynamic breast cancer screening policies, Operations Research, 56, 6, 1411-1427 (2008) |
| [56] | Mandema, J. W.; Salinger, D. H.; Baumgartner, S. W.; Gibbs, M. A., A dose-response meta-analysis for quantifying relative efficacy of biologics in rheumatoid arthritis, Clinical Pharmacology & Therapeutics, 90, 6, 828-835 (2011) |
| [57] | Martino, V. D.; Richou, C.; Cervoni, J.-P.; Sanchez-Tapias, J. M.; Jensen, D. M.; Mangia, A., Response-guided peg-interferon plus ribavirin treatment duration in chronic hepatitis C: Meta-analyses of randomized, controlled trials and implications for the future, Hepatology, 54, 3, 789-800 (2011) |
| [58] | Mason, J. E.; Denton, B. T.; Shah, N. D.; Smith, S. A., Optimizing the simultaneous management of blood pressure and cholesterol for type 2 diabetes patients, European Journal of Operational Research, 233, 3, 727-738 (2014) · Zbl 1339.90354 |
| [59] | Michelson, S.; Schofield, T., The biostatistics cookbook (1996), Kluwer Academic: Kluwer Academic Boston, MA, USA · Zbl 0910.62109 |
| [60] | Murphy, S. A., Optimal dynamic treatment regimes, Journal of the Royal Statistical Society, 65, 2, 331-366 (2003) · Zbl 1065.62006 |
| [61] | Murphy, S. A., An experimental design for the development of adaptive treatment strategies, Statistics in Medicine, 24, 10, 1455-1481 (2005) |
| [62] | Murphy, S. A.; Collins, L. M.; Rush, A. J., Customizing treatment to the patient: Adaptive treatment strategies, Drug and Alcohol Dependence, 88, Supplement 2, S1-S3 (2007) |
| [63] | Negoescu, D., Bimpikis, K., Brandeau, M. L., & Iancu, D. A. (2014). Dynamic learning of patient response types: An application to treating chronic diseases. http://www.isye.umn.edu/news/pdf/bandits_cronic_disease(1).pdf; Negoescu, D., Bimpikis, K., Brandeau, M. L., & Iancu, D. A. (2014). Dynamic learning of patient response types: An application to treating chronic diseases. http://www.isye.umn.edu/news/pdf/bandits_cronic_disease(1).pdf |
| [64] | Peterzan, M. A.; Hardy, R.; Chaturvedi, N.; Hughes, A. D., Meta-analysis of dose-response relationships for hydrochlorothiazide, chlorthalidone, and bendroflumethiazide on blood pressure, serum potassium, and urate, Hypertension, 59, 6, 1104-1109 (2012) |
| [65] | Pinheiro, J., Evaluation and recommendations on adaptive dose-ranging trials: Highlights from the PhRMA adaptive dose-ranging studies working group, The Journal of Clinical Pharmacology, 50, S9, 47S-49S (2010) |
| [66] | Pinheiro, J.; Bornkamp, B.; Glimm, E.; Bretz, F., Model-based dose finding under model uncertainty using general parametric models, Statistics in Medicine, 33, 10, 1646-1661 (2014) |
| [67] | Rahman, M. U.; Strusberg, I.; Geusens, P.; Berman, A.; Yocum, D.; Baker, D., Double-blinded infliximab dose escalation in patients with rheumatoid arthritis, Annals of Rheumatoid Disease, 66, 9, 1233-1238 (2007) |
| [68] | Rosenberger, W. F.; Haines, L. M., Competing designs for phase I clinical trials, Statistic in Medicine, 21, 18, 2757-2770 (2002) |
| [69] | Sandikci, B.; Maillart, L. M.; Schaefer, A. J.; Alagoz, O.; Roberts, M. S., Estimating the patient’s price of privacy in liver transplantation, Operations Research, 56, 6, 1393-1410 (2008) · Zbl 1167.90676 |
| [70] | Schacter, L.; Birkhofer, M.; Carter, S.; Canetta, R.; Hellmann, S.; Onetto, N., Anticancer drugs, (O’Grady, J.; Joubert, P. H., Handbook of phase I/phase II clinical trials (1997), CRC Press), 523-534 |
| [71] | Schaefer, A. J.; Bailey, M. D.; Shechter, S. M.; Roberts, M. S., Modeling medical treatment using Markov decision processes, Operations research and health care. Operations research and health care, International Series in Operations Research & Management Science, vol. 70, 593-612 (2005), Springer: Springer Berlin, Germany |
| [72] | Schumitzky, A., Application of stochastic control theory to optimal design of dosage regimens, (D’Argenio, D. Z., Advanced methods of pharmacokinetic and pharmacodynamic systems analysis, chapter 13, vol. 1 (1991), Plenum Press), 135-152 |
| [73] | Shechter, S. M.; Bailey, M. D.; Schaefer, A. J., A modeling framework for replacing medical therapies, IIE Transactions, 40, 9, 861-869 (2008) |
| [74] | Shechter, S. M.; Bailey, M. D.; Schaefer, A. J.; Roberts, M. S., The optimal time to initiate HIV therapy under ordered health states, Operations Research, 56, 1, 20-33 (2008) · Zbl 1167.90579 |
| [75] | Shen, L.; Peterson, S.; Sedaghat, A. R.; McMahon, M. A.; Callender, M.; Zhang, H., Dose-response curve slope sets class-specific limits on inhibitory potential of anti-HIV drugs, Nature Medicine, 14, 7, 762-766 (2008) |
| [76] | Slob, W., Dose-response modeling of continuous endpoints, Toxicological Sciences, 66, 298-312 (2002) |
| [77] | SMART Study Group, CD4+ count-guided interruption of antiretroviral treatment, New England Journal of Medicine, 355, 22, 2283-2296 (2006) |
| [78] | Smolen, J.; Beaulieu, A.; Rubbert-Roth, A.; Ramos-Remus, C.; Rovensky, J.; Alecock, E., Effect of interleukin-6 receptor inhibition with tocilizumab in patients with rheumatoid arthritis (option study): A double-blind, placebo-controlled, randomised trial, The Lancet, 371, 9617, 987-997 (2008) |
| [79] | Talal, A.; Ribeiro, R. M.; Powers, K. A.; Grace, M.; Cullen, C.; Hussain, M., Pharmacodynamics of peg-ifn differentiate HIV/HCV coinfected sustained virological responders from nonresponders, Hepatology, 43, 5, 943-953 (2006) |
| [80] | Ting, N., Dose finding in drug development. Dose finding in drug development, Statistics for biology and health (2006), Springer: Springer New York, NY, USA · Zbl 1129.62106 |
| [81] | Titman, A. C.; Sharples, L. D., Model diagnostics for multi-state models, Statistical Methods in Medical Research, 19, 6, 621-651 (2009) |
| [82] | U.S. Food and Drug Administration (1994). Industry guideline: Dose-response information to support drug registration. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm073115.pdf; U.S. Food and Drug Administration (1994). Industry guideline: Dose-response information to support drug registration. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm073115.pdf |
| [83] | U.S. Food and Drug Administration (2010). Guidance for industry: Adaptive design clinical trials for drugs and biologics. http://www.fda.gov/downloads/DrugsGuidanceComplianceRegulatoryInformation/Guidances/UCM201790.pdf; U.S. Food and Drug Administration (2010). Guidance for industry: Adaptive design clinical trials for drugs and biologics. http://www.fda.gov/downloads/DrugsGuidanceComplianceRegulatoryInformation/Guidances/UCM201790.pdf |
| [84] | van der Maas, A.; Kievit, W.; van den Bemt, B. J.F.; van den Hoogen, F. H.J.; van Riel, P. L.C. M.; den Broeder, A. A., Down-titration and discontinuation of infliximab in rheumatoid arthritis patients with stable low disease activity and stable treatment: An observational cohort study, Annals of Rheumatoid Disease, 71, 11, 1849-1854 (2012) |
| [85] | Waterhouse, W. C., Do symmetric problems have symmetric solutions?, The American Mathematical Monthly, 90, 6, 378-387 (1983) · Zbl 0533.26008 |
| [86] | Wein, L. M.; Zenios, S. A.; Nowak, M. A., Dynamic multidrug therapies for HIV: A control theoretic approach, Journal of Theoretical Biology, 185, 1, 15-29 (1997) |
| [87] | Zeuzem, S.; Pawlotsky, J.-M.; Lukasiewicz, E.; von Wagner, M.; Goulis, I.; Lurie, Y., International, multicenter, randomized, controlled study comparing dynamically individualized versus standard treatment in patients with chronic hepatitis C, Journal of Hepatology, 43, 2, 250-257 (2005) |
| [88] | Zhang, J.; Denton, B. T.; Balasubramanian, H.; Shah, N. D.; Inman, B. A., Optimization of prostate biopsy referral decisions, Manufacturing and Service Operations Management, 14, 4, 529-547 (2012) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
