Medicine:Dynamic treatment regime

In medical research, a dynamic treatment regime (DTR), adaptive intervention, or adaptive treatment strategy is a set of rules for choosing effective treatments for individual patients. Historically, medical research and the practice of medicine tended to rely on an acute care model for the treatment of all medical problems, including chronic illness. Treatment choices made for a particular patient under a dynamic regime are based on that individual's characteristics and history, with the goal of optimizing his or her long-term clinical outcome. A dynamic treatment regime is analogous to a policy in the field of reinforcement learning, and analogous to a controller in control theory. While most work on dynamic treatment regimes has been done in the context of medicine, the same ideas apply to time-varying policies in other fields, such as education, marketing, and economics.

See also

• Personalized medicine
• Reinforcement learning
• Q learning
• Optimal control
• Multi-armed bandit

Further reading

• Diaz, Francisco J.; Cogollo, Myladis R.; Spina, Edoardo; Santoro, Vincenza; Rendon, Diego M.; Leon, jose de (2012), "Drug Dosage Individualization Based on a Random-Effects Linear Model", Journal of Biopharmaceutical Statistics 22 (3): 463–484, doi:10.1080/10543406.2010.547264, PMID 22416835, https://uknowledge.uky.edu/psychiatry_facpub/12 
• Diaz, Francisco J.; Yeh, Hung-Weh; Leon, Jose de (2012), "Role of Statistical Random-Effects Linear Models in Personalized Medicine", Current Pharmacogenomics and Personalized Medicine 10 (1): 22–32, doi:10.2174/1875692111201010022, PMID 23467392 
• Banerjee, A.; Tsiatis, A. A. (2006), "Adaptive two-stage designs in phase II clinical trials", Statistics in Medicine 25 (19): 3382–3395, doi:10.1002/sim.2501, PMID 16479547 
• Collins, L. M. (2005), "A strategy for optimizing and evaluating behavioral interventions", Annals of Behavioral Medicine 30 (1): 65–73, doi:10.1207/s15324796abm3001_8, PMID 16097907 
• Guo, X.; Tsiatis, A. A. (2005), "Estimation of survival distributions in two-stage randomization designs with censored data", International Journal of Biostatistics 1 (1), doi:10.2202/1557-4679.1000 
• Hernán, Miguel A.; Lanoy, Emilie; Costagliola, Dominique; Robins, James M. (2006), "Comparison of Dynamic Treatment Regimes via Inverse Probability Weighting", Basic & Clinical Pharmacology & Toxicology 98 (3): 237–242, doi:10.1111/j.1742-7843.2006.pto_329.x, PMID 16611197 
• Lavori, P. W.; Dawson, R. (2000), "A design for testing clinical strategies: biased adaptive within-subject randomization", Journal of the Royal Statistical Society, Series A 163: 29–38, doi:10.1111/1467-985x.00154 
• Lavori, P.W.; Rush, A.J.; Wisniewski, S.R.; Alpert, J.; Fava, M.; Kupfer, D.J.; Nierenberg, A.; Quitkin, F.M. et al. (2001), "Strengthening clinical effectiveness trials: Equipoise-stratified randomization", Biological Psychiatry 50 (10): 792–801, doi:10.1016/s0006-3223(01)01223-9, PMID 11720698 
• Lavori, P. W.; Dawson, R (2003), "Dynamic treatment regimes: practical design considerations", Clinical Trials 1 (1): 9–20, doi:10.1191/1740774s04cn002oa, PMID 16281458 
• Lizotte, D. L.; Bowling, M. (2010), "Efficient Reinforcement Learning with Multiple Reward Functions for Randomized Clinical Trial Analysis", Twenty-Seventh Annual International Conference on Machine Learning, http://www.csd.uwo.ca/~dlizotte/publications/lizotte10multiple.pdf 
• Lokhnygina, Y; Tsiatis, A. A. (2008), "Optimal two-stage group sequential designs", Journal of Statistical Planning and Inference 138 (2): 489–499, doi:10.1016/j.jspi.2007.06.011 
• Lunceford, J. K. (2002), "Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials", Biometrics 58 (1): 48–57, doi:10.1111/j.0006-341x.2002.00048.x, PMID 11890326 
• Moodie, E. E. M.; Richardson, T. S.; Stephens, D. A. (2007), "Demystifying optimal dynamic treatment regimes", Biometrics 63 (2): 447–455, doi:10.1111/j.1541-0420.2006.00686.x, PMID 17688497 
• "Marginal Mean Models for Dynamic Regimes", Journal of the American Statistical Association 96 (456): 1410–1423, 2001, doi:10.1198/016214501753382327, PMID 20019887 
• "Optimal Dynamic Treatment Regimes", Journal of the Royal Statistical Society, Series B 65 (2): 331–366, 2003, doi:10.1111/1467-9868.00389 
• Murphy (2005), "An Experimental Design for the Development of Adaptive Treatment Strategies", Statistics in Medicine 24 (10): 1455–1481, doi:10.1002/sim.2022, PMID 15586395, https://deepblue.lib.umich.edu/bitstream/2027.42/39201/1/2022_ftp.pdf 
• "Dynamic Treatment Regimes", Encyclopedia of Medical Decision Making: #, 2008 
• Nair, V.; Strecher, V.; Fagerlin, A.; Ubel, P.; Resnicow, K. (2008), "Screening Experiments and Fractional Factorial Designs in Behavioral Intervention Research", American Journal of Public Health 98 (8): 1354–1359, doi:10.2105/ajph.2007.127563, PMID 18556602 
• Orellana, Liliana; Rotnitzky; Robins, James M. (2010), "Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part I: Main Content", The International Journal of Biostatistics 6 (2), doi:10.2202/1557-4679.1200, http://www.bepress.com/ijb/vol6/iss2/8, retrieved 2010-04-12 
• Orellana, Liliana; Rotnitzky; Robins, James M. (2010), "Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part II: Proofs of Results", The International Journal of Biostatistics 6 (2): 9, doi:10.2202/1557-4679.1242, PMID 20405047, PMC 2854089, http://www.bepress.com/ijb/vol6/iss2/9, retrieved 2010-04-12 
• Lin, D. Y.; Heagerty, P. J., eds. (2004), "Optimal structural nested models for optimal sequential decisions", Proceedings of the Second Seattle Symposium on Biostatistics, Springer, New York, pp. 189–326 
• "A new approach to causal inference in mortality studies with sustained exposure periods-application to control of the healthy worker survivor effect", Computers and Mathematics with Applications 14: 1393–1512, 1986 
• "Addendum to 'A new approach to causal inference in mortality studies with sustained exposure periods-application to control of the healthy worker survivor effect'", Computers and Mathematics with Applications 14 (9–12): 923–945, 1987, doi:10.1016/0898-1221(87)90238-0 
• Rush, A.J.; Trivedi, M.; Fava, M. (2003), "Depression IV: STAR*D treatment trial for depression", American Journal of Psychiatry 160 (2): 237, doi:10.1176/appi.ajp.160.2.237, PMID 12562566 
• Schneider, L.S.; Tariot, P.N.; Lyketsos, C.G.; Dagerman, K.S.; Davis, K.L.; Davis, S.; Hsiao, J.K.; Jeste, D.V. et al. (2001), "National Institute of Mental Health clinical antipsychotic trials of intervention effectiveness (CATIE) Alzheimer disease trial methodology", American Journal of Geriatric Psychiatry 9 (4): 346–360, doi:10.1097/00019442-200111000-00004, PMID 11739062 
• Sutton, R. S.; Barto, A. G. (1998), Reinforcement Learning: An Introduction, MIT Press, ISBN 978-0-262-19398-6, http://www.cs.ualberta.ca/~sutton/book/ebook/the-book.html 
• van der Laan, M. J. (2003), Unified Methods for Censored Longitudinal Data and Causality, Springer-Verlag, ISBN 978-0-387-95556-8 
• van der Laan, M. J.; Petersen, M. L. (2004), History-Adjusted Marginal Structural Models and Statically-Optimal Dynamic Treatment Regimes, http://works.bepress.com/mark_van_der_laan/83 
• Wagner, E. H.; Austin, B. T.; Davis, C.; Hindmarsh, M.; Schaefer, J.; Bonomi, A. (2001), "Improving Chronic Illness Care: Translating Evidence Into Action", Health Affairs 20 (6): 64–78, doi:10.1377/hlthaff.20.6.64, PMID 11816692 
• Wahed, A.. S.; Tsiatis, A. A. (2004), "Optimal estimator for the survival distribution and related quantities for treatment policies in two-stage randomization designs in clinical trials", Biometrics 60 (1): 124–133, doi:10.1111/j.0006-341X.2004.00160.x, PMID 15032782 
• Watkins, C. J. C. H. (1989), "Learning from Delayed Rewards", PhD thesis, Cambridge University, Cambridge, England 
• Zhao, Y.; Kosorok, M. R.; Zeng, D. (2009), "Reinforcement learning design for cancer clinical trials", Statistics in Medicine 28 (26): 3294–3315, doi:10.1002/sim.3720, PMID 19750510 
• Zajonc, T. (2010), Bayesian Inference for Dynamic Treatment Regimes: Mobility, Equity, and Efficiency in Student Tracking