Attrition in Conversational Agent-Delivered Mental Health Interventions: Systematic Review and Meta-Analysis
- PMID: 38412023
- PMCID: PMC10933752
- DOI: 10.2196/48168
Attrition in Conversational Agent-Delivered Mental Health Interventions: Systematic Review and Meta-Analysis
Abstract
Background: Conversational agents (CAs) or chatbots are computer programs that mimic human conversation. They have the potential to improve access to mental health interventions through automated, scalable, and personalized delivery of psychotherapeutic content. However, digital health interventions, including those delivered by CAs, often have high attrition rates. Identifying the factors associated with attrition is critical to improving future clinical trials.
Objective: This review aims to estimate the overall and differential rates of attrition in CA-delivered mental health interventions (CA interventions), evaluate the impact of study design and intervention-related aspects on attrition, and describe study design features aimed at reducing or mitigating study attrition.
Methods: We searched PubMed, Embase (Ovid), PsycINFO (Ovid), Cochrane Central Register of Controlled Trials, and Web of Science, and conducted a gray literature search on Google Scholar in June 2022. We included randomized controlled trials that compared CA interventions against control groups and excluded studies that lasted for 1 session only and used Wizard of Oz interventions. We also assessed the risk of bias in the included studies using the Cochrane Risk of Bias Tool 2.0. Random-effects proportional meta-analysis was applied to calculate the pooled dropout rates in the intervention groups. Random-effects meta-analysis was used to compare the attrition rate in the intervention groups with that in the control groups. We used a narrative review to summarize the findings.
Results: The systematic search retrieved 4566 records from peer-reviewed databases and citation searches, of which 41 (0.90%) randomized controlled trials met the inclusion criteria. The meta-analytic overall attrition rate in the intervention group was 21.84% (95% CI 16.74%-27.36%; I2=94%). Short-term studies that lasted ≤8 weeks showed a lower attrition rate (18.05%, 95% CI 9.91%- 27.76%; I2=94.6%) than long-term studies that lasted >8 weeks (26.59%, 95% CI 20.09%-33.63%; I2=93.89%). Intervention group participants were more likely to attrit than control group participants for short-term (log odds ratio 1.22, 95% CI 0.99-1.50; I2=21.89%) and long-term studies (log odds ratio 1.33, 95% CI 1.08-1.65; I2=49.43%). Intervention-related characteristics associated with higher attrition include stand-alone CA interventions without human support, not having a symptom tracker feature, no visual representation of the CA, and comparing CA interventions with waitlist controls. No participant-level factor reliably predicted attrition.
Conclusions: Our results indicated that approximately one-fifth of the participants will drop out from CA interventions in short-term studies. High heterogeneities made it difficult to generalize the findings. Our results suggested that future CA interventions should adopt a blended design with human support, use symptom tracking, compare CA intervention groups against active controls rather than waitlist controls, and include a visual representation of the CA to reduce the attrition rate.
Trial registration: PROSPERO International Prospective Register of Systematic Reviews CRD42022341415; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022341415.
Keywords: AI; artificial intelligence; attrition; chatbot; conversational agent; digital health interventions; dropout; mHealth; mental health; meta-analysis; mobile phone; systematic review.
©Ahmad Ishqi Jabir, Xiaowen Lin, Laura Martinengo, Gemma Sharp, Yin-Leng Theng, Lorainne Tudor Car. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 27.02.2024.
Conflict of interest statement
Conflicts of Interest: LTC is an associate editor of JMIR Medical Education. The other authors reported no conflict of interest.
Figures
Similar articles
-
Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas.Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
-
Dropout Rate in Digital Health Interventions for the Prevention of Skin Cancer: Systematic Review, Meta-analysis, and Metaregression.J Med Internet Res. 2022 Dec 9;24(12):e42397. doi: 10.2196/42397. J Med Internet Res. 2022. PMID: 36485027 Free PMC article. Review.
-
Rates of Attrition and Dropout in App-Based Interventions for Chronic Disease: Systematic Review and Meta-Analysis.J Med Internet Res. 2020 Sep 29;22(9):e20283. doi: 10.2196/20283. J Med Internet Res. 2020. PMID: 32990635 Free PMC article.
-
Effectiveness and Acceptance of Technology-Based Psychological Interventions for the Acute Treatment of Unipolar Depression: Systematic Review and Meta-analysis.J Med Internet Res. 2021 Jun 13;23(6):e24584. doi: 10.2196/24584. J Med Internet Res. 2021. PMID: 36260395 Free PMC article. Review.
-
Impact of Smartphone App-Based Psychological Interventions for Reducing Depressive Symptoms in People With Depression: Systematic Literature Review and Meta-analysis of Randomized Controlled Trials.JMIR Mhealth Uhealth. 2022 Jan 27;10(1):e29621. doi: 10.2196/29621. JMIR Mhealth Uhealth. 2022. PMID: 35084346 Free PMC article. Review.
Cited by
-
Person-Generated Health Data in Women's Health: Scoping Review.J Med Internet Res. 2024 May 16;26:e53327. doi: 10.2196/53327. J Med Internet Res. 2024. PMID: 38754098 Free PMC article. Review.
References
-
- GBD 2019 Mental Disorders Collaborators Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Psychiatry. 2022 Feb;9(2):137–50. doi: 10.1016/S2215-0366(21)00395-3. https://linkinghub.elsevier.com/retrieve/pii/S2215-0366(21)00395-3 S2215-0366(21)00395-3 - DOI - PMC - PubMed
-
- Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, Al-Hamzawi A, Alonso J, Andrade L, Borges G, Bruffaerts R, Bunting B, de Almeida JM, Florescu S, de Girolamo G, Gureje O, Haro JM, He Y, Hinkov H, Karam E, Kawakami N, Lee S, Navarro-Mateu F, Piazza M, Posada-Villa J, de Galvis YT, Kessler RC. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry. 2017 Feb;210(2):119–24. doi: 10.1192/bjp.bp.116.188078. https://europepmc.org/abstract/MED/27908899 S0007125000279671 - DOI - PMC - PubMed
-
- Chong SA, Abdin E, Vaingankar JA, Kwok KW, Subramaniam M. Where do people with mental disorders in Singapore go to for help? Ann Acad Med Singap. 2012 Apr;41(4):154–60. http://www.annals.edu.sg/pdf/41VolNo4Apr2012/V41N4p154.pdf - PubMed
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Miscellaneous