Using Machine Learning of Online Expression to Explain Recovery Trajectories: Content Analytic Approach to Studying a Substance Use Disorder Forum
- PMID: 37606984
- PMCID: PMC10481212
- DOI: 10.2196/45589
Using Machine Learning of Online Expression to Explain Recovery Trajectories: Content Analytic Approach to Studying a Substance Use Disorder Forum
Abstract
Background: Smartphone-based apps are increasingly used to prevent relapse among those with substance use disorders (SUDs). These systems collect a wealth of data from participants, including the content of messages exchanged in peer-to-peer support forums. How individuals self-disclose and exchange social support in these forums may provide insight into their recovery course, but a manual review of a large corpus of text by human coders is inefficient.
Objective: The study sought to evaluate the feasibility of applying supervised machine learning (ML) to perform large-scale content analysis of an online peer-to-peer discussion forum. Machine-coded data were also used to understand how communication styles relate to writers' substance use and well-being outcomes.
Methods: Data were collected from a smartphone app that connects patients with SUDs to online peer support via a discussion forum. Overall, 268 adult patients with SUD diagnoses were recruited from 3 federally qualified health centers in the United States beginning in 2014. Two waves of survey data were collected to measure demographic characteristics and study outcomes: at baseline (before accessing the app) and after 6 months of using the app. Messages were downloaded from the peer-to-peer forum and subjected to manual content analysis. These data were used to train supervised ML algorithms using features extracted from the Linguistic Inquiry and Word Count (LIWC) system to automatically identify the types of expression relevant to peer-to-peer support. Regression analyses examined how each expression type was associated with recovery outcomes.
Results: Our manual content analysis identified 7 expression types relevant to the recovery process (emotional support, informational support, negative affect, change talk, insightful disclosure, gratitude, and universality disclosure). Over 6 months of app use, 86.2% (231/268) of participants posted on the app's support forum. Of these participants, 93.5% (216/231) posted at least 1 message in the content categories of interest, generating 10,503 messages. Supervised ML algorithms were trained on the hand-coded data, achieving F1-scores ranging from 0.57 to 0.85. Regression analyses revealed that a greater proportion of the messages giving emotional support to peers was related to reduced substance use. For self-disclosure, a greater proportion of the messages expressing universality was related to improved quality of life, whereas a greater proportion of the negative affect expressions was negatively related to quality of life and mood.
Conclusions: This study highlights a method of natural language processing with potential to provide real-time insights into peer-to-peer communication dynamics. First, we found that our ML approach allowed for large-scale content coding while retaining moderate-to-high levels of accuracy. Second, individuals' expression styles were associated with recovery outcomes. The expression types of emotional support, universality disclosure, and negative affect were significantly related to recovery outcomes, and attending to these dynamics may be important for appropriate intervention.
Keywords: content analysis; expression effects; mobile phone; online peer support forum; substance use disorder; supervised machine learning.
©Ellie Fan Yang, Rachel Kornfield, Yan Liu, Ming-Yuan Chih, Prathusha Sarma, David Gustafson, John Curtin, Dhavan Shah. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 22.08.2023.
Conflict of interest statement
Conflicts of Interest: DG has an ownership stake in CHESS Health, a digital health company. This relationship is managed by the University of Wisconsin–Madison. All other authors declare no other conflicts of interest.
Figures
Similar articles
-
Detecting Recovery Problems Just in Time: Application of Automated Linguistic Analysis and Supervised Machine Learning to an Online Substance Abuse Forum.J Med Internet Res. 2018 Jun 12;20(6):e10136. doi: 10.2196/10136. J Med Internet Res. 2018. PMID: 29895517 Free PMC article.
-
Eliciting and receiving online support: using computer-aided content analysis to examine the dynamics of online social support.J Med Internet Res. 2015 Apr 20;17(4):e99. doi: 10.2196/jmir.3558. J Med Internet Res. 2015. PMID: 25896033 Free PMC article.
-
What Do You Say Before You Relapse? How Language Use in a Peer-to-peer Online Discussion Forum Predicts Risky Drinking among Those in Recovery.Health Commun. 2018 Sep;33(9):1184-1193. doi: 10.1080/10410236.2017.1350906. Epub 2017 Aug 9. Health Commun. 2018. PMID: 28792228 Free PMC article.
-
Social Connection and Online Engagement: Insights From Interviews With Users of a Mental Health Online Forum.JMIR Ment Health. 2019 Mar 26;6(3):e11084. doi: 10.2196/11084. JMIR Ment Health. 2019. PMID: 30912760 Free PMC article. Review.
-
Meeting the support needs of patients with complex regional pain syndrome through innovative use of wiki technology: a mixed-methods study.Southampton (UK): NIHR Journals Library; 2014 Jul. Southampton (UK): NIHR Journals Library; 2014 Jul. PMID: 25642496 Free Books & Documents. Review.
References
-
- Key substance use and mental health indicators in the United States: results from the 2018 national survey on drug use and health. Samhsa.gov. 2019. [2023-06-06]. https://www.samhsa.gov/data/sites/default/files/cbhsq-reports/NSDUHNatio... .
-
- Chih MY, Patton T, McTavish FM, Isham AJ, Judkins-Fisher CL, Atwood AK, Gustafson DH. Predictive modeling of addiction lapses in a mobile health application. J Subst Abuse Treat. 2014 Jan;46(1):29–35. doi: 10.1016/j.jsat.2013.08.004. https://europepmc.org/abstract/MED/24035143 S0740-5472(13)00182-7 - DOI - PMC - PubMed
-
- Paliwal P, Hyman SM, Sinha R. Craving predicts time to cocaine relapse: further validation of the now and brief versions of the cocaine craving questionnaire. Drug Alcohol Depend. 2008 Mar 01;93(3):252–9. doi: 10.1016/j.drugalcdep.2007.10.002. https://europepmc.org/abstract/MED/18063320 S0376-8716(07)00405-X - DOI - PMC - PubMed
-
- Farvolden P, Cunningham J, Selby P. Using e-health programs to overcome barriers to the effective treatment of mental health and addiction problems. J Technol Hum Serv. 2009 Feb 03;27(1):5–22. doi: 10.1080/15228830802458889. https://www.tandfonline.com/doi/abs/10.1080/15228830802458889 - DOI - DOI
-
- Green-Hamann S, Campbell EK, Sherblom J. An exploration of why people participate in second life social support groups. J Comput Mediat Commun. 2011 Jul 05;16(4):465–91. doi: 10.1111/j.1083-6101.2011.01543.x. https://onlinelibrary.wiley.com/doi/full/10.1111/j.1083-6101.2011.01543.x - DOI - DOI
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources