Use of sentiment analysis for capturing patient experience from free-text comments posted online
- PMID: 24184993
- PMCID: PMC3841376
- DOI: 10.2196/jmir.2721
Use of sentiment analysis for capturing patient experience from free-text comments posted online
Abstract
Background: There are large amounts of unstructured, free-text information about quality of health care available on the Internet in blogs, social networks, and on physician rating websites that are not captured in a systematic way. New analytical techniques, such as sentiment analysis, may allow us to understand and use this information more effectively to improve the quality of health care.
Objective: We attempted to use machine learning to understand patients' unstructured comments about their care. We used sentiment analysis techniques to categorize online free-text comments by patients as either positive or negative descriptions of their health care. We tried to automatically predict whether a patient would recommend a hospital, whether the hospital was clean, and whether they were treated with dignity from their free-text description, compared to the patient's own quantitative rating of their care.
Methods: We applied machine learning techniques to all 6412 online comments about hospitals on the English National Health Service website in 2010 using Weka data-mining software. We also compared the results obtained from sentiment analysis with the paper-based national inpatient survey results at the hospital level using Spearman rank correlation for all 161 acute adult hospital trusts in England.
Results: There was 81%, 84%, and 89% agreement between quantitative ratings of care and those derived from free-text comments using sentiment analysis for cleanliness, being treated with dignity, and overall recommendation of hospital respectively (kappa scores: .40-.74, P<.001 for all). We observed mild to moderate associations between our machine learning predictions and responses to the large patient survey for the three categories examined (Spearman rho 0.37-0.51, P<.001 for all).
Conclusions: The prediction accuracy that we have achieved using this machine learning process suggests that we are able to predict, from free-text, a reasonably accurate assessment of patients' opinion about different performance aspects of a hospital and that these machine learning predictions are associated with results of more conventional surveys.
Keywords: Internet; machine learning; patient experience; quality.
Conflict of interest statement
Conflicts of Interest: Professor Donaldson was Chief Medical Officer, England from 1997 to 2010. Professor Darzi was Parliamentary Under-Secretary of State (Lords) in the United Kingdom Department of Health from 2007 to 2009. The other authors declare no conflicts of interest.
Figures
Similar articles
-
Associations between Internet-based patient ratings and conventional surveys of patient experience in the English NHS: an observational study.BMJ Qual Saf. 2012 Jul;21(7):600-5. doi: 10.1136/bmjqs-2012-000906. Epub 2012 Apr 20. BMJ Qual Saf. 2012. PMID: 22523318
-
Predicting HCAHPS scores from hospitals' social media pages: A sentiment analysis.Health Care Manage Rev. 2018 Oct/Dec;43(4):359-367. doi: 10.1097/HMR.0000000000000154. Health Care Manage Rev. 2018. PMID: 28225448
-
Using natural language processing to understand, facilitate and maintain continuity in patient experience across transitions of care.Int J Med Inform. 2022 Jan;157:104642. doi: 10.1016/j.ijmedinf.2021.104642. Epub 2021 Nov 11. Int J Med Inform. 2022. PMID: 34781167
-
The Voice of Chinese Health Consumers: A Text Mining Approach to Web-Based Physician Reviews.J Med Internet Res. 2016 May 10;18(5):e108. doi: 10.2196/jmir.4430. J Med Internet Res. 2016. PMID: 27165558 Free PMC article. Review.
-
Machine Learning and Natural Language Processing in Mental Health: Systematic Review.J Med Internet Res. 2021 May 4;23(5):e15708. doi: 10.2196/15708. J Med Internet Res. 2021. PMID: 33944788 Free PMC article. Review.
Cited by
-
Empowering Medical Education: Unveiling the Impact of Reflective Writing and Tailored Assessment on Deep Learning.J Adv Med Educ Prof. 2024 Jul 1;12(3):163-171. doi: 10.30476/JAMP.2024.101594.1938. eCollection 2024 Jul. J Adv Med Educ Prof. 2024. PMID: 39175585 Free PMC article.
-
Differences in Fear and Negativity Levels Between Formal and Informal Health-Related Websites: Analysis of Sentiments and Emotions.J Med Internet Res. 2024 Aug 9;26:e55151. doi: 10.2196/55151. J Med Internet Res. 2024. PMID: 39120928 Free PMC article.
-
Twitter Discussions on #digitaldementia: Content and Sentiment Analysis.J Med Internet Res. 2024 Jul 16;26:e59546. doi: 10.2196/59546. J Med Internet Res. 2024. PMID: 39012679 Free PMC article.
-
Revealing patient-reported experiences in healthcare from social media using thedesign-acquire-process-model-analyse-visualise framework.Digit Health. 2024 May 15;10:20552076241251715. doi: 10.1177/20552076241251715. eCollection 2024 Jan-Dec. Digit Health. 2024. PMID: 38757085 Free PMC article.
-
Online Patient Attitudes Toward Cutaneous Immune-Related Adverse Events Attributed to Nivolumab and Pembrolizumab: Sentiment Analysis.JMIR Dermatol. 2024 May 2;7:e53792. doi: 10.2196/53792. JMIR Dermatol. 2024. PMID: 38696235 Free PMC article. No abstract available.
References
-
- Institute of Medicine . Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001. - PubMed
-
- Darzi A. High quality care for all: NHS Next Stage Review final report. London: Department of Health; 2008. - PubMed
-
- Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: analysis of patients' online ratings of their physicians over a 5-year period. J Med Internet Res. 2012;14(1):e38. doi: 10.2196/jmir.2003. http://www.jmir.org/2012/1/e38/ - DOI - PMC - PubMed
-
- Greaves F, Millett C. Consistently increasing numbers of online ratings of healthcare in England. J Med Internet Res. 2012;14(3):e94. doi: 10.2196/jmir.2157. http://www.jmir.org/2012/3/e94/ - DOI - PMC - PubMed
-
- Demographics of internet users. 2012. [2013-05-16]. Pew Research Center http://pewinternet.org/Static-Pages/Trend-Data-(Adults)/Whos-Online.aspx.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources