Skip to main content

Advertisement

Springer Nature Link
Log in

Automated Detection of Mental Stress Using Multimodal Characterization of PPG Signal for AI Based Healthcare Applications

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Purpose-The increasing complexity of our society has made mental stress a part of every human life. The early evaluation of long-term mental stress conditions is necessary since they may trigger a number of chronic disorders. Existing electroencephalogram (EEG) based methods for estimating mental stress are often complex, multi-channel, and expert-dependent. On the other hand, the respiratory signal offers intriguing information about stress, but its capture is challenging and requires multimodal support. Method-To overcome this challenge, the respiratory signal can be extracted from the Photoplethysmogram (PPG) signal. The easy-to-acquire PPG signal is multimodally characterised in this proposed approach to determine the stressed status. Exclusively, the created algorithm leverages a major PPG characteristic and, using streamlined approaches, additionally extracts the respiratory rate from the same PPG signal. The method is assessed using PPG records gathered from the publicly available DEAP dataset. Result-A simple threshold-based along with standard classification techniques are used to assess the effectiveness of the proposed algorithm, which classify the stressed and relaxed states with an average accuracy of 98.43%. The suggested approach outperforms the existing methods in terms of performance, and its simple methodology and low acquisition load support its use in real-time standalone, personal healthcare applications. Conclusion-The use of simple features and classification strategy ensures the applicability of the proposed method in artificial intelligence (AI) based healthcare applications. This will upgrade the existing system for monitoring and proper diagnosis of patients suffering from mental stress conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

Not applicable.

Code Availability

Not applicable.

References

  1. Daglius DR, Scalabrini NA. Stress levels during emergency care: a comparison between reality and simulated scenarios. J Crit Care. 2016. https://doi.org/10.1016/j.jcrc.2016.02.01.

    Article  Google Scholar 

  2. 2013 Comprehensive Mental Health Action Plan 2013–2020 (Geneva: World Health Organization) (available at www.who.int/publications/i/item/9789241506021). (Accessed 05 August 2022).

  3. Thapliyal H, Khalus V, Labrado C. Stress Detection and Management: a Survey of Wearable Smart Health devices. IEEE Consum Electron Mag. 2017;6(4):64–9. https://doi.org/10.1109/MCE.2017.2715578.

    Article  Google Scholar 

  4. Mitravinda KM, Nair DS, Srinivasa G. Mental Health in Tech: analysis of Workplace Risk factors and impact of COVID-19. SN COMPUT SCI. 2023;4:197. https://doi.org/10.1007/s42979-022-01613-z.

    Article  Google Scholar 

  5. Gedam S, Paul S. A review on Mental stress detection using Wearable sensors and Machine Learning techniques. IEEE Access. 2021;9:84045–66. https://doi.org/10.1109/ACCESS.2021.3085502.

    Article  Google Scholar 

  6. Cizmeci H, Ozcan C. Enhanced deep capsule network for EEG-based emotion recognition. Signal. Image Video Process. 2023;17(2):463–9. https://doi.org/10.1007/s11760-022-02251-x.

    Article  Google Scholar 

  7. Jebelli H, Khalili Mahdi M, Lee SA, Continuously Updated. Computationally efficient stress Recognition Framework using Electroencephalogram (EEG) by applying online Multitask Learning algorithms (OMTL). IEEE J Biomedical Health Inf. 2019;23(5):1928–39. https://doi.org/10.1109/JBHI.2018.2870963.

    Article  Google Scholar 

  8. Noroozi F, Corneanu CA, Kaminska D, Sapinski T, Escalera S, Anbarjafari G. Survey on emotional body gesture recognition. IEEE Trans Affect Comput. 2021;12(2):505–23. https://doi.org/10.1109/TAFFC.2018.2874986.

    Article  Google Scholar 

  9. Charlton PH, Celka P, Farukh B, Chowienczyk P, Alastruey J. Assessing mental stress from the photoplethysmogram: a numerical study. Physiol. Meas. 2018; 39(05).

  10. Andreou E, Alexopoulos EC, Lionis C, Varvogli L, Gnardellis C, Chrousos GP, Darviri C. Perceived stress scale: reliability and validity study in Greece. Int J Environ Res Public Health. 2011;8:3287–98.

    Article  Google Scholar 

  11. Giannakakis G, Grigoriadis D, Giannakaki K, Simantiraki O, Roniotis A, Tsiknakis M. Review on psychological stress detection using Biosignals. IEEE Trans Affect Comput. 2022;13(1):440–60. https://doi.org/10.1109/TAFFC.2019.2927337.

    Article  Google Scholar 

  12. Choi J, Ahmed B, Gutierrez-Osuna R. Development and evaluation of an ambulatory stress monitor based on wearable sensors. IEEE Trans Inf Technol Biomed. 2012;16(2):279–86. https://doi.org/10.1109/TITB.2011.2169804.

    Article  Google Scholar 

  13. Hernando A, Lázaro J, Gil E, Arza A, Garzón J, López-Antón R, De La Camara C, Laguna P, Aguiló J, Bailón R. Inclusion of respiratory frequency information in Heart Rate Variability analysis for stress Assessment. IEEE J Biomedical Health Inf. 2016;20(4):1016–25. https://doi.org/10.1109/JBHI.2016.2553578.

    Article  Google Scholar 

  14. Karlen W, Raman S, Ansermino JM, Dumont GA. Multiparameter Respiratory Rate Estimation from the Photoplethysmogram. IEEE Trans Biomed Eng. 2013;60:1946–53. https://doi.org/10.1109/TBME.2013.2246160.

    Article  Google Scholar 

  15. Banerjee JS, Mahmud M, Brown D. Heart Rate Variability-based Mental stress detection: an Explainable Machine Learning Approach. SN COMPUT SCI. 2023;4:176. https://doi.org/10.1007/s42979-022-01605-z.

    Article  Google Scholar 

  16. Gjoreski M, Luštrek M, Gams M, Gjoreski H. Monitoring stress with a wrist device using context. J Biomed Inf. 2017;73:159–70.

    Article  Google Scholar 

  17. Kalra P, Sharma V. Mental stress assessment using PPG signal a deep neural network approach. IETE J Res. 2020;69(2):879–85. https://doi.org/10.1080/03772063.2020.1844068.

    Article  Google Scholar 

  18. Bobade P, Vani M. Stress detection with machine learning and deep learning using multimodal physiological data. Proc. 2nd Int. Conf. Inventive Research in Computing Applications. 2020; pp 51–57.

  19. Mukherjee N, Mukhopadhyay S, Gupta R. Real-time mental stress detection technique using neural networks towards a wearable health monitor. Meas Sci Technol. 2022;33(4). https://doi.org/10.1088/1361-6501/ac3aae.

  20. Aristizabal S, Byun K, Wood N, Mullan AF, Porter PM, Campanella C, Jamrozik A, Nenadic I, Bauer BA. The feasibility of Wearable and Self-Report stress detection measures in a Semi-controlled Lab Environment. IEEE Access. 2021;9:102053–68. https://doi.org/10.1109/ACCESS.2021.3097038.

    Article  Google Scholar 

  21. Attar ET, Balasubramanian V, Subasi E, Kaya M. Stress analysis based on simultaneous heart rate variability and EEG monitoring. IEEE J Translational Eng Health Med. 2023;9. https://doi.org/10.1109/JTEHM.2021.3106803.

  22. Praveena G, Mathana JM. Review on Stress Detection and Management Techniques using Nano EEG Sensors. In 2022 IEEE International Conference on Nanoelectronics, Nanophotonics, Nanomaterials, Nanobioscience and Nanotechnology. 2022. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/5NANO53044.2022.9828889.

  23. Ishaque S, Khan N, Krishnan S. Physiological Signal Analysis and stress classification from VR simulations using decision tree methods. Bioengineering. 2023;10(7). https://doi.org/10.3390/bioengineering10070766.

  24. Pavel AM, Mathieson SR, Livingstone V, O’Toole JM, Pressler RM, de Vries L, Rennie J, Mitra S, Dempsey E, Murray D, Marnane W, Boylan GB. Heart rate variability analysis for the prediction of EEG grade in infants with hypoxic ischaemic encephalopathy within the first 12 h of birth. Front Pead. 2023;10. https://doi.org/10.3389/fped.2022.1016211.

  25. Paul A, Chakraborty A, Sadhukhan D, Pal S, Mitra M. A simplified PPG based approach for automated recognition of five distinct emotional states. Multimedia Tools Appl. 2023. https://doi.org/10.1007/s11042-023-16744-5.

    Article  Google Scholar 

  26. Hasanpoor Y, Motaman K, Tarvirdizadeh B, Alipour K, Ghamari M. Stress Detection Using PPG Signal and Combined Deep CNN-MLP Network. In 2022 29th National and 7th International Iranian Conference on Biomedical Engineering, ICBME. 2022 (pp. 223–228). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICBME57741.2022.10052957.

  27. Paul A, Pal S, Mitra M. Automated estimation of respiration rate from a single-Channel PPG Sensor for implementation in IoMT-Enabled portable gadgets. J Inst Eng India Ser B. 2023. https://doi.org/10.1007/s40031-023-00933-9.

    Article  Google Scholar 

  28. Smets E, Casale P, Großekathöfer U, Lamichhane B, De Raedt W, Bogaerts K, Van Diest I, Van Hoof C. Comparison of machine learning techniques for psychophysiological stress detection. In Communications in Computer and Information Science. 2016; Vol. 604: pp. 13–22. Springer Verlag. https://doi.org/10.1007/978-3-319-32270-4_2.

  29. Golgouneh A, Tarvirdizadeh B. Fabrication of a portable device for stress monitoring using wearable sensors and soft computing algorithms. Neural Comput Appl. 2020;32(11):7515–37. https://doi.org/10.1007/s00521-019-04278-7.

    Article  Google Scholar 

  30. Saeed SM, U, Anwar SM, Khalid H, Majid M, Bagci U. Electroencephalography based Classification of Long-term Stress using Psychological Labeling. Journal of Biomedical and Health Informatics. 2019. Retrieved from http://arxiv.org/abs/1907.07671.

  31. Castaldo R, Xu W, Melillo P, Pecchia L, Santamaria L, James C. Detection of mental stress due to oral academic examination via ultra-short-term HRV analysis. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. 2016; pp. 3805–3808. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/EMBC.2016.7591557.

  32. Rodríguez-Arce J, Lara-Flores L, Portillo-Rodríguez O, Martínez-Méndez R. Towards an anxiety and stress recognition system for academic environments based on physiological features. Comput Methods Programs Biomed. 2020;190. https://doi.org/10.1016/j.cmpb.2020.105408.

  33. Bobade P, Vani M. Stress Detection with Machine Learning and Deep Learning using Multimodal Physiological Data. In Proceedings of the 2nd International Conference on Inventive Research in Computing Applications, ICIRCA 2020. 2020; pp. 51–57. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICIRCA48905.2020.9183244.

  34. Mou L, Zhou C, Zhao P, Nakisa B, Rastgoo MN, Jain R, Gao W. Driver stress detection via multimodal fusion using attention-based CNN-LSTM. Expert Syst Appl. 2021;173. https://doi.org/10.1016/j.eswa.2021.114693.

  35. Koelstra S, Mühl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T, Patras I. DEAP: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput. 2012;3(1):18–31. https://doi.org/10.1109/T-AFFC.2011.15.

    Article  Google Scholar 

  36. Paul A, Das N, Pal S, Mitra M. Automated Detection of Cardinal Points of ECG Signal for feature extraction using a single median filter. J Inst Eng India Ser B. 2022. https://doi.org/10.1007/s40031-022-00754-2.

    Article  Google Scholar 

  37. Yamuza MTV, Bolea J, Orini M, Laguna P, Orrite C, Vallverdu M, Bailon R. Human emotion characterization by Heart Rate Variability Analysis guided by respiration. IEEE J Biomedical Health Inf. 2019;23:2446–54. https://doi.org/10.1109/JBHI.2019.2895589.

    Article  Google Scholar 

  38. Betti S, Lova R, Rovini E, Acerbi G, Santarelli L, Cabiati M, Ry S, Cavallo F. Evaluation of an Integrated System of Wearable Physiological Sensors for Stress Monitoring in Working environments by using biological markers. IEEE Trans Biomed Eng. 2018;65:1748–58. https://doi.org/10.1109/TBME.2017.2764507.

    Article  Google Scholar 

  39. Chakraborty A, Sadhukhan D, Mitra M. Accurate detection of dicrotic notch from PPG signal for telemonitoring applications. Int J BioMed Eng Technol. 2021;37(2):121–37. https://doi.org/10.1504/IJBET.2021.119500.

    Article  Google Scholar 

  40. Buda AJ, Pinsky MR, Ingels NB, Daughters GT, Stinson EB, Alderman EL. Effect of Intrathoracic pressure on left ventricular performance. N Engl J Med. 1979;301(9):453–9. https://doi.org/10.1056/nejm197908303010901.

    Article  Google Scholar 

  41. Bloch S, Lemeignan M, Aguilera-T N. Specific respiratory patterns distinguish among human basic emotions. Int J Psychophysiol. 1991;11(2):141–54. https://doi.org/10.1016/0167-8760(91)90006-J.

    Article  Google Scholar 

  42. Paul A, Chakraborty A, Sadhukhan D, Pal S, Mitra M. EEG based automated detection of six different Eye Movement conditions for implementation in Personal Assistive Application. Wireless Pers Commun. 2022;124(1):909–30. https://doi.org/10.1007/s11277-021-09389-w.

    Article  Google Scholar 

  43. Plutchik R. The nature of emotions: human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. Am Sci. 2001;89(4):344–50. http://www.jstor.org/stable/27857503.

    Article  Google Scholar 

  44. Marjanovic NS, Teiten C, Pallamin N, L’Her E. Evaluation of emotional excitation during standardized endotracheal intubation in simulated conditions. Ann Intensiv Care. 2018;8(1). https://doi.org/10.1186/s13613-018-0460-0.

  45. Chao H, Dong L, Liu Y, Lu B. Emotion recognition from multiband eeg signals using capsnet. Sens (Switzerland). 2019;19(9). https://doi.org/10.3390/s19092212.

  46. Rajendran VG, Jayalalitha S, Adalarasu K. EEG based evaluation of examination stress and test anxiety among College Students. IRBM. 2021;43(5):349–61. https://doi.org/10.1016/j.irbm.2021.06.011.

    Article  Google Scholar 

  47. Cui H, Liu A, Zhang X, Chen X, Wang K, Chen X. EEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network. Knowl Based Syst. 2020;205. https://doi.org/10.1016/j.knosys.2020.106243.

  48. Schmidt P, Reiss A, Duerichen R, Van Laerhoven K. Introducing WeSAD, a multimodal dataset for wearable stress and affect detection. In ICMI 2018 - Proceedings of the 2018 International Conference on Multimodal Interaction. 2018; pp. 400–408. Association for Computing Machinery, Inc. https://doi.org/10.1145/3242969.3242985.

  49. Heo S, Kwon S, Lee J. Stress detection with single PPG Sensor by orchestrating multiple denoising and peak-detecting methods. IEEE Access. 2021;9:47777–85. https://doi.org/10.1109/ACCESS.2021.3060441.

    Article  Google Scholar 

Download references

Funding

This work did not receive any grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Applied Electronics & Instrumentation Engineering, RCC Institute of Information Technology, Kolkata, West Bengal, India

    Avishek Paul

  2. Department of Applied Physics, University of Calcutta, Kolkata, West Bengal, India

    Avishek Paul, Abhishek Chakraborty, Deboleena Sadhukhan, Saurabh Pal & Madhuchhanda Mitra

  3. Institut Langevin, Université de Paris / CNRS, UMR 7587, ESPCI Paris / PSL University / Sorbonne Université, 1 rue Jussieu, Paris, 75238, France

    Deboleena Sadhukhan

Authors
  1. Avishek Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abhishek Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deboleena Sadhukhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saurabh Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Madhuchhanda Mitra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Avishek Paul.

Ethics declarations

Ethical Approval

All procedures performed in this study involving human participants were according to the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent and patient details

The authors declare that this report does not contain any personal information that could lead to the identification of the patient(s).

Competing Interest

The authors declare that they have no known competing financial or personal relationships that could be viewed as influencing the work reported in this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Paul, A., Chakraborty, A., Sadhukhan, D. et al. Automated Detection of Mental Stress Using Multimodal Characterization of PPG Signal for AI Based Healthcare Applications. SN COMPUT. SCI. 5, 736 (2024). https://doi.org/10.1007/s42979-024-03110-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-03110-x

Keywords

Search

Navigation