This paper is in the following e-collection/theme issue:

Anxiety and Stress Disorders (1171) Special Issue on Computing and Mental Health (2018) (33) Wearable Devices and Sensors (984) Novel Sensors and Data Acquisition Methods in Cardiology (214)

Published on in Vol 6, No 4 (2019): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10140, first published .
Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging

Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging

Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging

Authors of this article:

Youngjun Cho1, 2 Author Orcid Image ;   Simon J Julier1 Author Orcid Image ;   Nadia Bianchi-Berthouze2 Author Orcid Image
Article Authors Cited by (69) Tweetations (12) Metrics

Journals

  1. Filippini C, Perpetuini D, Cardone D, Chiarelli A, Merla A. Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review. Applied Sciences 2020;10(8):2924 View
  2. Delmastro F, Martino F, Dolciotti C. Cognitive Training and Stress Detection in MCI Frail Older People Through Wearable Sensors and Machine Learning. IEEE Access 2020;8:65573 View
  3. Elzeiny S, Qaraqe M. Stress Classification Using Photoplethysmogram-Based Spatial and Frequency Domain Images. Sensors 2020;20(18):5312 View
  4. Patlar Akbulut F, Ikitimur B, Akan A. Wearable sensor-based evaluation of psychosocial stress in patients with metabolic syndrome. Artificial Intelligence in Medicine 2020;104:101824 View
  5. Kiran kumar C, Manaswini M, Maruthy K, Siva Kumar A, Mahesh kumar K. Association of Heart rate variability measured by RR interval from ECG and pulse to pulse interval from Photoplethysmography. Clinical Epidemiology and Global Health 2021;10:100698 View
  6. Magalhaes C, Mendes J, Vardasca R. Meta-Analysis and Systematic Review of the Application of Machine Learning Classifiers in Biomedical Applications of Infrared Thermography. Applied Sciences 2021;11(2):842 View
  7. Perpetuini D, Chiarelli A, Cardone D, Filippini C, Rinella S, Massimino S, Bianco F, Bucciarelli V, Vinciguerra V, Fallica P, Perciavalle V, Gallina S, Conoci S, Merla A. Prediction of state anxiety by machine learning applied to photoplethysmography data. PeerJ 2021;9:e10448 View
  8. Singh M, Xu Q, Wang S, Hong T, Ghassemi M, Lo A, Gontis V. Real-time extended psychophysiological analysis of financial risk processing. PLOS ONE 2022;17(7):e0269752 View
  9. Vats V, Nagori A, Singh P, Dutt R, Bandhey H, Wason M, Lodha R, Sethi T. Early Prediction of Hemodynamic Shock in Pediatric Intensive Care Units With Deep Learning on Thermal Videos. Frontiers in Physiology 2022;13 View
  10. Inagaki K, Ohta Y. Capacity of Autonomous Sensory Meridian Response on the Reduction of Mental Stress. International Journal of Environmental Research and Public Health 2022;19(21):14577 View
  11. Novikov M, Anashkina D, Ivanov A, Konovalov A, Popov V. Thermographic analysis of postoperative changes in the nasal breathing efficiency in infants and young children with unilateral cleft lip. International Journal of Computer Assisted Radiology and Surgery 2021;16(12):2225 View
  12. Wang K, Julier S, Cho Y. Attention-Based Applications in Extended Reality to Support Autistic Users: A Systematic Review. IEEE Access 2022;10:15574 View
  13. Özdil A, Yilmaz B. Medical infrared thermal image based fatty liver classification using machine and deep learning. Quantitative InfraRed Thermography Journal 2024;21(2):102 View
  14. Ninan J, Kumar B, Padodara R. A Glimpse into Artificial Intelligence in Animal Physiology and Allied Sciences. Animal Reproduction Update 2022;2(1):72 View
  15. Rinella S, Massimino S, Fallica P, Giacobbe A, Donato N, Coco M, Neri G, Parenti R, Perciavalle V, Conoci S. Emotion Recognition: Photoplethysmography and Electrocardiography in Comparison. Biosensors 2022;12(10):811 View
  16. Rachim V, Baek J, Kim Y, Kim Y, Park S. High Sampling Rate Smartphone-PPG via Built-in Rolling Shutter Image Sensor. IEEE Internet of Things Journal 2023;10(1):512 View
  17. Tump D, Narayan N, Verbiest V, Hermsen S, Goris A, Chiu C, Van Stiphout R. Stressors and Destressors in Working From Home Based on Context and Physiology From Self-Reports and Smartwatch Measurements: International Observational Study Trial. JMIR Formative Research 2022;6(11):e38562 View
  18. Volkov I, Sagaidachnyi A, Fomin A. Photoplethysmographic Imaging of Hemodynamics and Two-Dimensional Oximetry. Optics and Spectroscopy 2022;130(7):452 View
  19. Aydemir T, Şahin M, Aydemir O. Sequential forward mother wavelet selection method for mental workload assessment on N-back task using photoplethysmography signals. Infrared Physics & Technology 2021;119:103966 View
  20. Holloway C, Bhot W, Yong K, McCarthy I, Suzuki T, Carton A, Yang B, Serougne R, Boampong D, Tyler N, Crutch S, Berthouze N, Cho Y. STEP-UP: Enabling Low-Cost IMU Sensors to Predict the Type of Dementia During Everyday Stair Climbing. Frontiers in Computer Science 2022;3 View
  21. Stojchevska M, Steenwinckel B, Van Der Donckt J, De Brouwer M, Goris A, De Turck F, Van Hoecke S, Ongenae F. Assessing the added value of context during stress detection from wearable data. BMC Medical Informatics and Decision Making 2022;22(1) View
  22. Mentis A, Lee D, Roussos P. Applications of artificial intelligence−machine learning for detection of stress: a critical overview. Molecular Psychiatry 2024;29(6):1882 View
  23. Gioia F, Nardelli M, Scilingo E, Greco A. Autonomic Regulation of Facial Temperature during Stress: A Cross-Mapping Analysis. Sensors 2023;23(14):6403 View
  24. Lyzwinski L, Elgendi M, Menon C. The Use of Photoplethysmography in the Assessment of Mental Health: Scoping Review. JMIR Mental Health 2023;10:e40163 View
  25. Joshi J, Wang K, Cho Y. PhysioKit: An Open-Source, Low-Cost Physiological Computing Toolkit for Single- and Multi-User Studies. Sensors 2023;23(19):8244 View
  26. Mohamed Y, Güneysu A, Lemaignan S, Leite I. Multi-modal Affect Detection Using Thermal and Optical Imaging in a Gamified Robotic Exercise. International Journal of Social Robotics 2024;16(5):981 View
  27. Baran K. Smartphone thermal imaging for stressed people classification using CNN+MobileNetV2. Procedia Computer Science 2023;225:2507 View
  28. Liu I, Liu F, Zhong Q, Ma F, Ni S. Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging. PeerJ Computer Science 2024;10:e1912 View
  29. Joshi J, Cho Y. iBVP Dataset: RGB-Thermal rPPG Dataset with High Resolution Signal Quality Labels. Electronics 2024;13(7):1334 View
  30. Farhan A, Mouhsen A, Labakoum B, Rattal M, Lyazidi A. Assessing Heart Rate Variability and Pulse Rate Variability Patterns in Cardiac Patients: Exploring the Utility of Photoplethysmography and Electrocardiography. Biomedical and Pharmacology Journal 2024;17(1):453 View
  31. Yan L, Yang J, Xia J, Gao R, Zhang L, Wan J, Tang Y. Self-supervised extracted contrast network for facial expression recognition. Multimedia Tools and Applications 2024 View
  32. Hendryani A, Rizkinia M, Gunawan D. Enhancement of Stress Classification Using Web Camera-Based Imaging Photoplethysmography With a Frame Alignment Method. IEEE Access 2024;12:122313 View
  33. Stanić V, Geršak G. Facial thermal imaging: A systematic review with guidelines and measurement uncertainty estimation. Measurement 2025;242:115879 View
  34. Jeon T, Byeol Bae H, Lee S. Multimodal Stress Recognition Using a Multimodal Neglecting Mask Module. IEEE Access 2024;12:144774 View
  35. Putrino A, Cassetta M, Raso M, Altieri F, Brilli D, Mezio M, Circosta F, Zaami S, Marinelli E. Clinical Applications, Legal Considerations and Implementation Challenges of Smartphone-Based Thermography: A Scoping Review. Journal of Clinical Medicine 2024;13(23):7117 View
  36. Baran K. ShuffleNet and XGBoost classifier for stress reactions detection. Procedia Computer Science 2024;246:3771 View
  37. Pinge A, Gad V, Jaisighani D, Ghosh S, Sen S. Detection and monitoring of stress using wearables: a systematic review. Frontiers in Computer Science 2024;6 View
  38. Aghaomidi P, Mirzaei F, Bahmani Z. MSLSNet: A combination of multi-task self-supervised learning and Swin transformer network for face and keypoint detection in thermal images. Expert Systems with Applications 2025;268:126238 View
  39. Vicnesh J, Salvi M, Hagiwara Y, Yee H, Mir H, Barua P, Chakraborty S, Molinari F, Rajendra Acharya U. Application of Infrared Thermography and Artificial Intelligence in Healthcare: A Systematic Review of Over a Decade (2013–2024). IEEE Access 2025;13:5949 View
  40. Wang K, Joshi J, Cho Y. Streaming to Connect: Exploring How Social Connectedness Relates to Empathy Types and Physiological States in Remote Virtual Audiences. Sensors 2025;25(3):872 View
  41. Liu Y, Xu Z, Oldfrey B, Cho Y. ShapeBand: Design of a Shape-Changing Wristband with Soft Materials and Physiological Sensors for Anxiety Regulation. Information 2025;16(2):101 View
  42. Jolly A, Pandey V, Sahni M, Leon-Castro E, Perez-Arellano L. Modern Smart Gadgets and Wearables for Diagnosis and Management of Stress, Wellness, and Anxiety: A Comprehensive Review. Healthcare 2025;13(4):411 View
  43. Tsai Y, Chen Y, Lin Y, Hsiao F, Hsu C, Liao L. Photoplethysmography-based HRV analysis and machine learning for real-time stress quantification in mental health applications. APL Bioengineering 2025;9(2) View
  44. Rayhan F, Joshi J, Ren G, Hernandez L, Petreca B, Baurley S, Berthouze N, Cho Y. Advancing Textile Damage Segmentation: A Novel RGBT Dataset and Thermal Frequency Normalization. Sensors 2025;25(7):2306 View
  45. Yeh Y, Jo S, Cho Y. Uplifting Moods: Augmented Reality-Based Gamified Mood Intervention App with Attention Bias Modification. Software 2025;4(2):8 View

Books/Policy Documents

  1. Lu P, Zhang W, Ma L, Zhao Q. Cross-Cultural Design. Applications in Health, Learning, Communication, and Creativity. View
  2. Sowmiya E, Nirmala K, Suganthi L. ICDSMLA 2021. View
  3. Królak A, Pilecka E. Biocybernetics and Biomedical Engineering – Current Trends and Challenges. View
  4. Mejía-Mejía E, Allen J, Budidha K, El-Hajj C, Kyriacou P, Charlton P. Photoplethysmography. View
  5. Fauquet-Alekhine P, Granry J. The Palgrave Handbook of Occupational Stress. View
  6. Gioia F, Nardelli M, Scilingo E, Greco A. MEDICON’23 and CMBEBIH’23. View
  7. Dutta T, Bandyopadhyay A. Emotion, Cognition and Silent Communication: Unsolved Mysteries. View

Books/Policy Documents

  1. . Disability Interactions. View

Conference Proceedings

  1. Cho Y, Bianchi-Berthouze N, Oliveira M, Holloway C, Julier S. 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). Nose Heat: Exploring Stress-induced Nasal Thermal Variability through Mobile Thermal Imaging View
  2. Wang K, Zhang B, Cho Y. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. Using Mobile Augmented Reality to Improve Attention in Adults with Autism Spectrum Disorder View
  3. Herrera R, Holloway C, Morgado Ramirez D, Zhang B, Cho Y. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Breathing Biofeedback Relaxation Intervention for Wheelchair Users in City Navigation View
  4. Cho Y. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Rethinking Eye-blink: Assessing Task Difficulty through Physiological Representation of Spontaneous Blinking View
  5. Clark J, Nath R, Thapliyal H. 2021 IEEE 7th World Forum on Internet of Things (WF-IoT). Machine Learning Based Prediction of Future Stress Events in a Driving Scenario View
  6. Roy S, Etemad A. 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII). Spatiotemporal Contrastive Learning of Facial Expressions in Videos View
  7. Roy S, Etemad A. 2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII). Analysis of Semi-Supervised Methods for Facial Expression Recognition View
  8. Moge C, Wang K, Cho Y. CHI Conference on Human Factors in Computing Systems. Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI View
  9. Panganiban F, de Leon F. 2021 IEEE Region 10 Symposium (TENSYMP). Stress Detection Using Smartphone Extracted Photoplethysmography View
  10. Clark J, Thapliyal H. 2022 IEEE International Conference on Consumer Electronics (ICCE). Edge Device Based Stress Detection For Older Adults With Cortisol Biomarker View
  11. Hasan M, Zaman S, Wesley A, Tsiamyrtzis P, Pavlidis I. Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. Sympathetic Activation in Deadlines of Deskbound Research - A Study in the Wild View
  12. Hendryani A, Rizkinia M, Gunawan D. 2022 International Conference on Healthcare Engineering (ICHE). A Review on Human Stress Detection using Biosignal Based on Image Processing Technique View
  13. Procka P, Menich L, Labuda M, Borik S. 2024 ELEKTRO (ELEKTRO). Multimodal Analysis of Tissue Dynamics - Spotlight on Perfusion and Temperature View
  14. Vardasca R, Magalhaes C, Mendes J. The 15th International Workshop on Advanced Infrared Technology and Applications. Biomedical Applications of Infrared Thermal Imaging: Current State of Machine Learning Classification View
  15. Jiang Y, Cao J, Yu C, Qiao Y. 2024 IEEE International Conference on Metaverse Computing, Networking, and Applications (MetaCom). Dog Heart Rate and Blood Oxygen Metaverse Interaction System View
  16. Motghare R, Bhoyar S, Verma P, Tale A, Barhate A. 2024 2nd DMIHER International Conference on Artificial Intelligence in Healthcare, Education and Industry (IDICAIEI). A Review on the Applications of Machine Learning Algorithms for Mental Stress Detection View