| Citation: | RAO Zhi, LI Bing-lin, SUI Ya-ru, JI Xiao-qiang, LI Ming-ye. Image photoplethysmography for mental stress detection[J]. Chinese Optics, 2022, 15(6): 1350-1359. doi: 10.37188/CO.2022-0180
|
To achieve non-contact daily mental stress detection, this paper proposes a image photoplethysmography to detect mental stress. First, a video of the subject's face is recorded by the cell phone camera. Then, the proposed Dynamic Region of Interest (ROI) extraction method based on Face Mesh is used to obtain the weak skin color changes caused by heart rate fluctuations. Next, the Fast Independent Component Analysis (FastICA) algorithm, wavelet transform and narrowband bandpass filtering are combined to extract the signal and heart rate variability information based on image photoplethysmography. Then, stress-induced experiments are conducted on 30 subjects to screen 14 features for mental stress detection by comparing the differences in heart rate variability parameters between normal and stressful states, and to explore the relationship between short-term mental stress and daily mental stress due to stress induction. Finally, an additional 67 subjects are tested for daily mental stress, and a triple classifier for mental stress detection is built using the machine learning algorithm. The experimental results show that the accuracy of the three classifications of mental stress can reach 95.2%. Given that this method does not require long-term measurements and can accurately detect human mental stress levels using only a smartphone, and that the measurement method is simple, and easy to administer, and does not affect the normal psychological and mental state of the subject, it can be used as a valid tool in psychological research.
| [1] |
MAERCKER A, BREWIN C R, BRYANT R A, et al. Proposals for mental disorders specifically associated with stress in the international classification of diseases-11[J]. The Lancet, 2013, 381(9878): 1683-1685. doi: 10.1016/S0140-6736(12)62191-6
|
| [2] |
SHAH S M A, MOHAMMAD D, QURESHI M F H, et al. Prevalence, psychological responses and associated correlates of depression, anxiety and stress in a global population, during the coronavirus disease (COVID-19) pandemic[J]. Community Mental Health Journal, 2021, 57(1): 101-110. doi: 10.1007/s10597-020-00728-y
|
| [3] |
ESLER M. Mental stress and human cardiovascular disease[J]. Neuroscience &Biobehavioral Reviews, 2017, 74: 269-276.
|
| [4] |
TOMIYAMA A J. Stress and obesity[J]. Annual Review of Psychology, 2019, 70: 703-718. doi: 10.1146/annurev-psych-010418-102936
|
| [5] |
MIRANDA A R, SCOTTA A V, MÉNDEZ A L, et al. Public sector workers’ mental health in Argentina: Comparative psychometrics of the perceived stress scale[J]. Journal of Preventive Medicine &Public Health, 2020, 53(6): 429-438.
|
| [6] |
LIMA B B, HAMMADAH M, PEARCE B D, et al. Association of posttraumatic stress disorder with mental stress–induced myocardial ischemia in adults after myocardial infarction[J]. JAMA Network Open, 2020, 3(4): e202734. doi: 10.1001/jamanetworkopen.2020.2734
|
| [7] |
CAMM A J, MALIK M, BIGGER J T, et al. Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Task force of the European society of cardiology and the North American society of pacing and electrophysiology[J]. Circulation, 1996, 93(5): 1043-1065. doi: 10.1161/01.CIR.93.5.1043
|
| [8] |
PHAM T, LAU Z J, CHEN S H A, et al. Heart rate variability in psychology: a review of HRV indices and an analysis tutorial[J]. Sensors, 2021, 21(12): 3998. doi: 10.3390/s21123998
|
| [9] |
GEDAM S, PAUL S. A review on mental stress detection using wearable sensors and machine learning techniques[J]. IEEE Access, 2021, 9: 84045-84066. doi: 10.1109/ACCESS.2021.3085502
|
| [10] |
ZENG ZH K, HUANG ZH, LENG K M, et al. Nonintrusive monitoring of mental fatigue status using epidermal electronic systems and machine-learning algorithms[J]. ACS Sensors, 2020, 5(5): 1305-1313. doi: 10.1021/acssensors.9b02451
|
| [11] |
CHARLTON P H, CELKA P, FARUKH B, et al. Assessing mental stress from the photoplethysmogram: a numerical study[J]. Physiological Measurement, 2018, 39(5): 054001. doi: 10.1088/1361-6579/aabe6a
|
| [12] |
CASTALDO R, MELILLO P, BRACALE U, et al. Acute mental stress assessment via short term HRV analysis in healthy adults: a systematic review with meta-analysis[J]. Biomedical Signal Processing and Control, 2015, 18: 370-377. doi: 10.1016/j.bspc.2015.02.012
|
| [13] |
HEALEY J A, PICARD R W. Detecting stress during real-world driving tasks using physiological sensors[J]. IEEE Transactions on Intelligent Transportation Systems, 2005, 6(2): 156-166. doi: 10.1109/TITS.2005.848368
|
| [14] |
CAI H SH, CHEN Y F, HAN J SH, et al. Study on feature selection methods for depression detection using three-electrode EEG data[J]. Interdisciplinary Sciences:Computational Life Sciences, 2018, 10(3): 558-565. doi: 10.1007/s12539-018-0292-5
|
| [15] |
POURMOHAMMADI S, MALEKI A. Stress detection using ECG and EMG signals: a comprehensive study[J]. Computer Methods and Programs in Biomedicine, 2020, 193: 105482. doi: 10.1016/j.cmpb.2020.105482
|
| [16] |
VERKRUYSSE W, SVAASAND L O, NELSON J S. Remote plethysmographic imaging using ambient light[J]. Optics Express, 2008, 16(26): 21434-21445. doi: 10.1364/OE.16.021434
|
| [17] |
KAUR B, MOSES S, LUTHRA M, et al. Remote stress detection using a visible spectrum camera[J]. Proceedings of SPIE, 2015, 9496: 949602.
|
| [18] |
ALGHOUL K, ALHARTHI S, AL OSMAN H, et al. Heart rate variability extraction from videos signals: ICA vs. EVM comparison[J]. IEEE Access, 2017, 5: 4711-4719. doi: 10.1109/ACCESS.2017.2678521
|
| [19] |
WEI J, LUO H, WU S J, et al. Transdermal optical imaging reveal basal stress via heart rate variability analysis: a novel methodology comparable to electrocardiography[J]. Frontiers in Psychology, 2018, 9: 98. doi: 10.3389/fpsyg.2018.00098
|
| [20] |
孔令琴, 陈飞, 赵跃进, 等. 融合心率变异性与表情的非接触心理压力检测[J]. 光学学报,2021,41(3):0310003. doi: 10.3788/AOS202141.0310003
KONG L Q, CHEN F, ZHAO Y J, et al. Non-contact psychological stress detection combining heart rate variability and facial expressions[J]. Acta Optica Sinica, 2021, 41(3): 0310003. (in Chinese) doi: 10.3788/AOS202141.0310003
|
| [21] |
LISTER T, WRIGHT P A, CHAPPELL P H. Optical properties of human skin[J]. Journal of Biomedical Optics, 2012, 17(9): 090901.
|
| [22] |
孔令琴. 非接触式生理信号检测关键技术研究[D]. 北京: 北京理工大学, 2014.
KONG L Q. Research on key techniques of the noncontact detection of physiological signals[D]. Beijing: Beijing Institute of Technology, 2014. (in Chinese)
|
| [23] |
LI X B, CHEN J, ZHAO G Y, et al. . Remote heart rate measurement from face videos under realistic situations[C]. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2014: 4264-4271.
|
| [24] |
ZHAO CH CH, CAO P CH, XU SH SH, et al. . Pruning rPPG networks: toward small dense network with limited number of training samples[C]. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, IEEE, 2022: 2054-2063.
|
| [25] |
HONG J, KWON J. Optimal visual tracking using Wasserstein transport proposals[J]. Expert Systems with Applications, 2022, 209: 118251. doi: 10.1016/j.eswa.2022.118251
|
| [26] |
GRISHCHENKO I, ABLAVATSKI A, KARTYNNIK Y, et al. Attention mesh: High-fidelity face mesh prediction in real-time[J]. arXiv preprint arXiv:
|
| [27] |
BAZAREVSKY V, KARTYNNIK Y, VAKUNOV A, et al. BlazeFace: sub-millisecond neural face detection on mobile GPus[J]. arXiv preprint arXiv:
|
| [28] |
嵇晓强, 刘振瑶, 李炳霖, 等. 面部视频非接触式生理参数感知[J]. 中国光学,2022,15(2):276-285. doi: 10.37188/CO.2021-0157
JI X Q, LIU ZH Y, LI B L, et al. Non-contact perception of physiological parameters from videos of faces[J]. Chinese Optics, 2022, 15(2): 276-285. (in Chinese) doi: 10.37188/CO.2021-0157
|
| [29] |
FRACHI Y, TAKAHASHI T, WANG F Q, et al. . Design of emotion-driven game interaction using biosignals[C]. Proceedings of the 4th International Conference on Human-Computer Interaction, Springer, 2022: 160-179.
|
| [30] |
ARSALAN A, ANWAR S M, MAJID M. Mental stress detection using data from wearable and non-wearable sensors: a review[J]. arXiv preprint arXiv:
|
| [31] |
李昂, 王永刚, 邬志强, 等. 光学加工过程中高次非球面的三坐标测量数据处理[J]. 中国光学,2020,13(2):302-312. doi: 10.3788/co.20201302.0302
LI A, WANG Y G, WU ZH Q, et al. Data processing of high-order aspheric surface measurements using CMM in optical fabrication[J]. Chinese Optics, 2020, 13(2): 302-312. (in Chinese) doi: 10.3788/co.20201302.0302
|
| [32] |
TOÇOĞLU M A, ONAN A. Sentiment analysis on students’ evaluation of higher educational institutions[C]. Proceedings of the International Conference on Intelligent and Fuzzy Systems, Springer, 2020: 1693-1700.
|
| [33] |
ONAN A, KORUKOĞLU S, BULUT H. Ensemble of keyword extraction methods and classifiers in text classification[J]. Expert Systems with Applications, 2016, 57: 232-247. doi: 10.1016/j.eswa.2016.03.045
|
Figures(4) / Tables(3)
DownLoad:
