成像式光体积描记术精神压力检测

饶治; 李炳霖; 隋雅茹; 嵇晓强; 李明烨

doi:10.37188/CO.2022-0180

成像式光体积描记术精神压力检测

doi: 10.37188/CO.2022-0180

饶治^1,,
李炳霖¹,
隋雅茹¹,
嵇晓强^1, ,,
李明烨^{2, 3}

1.
长春理工大学生命科学技术学院, 吉林长春 130022
2.
皇家墨尔本理工大学信息系统和商业分析系, 维多利亚州卡尔顿 3001
3.
墨尔本大学计算与信息系统学院, 墨尔本 3053

基金项目: 吉林省科技发展计划项目（No. 20210204131YY）

详细信息

作者简介:
饶　治（1998—），男，江西九江人，硕士研究生，2021年于景德镇陶瓷大学获得学士学位,主要从事医学图像处理、机器学习方面的研究。E-mail：raozhi@mails.cust.edu.cn

嵇晓强（1982—），女，吉林德惠人, 博士，副教授，研究生导师，2012 年于中国科学院长春光学精密机械与物理研究所获得光学工程博士学位，主要从事医学信号及图像处理方面的研究。E-mail：zuoanmulan@163.com

中图分类号: TP394.1;TH691.9
计量
- 文章访问数: 582
- HTML全文浏览量: 302
- PDF下载量: 163
- 被引次数: 0
出版历程
- 收稿日期: 2022-08-10
- 修回日期: 2022-09-06
- 网络出版日期: 2022-10-31

Image photoplethysmography for mental stress detection

RAO Zhi^1
,,
LI Bing-lin¹,
SUI Ya-ru¹,
JI Xiao-qiang^{1
, ,},
LI Ming-ye^{2, 3}

1.
School of Life Science and Technology, Changchun University of Science and Technology, Changchun 130022, China
2.
Department of Information Systems and Business Analytics, RMIT University, Melbourne 3001, Australia
3.
School of Computing and Information Systems, The University of Melbourne, Melbourne 3053, Australia

Funds: Supported by Science and Technology Development Plan Project of Jilin Province (No. 20210204131YY)

More Information

Corresponding author: zuoanmulan@163.com

摘要

摘要:
为了实现非接触式的日常精神压力检测，本文提出了一种基于成像式光体积描记术的精神压力检测方法。首先，通过手机摄像头记录受试者面部视频，再采用本文所提出的基于Face Mesh的动态感兴趣区域（Region of Interest，ROI）提取方法获得心率波动引起的皮肤微弱颜色变化。接下来，将快速独立成分分析（FastICA）算法、小波变换和窄带带通滤波相结合，提取基于图像的光体积描记术信号和心率变异性信息。然后，对30名受试者进行了压力诱导实验，通过比较受试者正常和应激状态下心率变异性参数的差异，筛选了用于精神压力检测的14个特征，并探讨了压力诱导的短期精神压力和日常精神压力之间的关系。最后，另外选取67名受试者进行日常精神压力检测，使用机器学习算法建立了精神压力检测的三分类器。实验结果表明：精神压力三分类准确率达到95.2%。鉴于这种方法不需要长期测量，仅使用智能手机就可以准确检测人类精神压力水平，而且测量方法简单，测量时间短，易操作，不会影响受试者的正常心理和精神状态，因此可以作为一种有效的心理学研究工具。
- 非接触 /
- 精神压力检测 /
- 成像式光体积描记术 /
- 心率变异性 /
- 三分类
Abstract:
To achieve non-contact daily mental stress detection, this paper proposes a image photoplethysmography to detect mental stress. First, a video of the subject's face is recorded by the cell phone camera. Then, the proposed Dynamic Region of Interest (ROI) extraction method based on Face Mesh is used to obtain the weak skin color changes caused by heart rate fluctuations. Next, the Fast Independent Component Analysis (FastICA) algorithm, wavelet transform and narrowband bandpass filtering are combined to extract the signal and heart rate variability information based on image photoplethysmography. Then, stress-induced experiments are conducted on 30 subjects to screen 14 features for mental stress detection by comparing the differences in heart rate variability parameters between normal and stressful states, and to explore the relationship between short-term mental stress and daily mental stress due to stress induction. Finally, an additional 67 subjects are tested for daily mental stress, and a triple classifier for mental stress detection is built using the machine learning algorithm. The experimental results show that the accuracy of the three classifications of mental stress can reach 95.2%. Given that this method does not require long-term measurements and can accurately detect human mental stress levels using only a smartphone, and that the measurement method is simple, and easy to administer, and does not affect the normal psychological and mental state of the subject, it can be used as a valid tool in psychological research.
- non-contact /
- mental pressure detection /
- image photoplethysmography /
- heart rate variability /
- ternary classification

HTML全文

图 1 iPPG基本原理图

Figure 1. Schematic diagram of iPPG principle

下载: 全尺寸图片幻灯片

图 2 iPPG信号处理结果。（a）iPPG信号提取流程；（b）动态ROI提取过程；（c）不同ROI提取的iPPG信号；（d）R-R间期提取

Figure 2. Result of iPPG signal processing. (a) iPPG signal extraction process; (b) dynamic ROI extraction process; (c) iPPG signal extracted by different ROI; (d) extraction process in R-R interval

下载: 全尺寸图片幻灯片

图 3 压力诱导实验及结果。（a）压力诱导实验过程；（b） HRV特征参数变化箱形图

Figure 3. Stress-induced test and result. (a) Stress induction test process; (b) box diagram of HRV characteristic parameter change

下载: 全尺寸图片幻灯片

图 4 日常精神压力检测系统。（a）日常精神压力检测系统的流程；（b）不同机器学习分类器在日常精神压力检测的准确率对比；（c）基于随机森林的日常精神压力检测模型图；（d）基于随机森林的日常精神压力检测的ROC曲线

Figure 4. Daily mental stress detection system. (a) Process of daily mental stress detection system; (b) accuracy comparison of daily mental stress detection by different machine learning classifiers; (c) a random forest-based model for detecting daily mental stress; (d) ROC curves for daily mental stress detection based on random forest

下载: 全尺寸图片幻灯片

表 1 HRV特征

Table 1. HRV characteristics

HRV特征	单位	定义
HR	bmp	心率
SDNN	ms	R-R间期的标准偏差
PNN50	%	平均R-R间期大于50 ms占总数的百分比
PNN20	%	平均R-R间期大于20 ms占总数的百分比
RMSSD	ms	相邻R-R间期的不同平方的平均和的平方根
SDSD	ms	相邻R-R间期之差的标准偏差
CVSD	−	连续差异的变化系数
CVnni	−	变异系数
std_hr	bmp	心率标准差
TP	ms²	0~0.4 Hz，总光谱能量
LF_nu	%	0.04~0.15 Hz，标准化的低频功率
HF_nu	%	0.15~0.4 Hz，标准化的高频功率
LF/HF	−	低频与高频的比例
SD1	ms	垂直于方程线的直线上的Poincaré图的标准偏差
SD2	ms	沿着Poincaré图表中标记线的标准偏差
SD2/SD1	−	SD2与SD1之比
Sampen	−	数据的样本熵

下载: 导出CSV

表 2 实验描述

Table 2. Description of the test

	精神压力诱导实验	日常精神压力实验
实验对象	学生	老师和学生
数量	30	67
年龄(min/max)	17/26	19/49
身体状况	健康	健康

下载: 导出CSV

表 3 压力检测相关工作的比较

Table 3. Comparison of pressure detection-related work

	Healey^[13]	Kaur^[17]	孔^[20]	本文
实验对象	司机	志愿者	学生	学生和老师
样本数量	112	15	73	67
年龄(min/max)	−	18/50	23/28	19/49
传感器	接触式传感器	特定相机	普通相机	智能手机
特征	−	HRV(6)	HRV(9)& Expression(1)	HRV(14)
分类器	−	LDA and LR(3)	SVM(2)	RF(3)
测量方式	接触式	非接触式	非接触式	非接触式
准确率	97.4%	75.0%	81.4%	95.2%

下载: 导出CSV

参考文献(33)

[1]	MAERCKER A, BREWIN C R, BRYANT R A, et al. Proposals for mental disorders specifically associated with stress in the international classification of diseases-11[J]. The Lancet, 2013, 381(9878): 1683-1685. doi: 10.1016/S0140-6736(12)62191-6
[2]	SHAH S M A, MOHAMMAD D, QURESHI M F H, et al. Prevalence, psychological responses and associated correlates of depression, anxiety and stress in a global population, during the coronavirus disease (COVID-19) pandemic[J]. Community Mental Health Journal, 2021, 57(1): 101-110. doi: 10.1007/s10597-020-00728-y
[3]	ESLER M. Mental stress and human cardiovascular disease[J]. Neuroscience &Biobehavioral Reviews, 2017, 74: 269-276.
[4]	TOMIYAMA A J. Stress and obesity[J]. Annual Review of Psychology, 2019, 70: 703-718. doi: 10.1146/annurev-psych-010418-102936
[5]	MIRANDA A R, SCOTTA A V, MÉNDEZ A L, et al. Public sector workers’ mental health in Argentina: Comparative psychometrics of the perceived stress scale[J]. Journal of Preventive Medicine &Public Health, 2020, 53(6): 429-438.
[6]	LIMA B B, HAMMADAH M, PEARCE B D, et al. Association of posttraumatic stress disorder with mental stress–induced myocardial ischemia in adults after myocardial infarction[J]. JAMA Network Open, 2020, 3(4): e202734. doi: 10.1001/jamanetworkopen.2020.2734
[7]	CAMM A J, MALIK M, BIGGER J T, et al. Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Task force of the European society of cardiology and the North American society of pacing and electrophysiology[J]. Circulation, 1996, 93(5): 1043-1065. doi: 10.1161/01.CIR.93.5.1043
[8]	PHAM T, LAU Z J, CHEN S H A, et al. Heart rate variability in psychology: a review of HRV indices and an analysis tutorial[J]. Sensors, 2021, 21(12): 3998. doi: 10.3390/s21123998
[9]	GEDAM S, PAUL S. A review on mental stress detection using wearable sensors and machine learning techniques[J]. IEEE Access, 2021, 9: 84045-84066. doi: 10.1109/ACCESS.2021.3085502
[10]	ZENG ZH K, HUANG ZH, LENG K M, et al. Nonintrusive monitoring of mental fatigue status using epidermal electronic systems and machine-learning algorithms[J]. ACS Sensors, 2020, 5(5): 1305-1313. doi: 10.1021/acssensors.9b02451
[11]	CHARLTON P H, CELKA P, FARUKH B, et al. Assessing mental stress from the photoplethysmogram: a numerical study[J]. Physiological Measurement, 2018, 39(5): 054001. doi: 10.1088/1361-6579/aabe6a
[12]	CASTALDO R, MELILLO P, BRACALE U, et al. Acute mental stress assessment via short term HRV analysis in healthy adults: a systematic review with meta-analysis[J]. Biomedical Signal Processing and Control, 2015, 18: 370-377. doi: 10.1016/j.bspc.2015.02.012
[13]	HEALEY J A, PICARD R W. Detecting stress during real-world driving tasks using physiological sensors[J]. IEEE Transactions on Intelligent Transportation Systems, 2005, 6(2): 156-166. doi: 10.1109/TITS.2005.848368
[14]	CAI H SH, CHEN Y F, HAN J SH, et al. Study on feature selection methods for depression detection using three-electrode EEG data[J]. Interdisciplinary Sciences:Computational Life Sciences, 2018, 10(3): 558-565. doi: 10.1007/s12539-018-0292-5
[15]	POURMOHAMMADI S, MALEKI A. Stress detection using ECG and EMG signals: a comprehensive study[J]. Computer Methods and Programs in Biomedicine, 2020, 193: 105482. doi: 10.1016/j.cmpb.2020.105482
[16]	VERKRUYSSE W, SVAASAND L O, NELSON J S. Remote plethysmographic imaging using ambient light[J]. Optics Express, 2008, 16(26): 21434-21445. doi: 10.1364/OE.16.021434
[17]	KAUR B, MOSES S, LUTHRA M, et al. Remote stress detection using a visible spectrum camera[J]. Proceedings of SPIE, 2015, 9496: 949602.
[18]	ALGHOUL K, ALHARTHI S, AL OSMAN H, et al. Heart rate variability extraction from videos signals: ICA vs. EVM comparison[J]. IEEE Access, 2017, 5: 4711-4719. doi: 10.1109/ACCESS.2017.2678521
[19]	WEI J, LUO H, WU S J, et al. Transdermal optical imaging reveal basal stress via heart rate variability analysis: a novel methodology comparable to electrocardiography[J]. Frontiers in Psychology, 2018, 9: 98. doi: 10.3389/fpsyg.2018.00098
[20]	孔令琴, 陈飞, 赵跃进, 等. 融合心率变异性与表情的非接触心理压力检测[J]. 光学学报,2021,41(3):0310003. doi: 10.3788/AOS202141.0310003 KONG L Q, CHEN F, ZHAO Y J, et al. Non-contact psychological stress detection combining heart rate variability and facial expressions[J]. Acta Optica Sinica, 2021, 41(3): 0310003. (in Chinese) doi: 10.3788/AOS202141.0310003
[21]	LISTER T, WRIGHT P A, CHAPPELL P H. Optical properties of human skin[J]. Journal of Biomedical Optics, 2012, 17(9): 090901.
[22]	孔令琴. 非接触式生理信号检测关键技术研究[D]. 北京: 北京理工大学, 2014. KONG L Q. Research on key techniques of the noncontact detection of physiological signals[D]. Beijing: Beijing Institute of Technology, 2014. (in Chinese)
[23]	LI X B, CHEN J, ZHAO G Y, et al. . Remote heart rate measurement from face videos under realistic situations[C]. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2014: 4264-4271.
[24]	ZHAO CH CH, CAO P CH, XU SH SH, et al. . Pruning rPPG networks: toward small dense network with limited number of training samples[C]. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, IEEE, 2022: 2054-2063.
[25]	HONG J, KWON J. Optimal visual tracking using Wasserstein transport proposals[J]. Expert Systems with Applications, 2022, 209: 118251. doi: 10.1016/j.eswa.2022.118251
[26]	GRISHCHENKO I, ABLAVATSKI A, KARTYNNIK Y, et al. Attention mesh: High-fidelity face mesh prediction in real-time[J]. arXiv preprint arXiv:, 2006, 10962: 2020.
[27]	BAZAREVSKY V, KARTYNNIK Y, VAKUNOV A, et al. BlazeFace: sub-millisecond neural face detection on mobile GPus[J]. arXiv preprint arXiv:, 1907, 05047: 2019.
[28]	嵇晓强, 刘振瑶, 李炳霖, 等. 面部视频非接触式生理参数感知[J]. 中国光学,2022,15(2):276-285. doi: 10.37188/CO.2021-0157 JI X Q, LIU ZH Y, LI B L, et al. Non-contact perception of physiological parameters from videos of faces[J]. Chinese Optics, 2022, 15(2): 276-285. (in Chinese) doi: 10.37188/CO.2021-0157
[29]	FRACHI Y, TAKAHASHI T, WANG F Q, et al. . Design of emotion-driven game interaction using biosignals[C]. Proceedings of the 4th International Conference on Human-Computer Interaction, Springer, 2022: 160-179.
[30]	ARSALAN A, ANWAR S M, MAJID M. Mental stress detection using data from wearable and non-wearable sensors: a review[J]. arXiv preprint arXiv:, 2202, 03033: 2022.
[31]	李昂, 王永刚, 邬志强, 等. 光学加工过程中高次非球面的三坐标测量数据处理[J]. 中国光学,2020,13(2):302-312. doi: 10.3788/co.20201302.0302 LI A, WANG Y G, WU ZH Q, et al. Data processing of high-order aspheric surface measurements using CMM in optical fabrication[J]. Chinese Optics, 2020, 13(2): 302-312. (in Chinese) doi: 10.3788/co.20201302.0302
[32]	TOÇOĞLU M A, ONAN A. Sentiment analysis on students’ evaluation of higher educational institutions[C]. Proceedings of the International Conference on Intelligent and Fuzzy Systems, Springer, 2020: 1693-1700.
[33]	ONAN A, KORUKOĞLU S, BULUT H. Ensemble of keyword extraction methods and classifiers in text classification[J]. Expert Systems with Applications, 2016, 57: 232-247. doi: 10.1016/j.eswa.2016.03.045