留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

成像式光体积描记术精神压力检测

饶治 李炳霖 隋雅茹 嵇晓强 李明烨

饶治, 李炳霖, 隋雅茹, 嵇晓强, 李明烨. 成像式光体积描记术精神压力检测[J]. 中国光学(中英文), 2022, 15(6): 1350-1359. doi: 10.37188/CO.2022-0180
引用本文: 饶治, 李炳霖, 隋雅茹, 嵇晓强, 李明烨. 成像式光体积描记术精神压力检测[J]. 中国光学(中英文), 2022, 15(6): 1350-1359. doi: 10.37188/CO.2022-0180
RAO Zhi, LI Bing-lin, SUI Ya-ru, JI Xiao-qiang, LI Ming-ye. Image photoplethysmography for mental stress detection[J]. Chinese Optics, 2022, 15(6): 1350-1359. doi: 10.37188/CO.2022-0180
Citation: RAO Zhi, LI Bing-lin, SUI Ya-ru, JI Xiao-qiang, LI Ming-ye. Image photoplethysmography for mental stress detection[J]. Chinese Optics, 2022, 15(6): 1350-1359. doi: 10.37188/CO.2022-0180

成像式光体积描记术精神压力检测

基金项目: 吉林省科技发展计划项目(No. 20210204131YY)
详细信息
    作者简介:

    饶 治(1998—),男,江西九江人, 硕士研究生,2021年于景德镇陶瓷大学获得学士学位,主要从事医学图像处理、机器学习方面的研究。E-mail:raozhi@mails.cust.edu.cn

    嵇晓强(1982—),女,吉林德惠人, 博士,副教授,研究生导师,2012 年于中国科学院长春光学精密机械与物理研究所获得光学工程博士学位,主要从事医学信号及图像处理方面的研究。E-mail:zuoanmulan@163.com

  • 中图分类号: TP394.1;TH691.9

Image photoplethysmography for mental stress detection

Funds: Supported by Science and Technology Development Plan Project of Jilin Province (No. 20210204131YY)
More Information
  • 摘要:

    为了实现非接触式的日常精神压力检测,本文提出了一种基于成像式光体积描记术的精神压力检测方法。首先,通过手机摄像头记录受试者面部视频,再采用本文所提出的基于Face Mesh的动态感兴趣区域(Region of Interest,ROI)提取方法获得心率波动引起的皮肤微弱颜色变化。接下来,将快速独立成分分析(FastICA)算法、小波变换和窄带带通滤波相结合,提取基于图像的光体积描记术信号和心率变异性信息。然后,对30名受试者进行了压力诱导实验,通过比较受试者正常和应激状态下心率变异性参数的差异,筛选了用于精神压力检测的14个特征,并探讨了压力诱导的短期精神压力和日常精神压力之间的关系。最后,另外选取67名受试者进行日常精神压力检测,使用机器学习算法建立了精神压力检测的三分类器。实验结果表明:精神压力三分类准确率达到95.2%。鉴于这种方法不需要长期测量,仅使用智能手机就可以准确检测人类精神压力水平,而且测量方法简单,测量时间短,易操作,不会影响受试者的正常心理和精神状态,因此可以作为一种有效的心理学研究工具。

     

  • 图 1  iPPG基本原理图

    Figure 1.  Schematic diagram of iPPG principle

    图 2  iPPG信号处理结果。(a)iPPG信号提取流程;(b)动态ROI提取过程;(c)不同ROI提取的iPPG信号;(d)R-R间期提取

    Figure 2.  Result of iPPG signal processing. (a) iPPG signal extraction process; (b) dynamic ROI extraction process; (c) iPPG signal extracted by different ROI; (d) extraction process in R-R interval

    图 3  压力诱导实验及结果。(a) 压力诱导实验过程;(b) HRV特征参数变化箱形图

    Figure 3.  Stress-induced test and result. (a) Stress induction test process; (b) box diagram of HRV characteristic parameter change

    图 4  日常精神压力检测系统。(a)日常精神压力检测系统的流程;(b)不同机器学习分类器在日常精神压力检测的准确率对比;(c)基于随机森林的日常精神压力检测模型图;(d)基于随机森林的日常精神压力检测的ROC曲线

    Figure 4.  Daily mental stress detection system. (a) Process of daily mental stress detection system; (b) accuracy comparison of daily mental stress detection by different machine learning classifiers; (c) a random forest-based model for detecting daily mental stress; (d) ROC curves for daily mental stress detection based on random forest

    表  1  HRV特征

    Table  1.   HRV characteristics

    HRV特征单位定义
    HRbmp心率
    SDNNmsR-R间期的标准偏差
    PNN50%平均R-R间期大于50 ms占总数的百分比
    PNN20%平均R-R间期大于20 ms占总数的百分比
    RMSSDms相邻R-R间期的不同平方的平均和的平方根
    SDSDms相邻R-R间期之差的标准偏差
    CVSD连续差异的变化系数
    CVnni变异系数
    std_hrbmp心率标准差
    TPms20~0.4 Hz,总光谱能量
    LFnu%0.04~0.15 Hz,标准化的低频功率
    HFnu%0.15~0.4 Hz,标准化的高频功率
    LF/HF低频与高频的比例
    SD1ms垂直于方程线的直线上的Poincaré图的标准偏差
    SD2ms沿着Poincaré图表中标记线的标准偏差
    SD2/SD1SD2与SD1之比
    Sampen数据的样本熵
    下载: 导出CSV

    表  2  实验描述

    Table  2.   Description of the test

    精神
    压力诱导实验
    日常
    精神压力实验
    实验对象学生老师和学生
    数量3067
    年龄(min/max)17/2619/49
    身体状况健康健康
    下载: 导出CSV

    表  3  压力检测相关工作的比较

    Table  3.   Comparison of pressure detection-related work

    Healey[13]Kaur[17][20]本文
    实验对象司机志愿者学生学生和老师
    样本数量112157367
    年龄(min/max)18/5023/2819/49
    传感器接触式
    传感器
    特定相机普通相机智能手机
    特征HRV(6)HRV(9)&
    Expression(1)
    HRV(14)
    分类器LDA and LR(3)SVM(2)RF(3)
    测量方式接触式非接触式非接触式非接触式
    准确率97.4%75.0%81.4%95.2%
    下载: 导出CSV
  • [1] MAERCKER A, BREWIN C R, BRYANT R A, et al. Proposals for mental disorders specifically associated with stress in the international classification of diseases-11[J]. The Lancet, 2013, 381(9878): 1683-1685. doi: 10.1016/S0140-6736(12)62191-6
    [2] SHAH S M A, MOHAMMAD D, QURESHI M F H, et al. Prevalence, psychological responses and associated correlates of depression, anxiety and stress in a global population, during the coronavirus disease (COVID-19) pandemic[J]. Community Mental Health Journal, 2021, 57(1): 101-110. doi: 10.1007/s10597-020-00728-y
    [3] ESLER M. Mental stress and human cardiovascular disease[J]. Neuroscience &Biobehavioral Reviews, 2017, 74: 269-276.
    [4] TOMIYAMA A J. Stress and obesity[J]. Annual Review of Psychology, 2019, 70: 703-718. doi: 10.1146/annurev-psych-010418-102936
    [5] MIRANDA A R, SCOTTA A V, MÉNDEZ A L, et al. Public sector workers’ mental health in Argentina: Comparative psychometrics of the perceived stress scale[J]. Journal of Preventive Medicine &Public Health, 2020, 53(6): 429-438.
    [6] LIMA B B, HAMMADAH M, PEARCE B D, et al. Association of posttraumatic stress disorder with mental stress–induced myocardial ischemia in adults after myocardial infarction[J]. JAMA Network Open, 2020, 3(4): e202734. doi: 10.1001/jamanetworkopen.2020.2734
    [7] CAMM A J, MALIK M, BIGGER J T, et al. Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Task force of the European society of cardiology and the North American society of pacing and electrophysiology[J]. Circulation, 1996, 93(5): 1043-1065. doi: 10.1161/01.CIR.93.5.1043
    [8] PHAM T, LAU Z J, CHEN S H A, et al. Heart rate variability in psychology: a review of HRV indices and an analysis tutorial[J]. Sensors, 2021, 21(12): 3998. doi: 10.3390/s21123998
    [9] GEDAM S, PAUL S. A review on mental stress detection using wearable sensors and machine learning techniques[J]. IEEE Access, 2021, 9: 84045-84066. doi: 10.1109/ACCESS.2021.3085502
    [10] ZENG ZH K, HUANG ZH, LENG K M, et al. Nonintrusive monitoring of mental fatigue status using epidermal electronic systems and machine-learning algorithms[J]. ACS Sensors, 2020, 5(5): 1305-1313. doi: 10.1021/acssensors.9b02451
    [11] CHARLTON P H, CELKA P, FARUKH B, et al. Assessing mental stress from the photoplethysmogram: a numerical study[J]. Physiological Measurement, 2018, 39(5): 054001. doi: 10.1088/1361-6579/aabe6a
    [12] CASTALDO R, MELILLO P, BRACALE U, et al. Acute mental stress assessment via short term HRV analysis in healthy adults: a systematic review with meta-analysis[J]. Biomedical Signal Processing and Control, 2015, 18: 370-377. doi: 10.1016/j.bspc.2015.02.012
    [13] HEALEY J A, PICARD R W. Detecting stress during real-world driving tasks using physiological sensors[J]. IEEE Transactions on Intelligent Transportation Systems, 2005, 6(2): 156-166. doi: 10.1109/TITS.2005.848368
    [14] CAI H SH, CHEN Y F, HAN J SH, et al. Study on feature selection methods for depression detection using three-electrode EEG data[J]. Interdisciplinary Sciences:Computational Life Sciences, 2018, 10(3): 558-565. doi: 10.1007/s12539-018-0292-5
    [15] POURMOHAMMADI S, MALEKI A. Stress detection using ECG and EMG signals: a comprehensive study[J]. Computer Methods and Programs in Biomedicine, 2020, 193: 105482. doi: 10.1016/j.cmpb.2020.105482
    [16] VERKRUYSSE W, SVAASAND L O, NELSON J S. Remote plethysmographic imaging using ambient light[J]. Optics Express, 2008, 16(26): 21434-21445. doi: 10.1364/OE.16.021434
    [17] KAUR B, MOSES S, LUTHRA M, et al. Remote stress detection using a visible spectrum camera[J]. Proceedings of SPIE, 2015, 9496: 949602.
    [18] ALGHOUL K, ALHARTHI S, AL OSMAN H, et al. Heart rate variability extraction from videos signals: ICA vs. EVM comparison[J]. IEEE Access, 2017, 5: 4711-4719. doi: 10.1109/ACCESS.2017.2678521
    [19] WEI J, LUO H, WU S J, et al. Transdermal optical imaging reveal basal stress via heart rate variability analysis: a novel methodology comparable to electrocardiography[J]. Frontiers in Psychology, 2018, 9: 98. doi: 10.3389/fpsyg.2018.00098
    [20] 孔令琴, 陈飞, 赵跃进, 等. 融合心率变异性与表情的非接触心理压力检测[J]. 光学学报,2021,41(3):0310003. doi: 10.3788/AOS202141.0310003

    KONG L Q, CHEN F, ZHAO Y J, et al. Non-contact psychological stress detection combining heart rate variability and facial expressions[J]. Acta Optica Sinica, 2021, 41(3): 0310003. (in Chinese) doi: 10.3788/AOS202141.0310003
    [21] LISTER T, WRIGHT P A, CHAPPELL P H. Optical properties of human skin[J]. Journal of Biomedical Optics, 2012, 17(9): 090901.
    [22] 孔令琴. 非接触式生理信号检测关键技术研究[D]. 北京: 北京理工大学, 2014.

    KONG L Q. Research on key techniques of the noncontact detection of physiological signals[D]. Beijing: Beijing Institute of Technology, 2014. (in Chinese)
    [23] LI X B, CHEN J, ZHAO G Y, et al. . Remote heart rate measurement from face videos under realistic situations[C]. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2014: 4264-4271.
    [24] ZHAO CH CH, CAO P CH, XU SH SH, et al. . Pruning rPPG networks: toward small dense network with limited number of training samples[C]. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, IEEE, 2022: 2054-2063.
    [25] HONG J, KWON J. Optimal visual tracking using Wasserstein transport proposals[J]. Expert Systems with Applications, 2022, 209: 118251. doi: 10.1016/j.eswa.2022.118251
    [26] GRISHCHENKO I, ABLAVATSKI A, KARTYNNIK Y, et al. Attention mesh: High-fidelity face mesh prediction in real-time[J]. arXiv preprint arXiv:, 2006, 10962: 2020.
    [27] BAZAREVSKY V, KARTYNNIK Y, VAKUNOV A, et al. BlazeFace: sub-millisecond neural face detection on mobile GPus[J]. arXiv preprint arXiv:, 1907, 05047: 2019.
    [28] 嵇晓强, 刘振瑶, 李炳霖, 等. 面部视频非接触式生理参数感知[J]. 中国光学,2022,15(2):276-285. doi: 10.37188/CO.2021-0157

    JI X Q, LIU ZH Y, LI B L, et al. Non-contact perception of physiological parameters from videos of faces[J]. Chinese Optics, 2022, 15(2): 276-285. (in Chinese) doi: 10.37188/CO.2021-0157
    [29] FRACHI Y, TAKAHASHI T, WANG F Q, et al. . Design of emotion-driven game interaction using biosignals[C]. Proceedings of the 4th International Conference on Human-Computer Interaction, Springer, 2022: 160-179.
    [30] ARSALAN A, ANWAR S M, MAJID M. Mental stress detection using data from wearable and non-wearable sensors: a review[J]. arXiv preprint arXiv:, 2202, 03033: 2022.
    [31] 李昂, 王永刚, 邬志强, 等. 光学加工过程中高次非球面的三坐标测量数据处理[J]. 中国光学,2020,13(2):302-312. doi: 10.3788/co.20201302.0302

    LI A, WANG Y G, WU ZH Q, et al. Data processing of high-order aspheric surface measurements using CMM in optical fabrication[J]. Chinese Optics, 2020, 13(2): 302-312. (in Chinese) doi: 10.3788/co.20201302.0302
    [32] TOÇOĞLU M A, ONAN A. Sentiment analysis on students’ evaluation of higher educational institutions[C]. Proceedings of the International Conference on Intelligent and Fuzzy Systems, Springer, 2020: 1693-1700.
    [33] ONAN A, KORUKOĞLU S, BULUT H. Ensemble of keyword extraction methods and classifiers in text classification[J]. Expert Systems with Applications, 2016, 57: 232-247. doi: 10.1016/j.eswa.2016.03.045
  • 加载中
图(4) / 表(3)
计量
  • 文章访问数:  567
  • HTML全文浏览量:  297
  • PDF下载量:  161
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-08-10
  • 修回日期:  2022-09-06
  • 网络出版日期:  2022-10-31

目录

    /

    返回文章
    返回