留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

面部视频非接触式生理参数感知

嵇晓强 刘振瑶 李炳霖 饶治 李贵文 粟立威

嵇晓强, 刘振瑶, 李炳霖, 饶治, 李贵文, 粟立威. 面部视频非接触式生理参数感知[J]. 中国光学(中英文), 2022, 15(2): 276-285. doi: 10.37188/CO.2021-0157
引用本文: 嵇晓强, 刘振瑶, 李炳霖, 饶治, 李贵文, 粟立威. 面部视频非接触式生理参数感知[J]. 中国光学(中英文), 2022, 15(2): 276-285. doi: 10.37188/CO.2021-0157
JI Xiao-qiang, LIU Zhen-yao, LI Bing-lin, RAO Zhi, LI Gui-wen, SU Li-wei. Non-contact perception of physiological parameters from videos of faces[J]. Chinese Optics, 2022, 15(2): 276-285. doi: 10.37188/CO.2021-0157
Citation: JI Xiao-qiang, LIU Zhen-yao, LI Bing-lin, RAO Zhi, LI Gui-wen, SU Li-wei. Non-contact perception of physiological parameters from videos of faces[J]. Chinese Optics, 2022, 15(2): 276-285. doi: 10.37188/CO.2021-0157

面部视频非接触式生理参数感知

doi: 10.37188/CO.2021-0157
基金项目: 吉林省科技发展计划项目(No. 20210204131YY)
详细信息
    作者简介:

    嵇晓强(1982—),女,吉林德惠人,博士,副教授,硕士生导师,2012 年于中国科学院长春光学精密机械与物理研究所获得光学工程博士学位,主要从事医学信号及图像处理方面的研究。E-mail: zuoanmulan@163.com

    刘振瑶(1997—),女,山东济宁人,硕士研究生,2019年于长春理工大学获得学士学位,主要从事医学信号及图像处理方面的研究。E-mail: 1011510854@qq.com

  • 中图分类号: TN911.7;TP391

Non-contact perception of physiological parameters from videos of faces

Funds: Supported by Department of Science and Technology of Natural Science Foundation of Jilin Province under grant number (No. 20210204131YY)
More Information
  • 摘要: 为了在非接触条件下检测受试者的各项生理参数,本文设计了一种基于成像式光电容积描记技术,从手机录制的人脸视频中估算生理参数的方法。首先,提出了“小波变换-主成分分析-盲源分离”算法,用于提取出高信噪比的RGB三通道脉搏波信号。然后,分别从频域和时域角度对绿色通道信号进行处理,估算出心率值和呼吸率值;对红蓝通道的脉搏波信号进行处理,并结合血氧仪检测的血氧饱和度结果,进行数据拟合,从而找到从面部视频中估算血氧饱和度值的最佳线性方程。最后,对比了自然光下各生理参数的估算结果误差,分析了在3种光照环境下各参数的估算结果。结果表明:3种光照环境下得到的心率平均误差为0.5512次/min,呼吸率平均误差为−0.6321次/min,血氧饱和度平均误差为−0.2743%。综上,本文提出的非接触式生理参数估算方法精度高,具有普适性和稳定性,估算结果同标准仪器的测量结果具有高度一致性,可满足日常生理参数测量的需求。

     

  • 图 1  视频图像处理流程图

    Figure 1.  Flow chart of video image processing

    图 2  放大前、后IPPG源信号对比

    Figure 2.  Comparison of IPPG source signals before and after amplification

    图 3  IPPG源信号处理流程图

    Figure 3.  Flow chart of IPPG source signal processing

    图 4  PCA降维后各成分信号

    Figure 4.  The signal of each component after PCA dimensionality reduction

    图 5  盲源分离出的独立源信号

    Figure 5.  Independent source signal separated by a blind source

    图 6  经带通滤波后的脉搏波信号

    Figure 6.  Pulse wave signal after bandpass filtering

    图 7  傅立叶变换频谱图

    Figure 7.  Fourier transform spectrogram

    图 8  滤波后呼吸信号

    Figure 8.  Respiration signal after filtering

    图 9  实验采集示意图及装置图

    Figure 9.  Schematic diagram and real diagram of the experimental acquisition device

    图 10  3种场景下心率估算结果比较

    Figure 10.  Comparison of heart rate estimation results in three scenarios

    图 11  3种场景下心率结果Bland-Altman 一致性分析

    Figure 11.  Bland-Altman consistency analysis of heart rate results in three scenarios

    图 12  3种场景下呼吸率估算结果比较

    Figure 12.  Comparison of respiratory rate estimation results in three scenarios

    图 13  3种场景下呼吸率结果Bland-Altman 一致性分析

    Figure 13.  Bland-Altman consistency analysis of respiratory rate results in three scenarios

    图 14  3种场景下SpO2估算结果比较

    Figure 14.  Comparison of SpO2 estimation results in three scenarios

    图 15  3种场景下SpO2结果Bland–Altman 一致性分析

    Figure 15.  Bland-Altman consistency analysis of SpO2 results in three scenarios

    表  1  自然光下各心率检测算法性能比较

    Table  1.   Performance comparison of various heart rate detection algorithms under natural light

    方法|Me|
    (time·min−1)
    SDe
    (time·min−1)
    RMSE
    (time·min−1)
    MerCor
    文献[6]1.451.942.061.95%0.9278
    文献[12]2.393.563.383.05%
    本文方法1.781.771.982.54%0.9668
    下载: 导出CSV

    表  2  自然光下各呼吸率检测算法性能比较

    Table  2.   Performance comparison of various respiratory rate detection algorithms under natural light

    方法Me
    (time·min−1)
    |Me|
    (time·min−1)
    SDe
    (time·min−1)
    RMSE
    (time/min−1)
    Cor
    文献[10]−0.582.543.984.020.61
    本文方法0.631.781.881.980.60
    下载: 导出CSV

    表  3  自然光下各SpO2检测算法性能比较

    Table  3.   Performance comparison of SpO2 detection algorithms under natural light

    方法MeSDeRMSEMer
    文献[8]0.043%1.10%2.02%2.7%
    文献[11]1.00%1.32%0.87%
    本文方法−0.27%1.05%1.08%0.82%
    下载: 导出CSV
  • [1] TULPPO M P, KIVINIEMI A M, JUNTTILA M J, et al. Home monitoring of heart rate as a predictor of imminent cardiovascular events[J]. Frontiers in Physiology, 2019, 10: 341. doi: 10.3389/fphys.2019.00341
    [2] KAUSER M A, SAFIUDDIN M. Heart rate-an emerging cardiovascular risk factor[J]. University Heart Journal, 2012, 8(1): 30-35. doi: 10.3329/uhj.v8i1.11665
    [3] MATHER M, THAYER J F. How heart rate variability affects emotion regulation brain networks[J]. Current Opinion in Behavioral Sciences, 2018, 19: 98-104. doi: 10.1016/j.cobeha.2017.12.017
    [4] 孙莉娜, 李䶮, 郭汉涛, 等. 氮、铁共掺杂碳纳米粒子的制备及在过氧化氢和葡萄糖检测中的应用[J]. 应用化学,2020,37(3):350-358.

    SUN L N, LI Y, GUO H T, et al. Preparation of nitrogen and iron co-doped carbon nanoparticles and their applications in detection of hydrogen peroxide and glucose[J]. Chinese Journal of Applied Chemistry, 2020, 37(3): 350-358. (in Chinese)
    [5] TARASSENKO L, VILLARROEL M, GUAZZI A, et al. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models[J]. Physiological Measurement, 2014, 35(5): 807-831. doi: 10.1088/0967-3334/35/5/807
    [6] BAL U. Non-contact estimation of heart rate and oxygen saturation using ambient light[J]. Biomedical Optics Express, 2015, 6(1): 86-97. doi: 10.1364/BOE.6.000086
    [7] CASALINO G, CASTELLANO G, PASQUADIBISCEGLIE V, et al. Contact-less real-time monitoring of cardiovascular risk using video imaging and fuzzy inference rules[J]. Information, 2019, 10(1): 9.
    [8] 荣猛, 范强, 李凯扬. 基于IPPG非接触式生理参数测量算法的研究[J]. 生物医学工程研究,2018,37(1):27-31,35.

    RONG M, FAN Q, LI K Y. Study on the measurement algorithm of contactless physiological parameter based on imaging photoplenthysmography[J]. Journal of Biomedical Engineering Research, 2018, 37(1): 27-31,35. (in Chinese)
    [9] 李晓媛, 武鹏, 刘允, 等. 基于人脸视频的心率参数提取[J]. 光学 精密工程,2020,28(3):548-557. doi: 10.3788/OPE.20202803.0548

    LI X Y, WU P, LIU Y, et al. Extraction of heart rate parameters from video of human face[J]. Optics and Precision Engineering, 2020, 28(3): 548-557. (in Chinese) doi: 10.3788/OPE.20202803.0548
    [10] LUGUERN D, MACWAN R, BENEZETH Y, et al. Wavelet variance maximization: a contactless respiration rate estimation method based on remote photoplethysmography[J]. Biomedical Signal Processing and Control, 2021, 63: 102263. doi: 10.1016/j.bspc.2020.102263
    [11] RAHMAN H, AHMED M U, BEGUM S. Non-contact physiological parameters extraction using facial video considering illumination, motion, movement and vibration[J]. IEEE Transactions on Biomedical Engineering, 2020, 67(1): 88-98. doi: 10.1109/TBME.2019.2908349
    [12] 戴阳, 郑婷婷, 杨雪. 基于视频放大与盲源分离的非接触式心率检测[J]. 计算机系统应用,2021,30(1):228-234.

    DAI Y, ZHENG T T, YANG X. Non-contact heart rate detection based on video amplification and blind source separation[J]. Computer Systems &Applications, 2021, 30(1): 228-234. (in Chinese)
    [13] CHEN Q, JIANG X Y, LIU X Y, et al.. Non-contact heart rate monitoring in neonatal intensive care unit using RGB camera[C]. Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, IEEE, 2020: 5822-5825.
    [14] WU H Y, RUBINSTEIN M, SHIH E, et al. Eulerian video magnification for revealing subtle changes in the world[J]. ACM Transactions on Graphics, 2012, 31(4): 65.
    [15] 田昕, 赖翰文, 刘亚栋, 等. 基于卷积运算的嵌段共聚物自组装图像缺陷分析统计方法[J]. 应用化学,2021,38(9):1199-1208.

    TIAN X, LAI H W, LIU Y D, et al. Analysis of defects in block copolymer films by a convolution algorithm[J]. Chinese Journal of Applied Chemistry, 2021, 38(9): 1199-1208. (in Chinese)
    [16] LIU J G, LUO H, ZHENG P P. et al. Transdermal optical imaging revealed different spatiotemporal patterns of facial cardiovascular activities[J]. Scientific Reports, 2018, 8(1): 10588. doi: 10.1038/s41598-018-28804-0
    [17] 彭仁杰, 姚云霞. 小波分解和重构的光学可变图像融合研究[J]. 激光杂志,2021,42(3):145-148.

    PENG R J, YAO Y X. Research on optical variable image fusion based on wavelet decomposition and reconstruction[J]. Laser Journal, 2021, 42(3): 145-148. (in Chinese)
    [18] YING L, ZHANG F B, ZHANG Y L, et al. Identifying distributed dynamic loading in one spatial dimension based on combing wavelet decomposition and Kalman filter with unknown input[J]. Journal of Aerospace Engineering, 2021, 34(4): 04021025. doi: 10.1061/(ASCE)AS.1943-5525.0001265
    [19] SONG K, ZHANG B Y, LI W, et al. Research on parallel principal component analysis based on ternary optical computer[J]. Optik, 2021, 241: 167176. doi: 10.1016/j.ijleo.2021.167176
    [20] 张申华, 杨延西, 秦峤孟. 针对光栅图像的快速盲去噪方法[J]. 中国光学,2021,14(3):596-604. doi: 10.37188/CO.2020-0166

    ZHANG SH H, YANG Y X, QIN Q M. A fast blind denoising method for grating image[J]. Chinese Optics, 2021, 14(3): 596-604. (in Chinese) doi: 10.37188/CO.2020-0166
    [21] 邱晓华, 李敏, 邓光芒, 等. 多层卷积特征融合的双波段决策级船舶识别[J]. 光学 精密工程,2021,29(1):183-190. doi: 10.37188/OPE.20212901.0183

    QIU X H, LI M, DENG G M, et al. Multi-layer convolutional features fusion for dual-band decision-level ship recognition[J]. Optics and Precision Engineering, 2021, 29(1): 183-190. (in Chinese) doi: 10.37188/OPE.20212901.0183
    [22] VERKRUYSSE W, SVAASAND L O, NELSON J S. Remote plethysmographic imaging using ambient light[J]. Optics Express, 2008, 16(26): 21434-21445. doi: 10.1364/OE.16.021434
    [23] 乔闹生, 孙萍. CCD非线性效应对双频光栅三维面形测量的影响[J]. 中国光学,2021,14(3):661-669. doi: 10.37188/CO.2020-0143

    QIAO N SH, SUN P. Influence of CCD nonlinearity effect on the three-dimensional shape measurement of dual frequency grating[J]. Chinese Optics, 2021, 14(3): 661-669. (in Chinese) doi: 10.37188/CO.2020-0143
    [24] 赵云, 吕金光, 秦余欣, 等. 微型傅立叶变换光谱仪的优化设计与实验研究[J]. 中国光学,2020,13(2):411-425. doi: 10.3788/co.20201302.0411

    ZHAO Y, LV J G, QIN Y X, et al. Optimization design and experimental study of micro-Fourier transform spectrometer[J]. Chinese Optics, 2020, 13(2): 411-425. (in Chinese) doi: 10.3788/co.20201302.0411
    [25] 孙学辉, 赵冰, 骆震, 等. 离散傅立叶变换用于非连续工业数据分析[J]. 分析化学,2020,48(10):1422-1427.

    SUN X H, ZHAO B, TUO Z, et al. Non-continuous industrial data Analysis using discrete fourier fransform[J]. Chinese Journal of Analytical Chemistry, 2020, 48(10): 1422-1427. (in Chinese)
    [26] 刘今越, 刘浩, 贾晓辉, 等. 基于视觉的非接触呼吸频率自动检测方法[J]. 仪器仪表学报,2019,40(2):51-58.

    LIU J Y, LIU H, JIA X H, et al. Vision-based automatic detection method for non-contact respiratory rate[J]. Chinese Journal of Scientific Instrument, 2019, 40(2): 51-58. (in Chinese)
    [27] 任繁栋, 丁筱雪, 蔡芳, 等. 基于超高效液相色谱-高分辨质谱联用技术研究冠心病及冠心病合并2型糖尿病患者代谢特征[J]. 分析化学,2020,48(1):49-56.

    REN F D, DING X X, CAI F, et al. Investigation of metabolic features of patients with coronary heart disease or coronary heart disease-type 2 diabetes mellitus based on ultra-high performance liquid chromatography high resolution mass spectrometry[J]. Chinese Journal of Analytical Chemistry, 2020, 48(1): 49-56. (in Chinese)
  • 加载中
图(15) / 表(3)
计量
  • 文章访问数:  1223
  • HTML全文浏览量:  756
  • PDF下载量:  156
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-08-12
  • 修回日期:  2021-09-14
  • 网络出版日期:  2021-10-19
  • 刊出日期:  2022-03-21

目录

    /

    返回文章
    返回

    重要通知

    2024年2月16日科睿唯安通过Blog宣布,2024年将要发布的JCR2023中,229个自然科学和社会科学学科将SCI/SSCI和ESCI期刊一起进行排名!《中国光学(中英文)》作为ESCI期刊将与全球SCI期刊共同排名!