面部视频非接触式生理参数感知

嵇晓强; 刘振瑶; 李炳霖; 饶治; 李贵文; 粟立威

doi:10.37188/CO.2021-0157

Volume 15 Issue 2

Mar. 2022

Turn off MathJax

Article Contents

Chinese Optics > 2022 > 15(2): 276-285.

JI Xiao-qiang, LIU Zhen-yao, LI Bing-lin, RAO Zhi, LI Gui-wen, SU Li-wei. Non-contact perception of physiological parameters from videos of faces[J]. Chinese Optics, 2022, 15(2): 276-285. doi: 10.37188/CO.2021-0157

Citation:

JI Xiao-qiang, LIU Zhen-yao, LI Bing-lin, RAO Zhi, LI Gui-wen, SU Li-wei. Non-contact perception of physiological parameters from videos of faces[J]. Chinese Optics, 2022, 15(2): 276-285. doi: 10.37188/CO.2021-0157

Citation:

PDF( 4176 KB)

Non-contact perception of physiological parameters from videos of faces

doi: 10.37188/CO.2021-0157

School of Life Science and Technology, Changchun University of Science and Technology, Changchun 130022, China

Funds: Supported by Department of Science and Technology of Natural Science Foundation of Jilin Province under grant number (No. 20210204131YY)

More Information

Corresponding author: zuoanmulan@163.com
Received Date: 12 Aug 2021
Rev Recd Date: 14 Sep 2021

Available Online: 19 Oct 2021

Publish Date: 21 Mar 2022

Abstract

Abstract

Non-contact detection of various physiological parameters has attract great attention. In this paper, a method of estimating physiological parameters based on imaging photoplethysmography from videos of people’s faces recorded by mobile phone is proposed. First, a "wavelet transform-principal component analysis-blind source separation" algorithm is proposed to extract the video’s RGB three-channel pulse wave signal with a high signal-to-noise ratio. Then, the green channel signal is processed separately in the frequency and the time domains to estimate heart and respiratory rates. The pulse wave signals of the red and blue channels are processed, and combined with the oxygen saturation detected by an oximeter to perform data fitting, the best linear equation for estimating the oxygen saturation value from the facial video is found. Finally, the error of the estimation results of various physiological parameters under natural light is compared, and the estimation results of each parameter under three lighting environments are analyzed. The results show that under the three lighting environments, the average error of heart rate detection is 0.5512 time/min, the average error of respiration rate is −0.6321 time/min , and the average error of oxygen saturation is −0.2743%. In summary, the non-contact physiological parameter estimation method proposed in this paper is highly accurate, universally applicable and stable. Its estimation results are highly consistent with the measurement result of standard instruments, which meets the needs of daily physiological parameter measurement.
- imaging photoplethysmography,
- non-contact,
- wavelet transform-principal component analysis-blind signal separation,
- heart rate,
- respiratory rate,
- oxygen saturation

FullText(HTML)

References(27)

References

[1]	TULPPO M P, KIVINIEMI A M, JUNTTILA M J, et al. Home monitoring of heart rate as a predictor of imminent cardiovascular events[J]. Frontiers in Physiology, 2019, 10: 341. doi: 10.3389/fphys.2019.00341
[2]	KAUSER M A, SAFIUDDIN M. Heart rate-an emerging cardiovascular risk factor[J]. University Heart Journal, 2012, 8(1): 30-35. doi: 10.3329/uhj.v8i1.11665
[3]	MATHER M, THAYER J F. How heart rate variability affects emotion regulation brain networks[J]. Current Opinion in Behavioral Sciences, 2018, 19: 98-104. doi: 10.1016/j.cobeha.2017.12.017
[4]	孙莉娜, 李䶮, 郭汉涛, 等. 氮、铁共掺杂碳纳米粒子的制备及在过氧化氢和葡萄糖检测中的应用[J]. 应用化学,2020,37(3):350-358. SUN L N, LI Y, GUO H T, et al. Preparation of nitrogen and iron co-doped carbon nanoparticles and their applications in detection of hydrogen peroxide and glucose[J]. Chinese Journal of Applied Chemistry, 2020, 37(3): 350-358. (in Chinese)
[5]	TARASSENKO L, VILLARROEL M, GUAZZI A, et al. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models[J]. Physiological Measurement, 2014, 35(5): 807-831. doi: 10.1088/0967-3334/35/5/807
[6]	BAL U. Non-contact estimation of heart rate and oxygen saturation using ambient light[J]. Biomedical Optics Express, 2015, 6(1): 86-97. doi: 10.1364/BOE.6.000086
[7]	CASALINO G, CASTELLANO G, PASQUADIBISCEGLIE V, et al. Contact-less real-time monitoring of cardiovascular risk using video imaging and fuzzy inference rules[J]. Information, 2019, 10(1): 9.
[8]	荣猛, 范强, 李凯扬. 基于IPPG非接触式生理参数测量算法的研究[J]. 生物医学工程研究,2018,37(1):27-31,35. RONG M, FAN Q, LI K Y. Study on the measurement algorithm of contactless physiological parameter based on imaging photoplenthysmography[J]. Journal of Biomedical Engineering Research, 2018, 37(1): 27-31,35. (in Chinese)
[9]	李晓媛, 武鹏, 刘允, 等. 基于人脸视频的心率参数提取[J]. 光学精密工程,2020,28(3):548-557. doi: 10.3788/OPE.20202803.0548 LI X Y, WU P, LIU Y, et al. Extraction of heart rate parameters from video of human face[J]. Optics and Precision Engineering, 2020, 28(3): 548-557. (in Chinese) doi: 10.3788/OPE.20202803.0548
[10]	LUGUERN D, MACWAN R, BENEZETH Y, et al. Wavelet variance maximization: a contactless respiration rate estimation method based on remote photoplethysmography[J]. Biomedical Signal Processing and Control, 2021, 63: 102263. doi: 10.1016/j.bspc.2020.102263
[11]	RAHMAN H, AHMED M U, BEGUM S. Non-contact physiological parameters extraction using facial video considering illumination, motion, movement and vibration[J]. IEEE Transactions on Biomedical Engineering, 2020, 67(1): 88-98. doi: 10.1109/TBME.2019.2908349
[12]	戴阳, 郑婷婷, 杨雪. 基于视频放大与盲源分离的非接触式心率检测[J]. 计算机系统应用,2021,30(1):228-234. DAI Y, ZHENG T T, YANG X. Non-contact heart rate detection based on video amplification and blind source separation[J]. Computer Systems &Applications, 2021, 30(1): 228-234. (in Chinese)
[13]	CHEN Q, JIANG X Y, LIU X Y, et al.. Non-contact heart rate monitoring in neonatal intensive care unit using RGB camera[C]. Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, IEEE, 2020: 5822-5825.
[14]	WU H Y, RUBINSTEIN M, SHIH E, et al. Eulerian video magnification for revealing subtle changes in the world[J]. ACM Transactions on Graphics, 2012, 31(4): 65.
[15]	田昕, 赖翰文, 刘亚栋, 等. 基于卷积运算的嵌段共聚物自组装图像缺陷分析统计方法[J]. 应用化学,2021,38(9):1199-1208. TIAN X, LAI H W, LIU Y D, et al. Analysis of defects in block copolymer films by a convolution algorithm[J]. Chinese Journal of Applied Chemistry, 2021, 38(9): 1199-1208. (in Chinese)
[16]	LIU J G, LUO H, ZHENG P P. et al. Transdermal optical imaging revealed different spatiotemporal patterns of facial cardiovascular activities[J]. Scientific Reports, 2018, 8(1): 10588. doi: 10.1038/s41598-018-28804-0
[17]	彭仁杰, 姚云霞. 小波分解和重构的光学可变图像融合研究[J]. 激光杂志,2021,42(3):145-148. PENG R J, YAO Y X. Research on optical variable image fusion based on wavelet decomposition and reconstruction[J]. Laser Journal, 2021, 42(3): 145-148. (in Chinese)
[18]	YING L, ZHANG F B, ZHANG Y L, et al. Identifying distributed dynamic loading in one spatial dimension based on combing wavelet decomposition and Kalman filter with unknown input[J]. Journal of Aerospace Engineering, 2021, 34(4): 04021025. doi: 10.1061/(ASCE)AS.1943-5525.0001265
[19]	SONG K, ZHANG B Y, LI W, et al. Research on parallel principal component analysis based on ternary optical computer[J]. Optik, 2021, 241: 167176. doi: 10.1016/j.ijleo.2021.167176
[20]	张申华, 杨延西, 秦峤孟. 针对光栅图像的快速盲去噪方法[J]. 中国光学,2021,14(3):596-604. doi: 10.37188/CO.2020-0166 ZHANG SH H, YANG Y X, QIN Q M. A fast blind denoising method for grating image[J]. Chinese Optics, 2021, 14(3): 596-604. (in Chinese) doi: 10.37188/CO.2020-0166
[21]	邱晓华, 李敏, 邓光芒, 等. 多层卷积特征融合的双波段决策级船舶识别[J]. 光学精密工程,2021,29(1):183-190. doi: 10.37188/OPE.20212901.0183 QIU X H, LI M, DENG G M, et al. Multi-layer convolutional features fusion for dual-band decision-level ship recognition[J]. Optics and Precision Engineering, 2021, 29(1): 183-190. (in Chinese) doi: 10.37188/OPE.20212901.0183
[22]	VERKRUYSSE W, SVAASAND L O, NELSON J S. Remote plethysmographic imaging using ambient light[J]. Optics Express, 2008, 16(26): 21434-21445. doi: 10.1364/OE.16.021434
[23]	乔闹生, 孙萍. CCD非线性效应对双频光栅三维面形测量的影响[J]. 中国光学,2021,14(3):661-669. doi: 10.37188/CO.2020-0143 QIAO N SH, SUN P. Influence of CCD nonlinearity effect on the three-dimensional shape measurement of dual frequency grating[J]. Chinese Optics, 2021, 14(3): 661-669. (in Chinese) doi: 10.37188/CO.2020-0143
[24]	赵云, 吕金光, 秦余欣, 等. 微型傅立叶变换光谱仪的优化设计与实验研究[J]. 中国光学,2020,13(2):411-425. doi: 10.3788/co.20201302.0411 ZHAO Y, LV J G, QIN Y X, et al. Optimization design and experimental study of micro-Fourier transform spectrometer[J]. Chinese Optics, 2020, 13(2): 411-425. (in Chinese) doi: 10.3788/co.20201302.0411
[25]	孙学辉, 赵冰, 骆震, 等. 离散傅立叶变换用于非连续工业数据分析[J]. 分析化学,2020,48(10):1422-1427. SUN X H, ZHAO B, TUO Z, et al. Non-continuous industrial data Analysis using discrete fourier fransform[J]. Chinese Journal of Analytical Chemistry, 2020, 48(10): 1422-1427. (in Chinese)
[26]	刘今越, 刘浩, 贾晓辉, 等. 基于视觉的非接触呼吸频率自动检测方法[J]. 仪器仪表学报,2019,40(2):51-58. LIU J Y, LIU H, JIA X H, et al. Vision-based automatic detection method for non-contact respiratory rate[J]. Chinese Journal of Scientific Instrument, 2019, 40(2): 51-58. (in Chinese)
[27]	任繁栋, 丁筱雪, 蔡芳, 等. 基于超高效液相色谱-高分辨质谱联用技术研究冠心病及冠心病合并2型糖尿病患者代谢特征[J]. 分析化学,2020,48(1):49-56. REN F D, DING X X, CAI F, et al. Investigation of metabolic features of patients with coronary heart disease or coronary heart disease-type 2 diabetes mellitus based on ultra-high performance liquid chromatography high resolution mass spectrometry[J]. Chinese Journal of Analytical Chemistry, 2020, 48(1): 49-56. (in Chinese)