高动态范围条纹结构光在机检测技术及应用进展

刘泽隆; 李茂月; 卢新元; 张明垒

doi:10.37188/CO.2023-0068

Volume 17 Issue 1

Jan. 2024

Turn off MathJax

Article Contents

Chinese Optics > 2024 > 17(1): 1-18.

LIU Ze-long, LI Mao-yue, LU Xin-yuan, ZHANG Ming-lei. On-machine detection technology and application progress of high dynamic range fringe structured light[J]. Chinese Optics, 2024, 17(1): 1-18. doi: 10.37188/CO.2023-0068

Citation:

LIU Ze-long, LI Mao-yue, LU Xin-yuan, ZHANG Ming-lei. On-machine detection technology and application progress of high dynamic range fringe structured light[J]. Chinese Optics, 2024, 17(1): 1-18. doi: 10.37188/CO.2023-0068

Citation:

PDF( 6064 KB)

On-machine detection technology and application progress of high dynamic range fringe structured light

doi: 10.37188/CO.2023-0068

Key Laboratory of Advanced Manufacturing and Intelligent Technology, Ministry of Education, Harbin University of Science and Technology, Harbin 150080, China

Funds: Supported by National Natural Science Foundation of China (No. 51975169); Natural Science Foundation of Heilongjiang Province(No. LH2022E085)

More Information

Corresponding author: lmy0500@163.com
Received Date: 16 Apr 2023
Rev Recd Date: 15 May 2023

Available Online: 21 Sep 2023

Abstract

Abstract

Fringe structured light technology is a non-contact measurement method, which has developed rapidly in recent years and provides a new solution for on-machine detection in mechanical processing. However, the accuracy of structured light for on-machine detection is compromised by the convoluted lighting in machining environments and metal parts’ high reflectivity, leading to inaccurate measurements. Applying high dynamic range (HDR) technology to structured light detection can reduce the effect of high reflectivity, achieving the measurement of metal parts in complex scenes. This paper introduces the measurement principle of structured light and summarizes the challenges of on-machine detection for HDR structured light. Subsequently, this paper provides a comprehensive review of HDR structured light technology. In the context of on-machine detection of mechanical processing, the HDR technology based on hardware equipment and the HDR technology based on stripe algorithm are discussed and analyzed, respectively. Following this, different technologies are summarized according to the requirements of on-machine detection. The advantages and disadvantages of various methods are presented, and the applicability of on-machine detection is compared. Finally, the potential applications are analyzed, and the technological prospects will be proposed in combination with the research hotspots of advanced manufacturing technology and precision measurement in recent years.
- three-dimensional measurement,
- structured light,
- fringe projection,
- high dynamic range,
- on-machine detection

FullText(HTML)

References(76)

References

[1]	李文龙, 李中伟, 毛金城. iPoint3D曲面检测软件开发与工程应用综述[J]. 机械工程学报,2020,56(7):127-150. doi: 10.3901/JME.2020.07.127 LI W L, LI ZH W, MAO J CH. The development and application review of iPoint3D software for surface inspection[J]. Journal of Mechanical Engineering, 2020, 56(7): 127-150. (in Chinese) doi: 10.3901/JME.2020.07.127
[2]	李茂月, 刘泽隆, 赵伟翔, 等. 面结构光在机检测的叶片反光抑制技术[J]. 中国光学,2022,15(3):464-475. doi: 10.37188/CO.2021-0194 LI M Y, LIU Z L, ZHAO W X, et al. Blade reflection suppression technology based on surface structured light on-machine detection[J]. Chinese Optics, 2022, 15(3): 464-475. (in Chinese) doi: 10.37188/CO.2021-0194
[3]	任明阳, 王立忠, 赵建博, 等. 复杂曲面零件面结构光扫描视点规划[J]. 中国光学(中英文),2023,16(1):113-126. doi: 10.37188/CO.2022-0026 REN M Y, WANG L ZH, ZHAO J B, et al. Viewpoint planning of surface structured light scanning for complex surface parts[J]. Chinese Optics, 2023, 16(1): 113-126. (in Chinese) doi: 10.37188/CO.2022-0026
[4]	张宗华, 刘巍, 刘国栋, 等. 三维视觉测量技术及应用进展[J]. 中国图象图形学报,2021,26(6):1483-1502. ZHANG Z H, LIU W, LIU G D, et al. Overview of the development and application of 3D vision measurement technology[J]. Journal of Image and Graphics, 2021, 26(6): 1483-1502. (in Chinese)
[5]	XU J, ZHANG S. Status, challenges, and future perspectives of fringe projection profilometry[J]. Optics and Lasers in Engineering, 2020, 135: 106193. doi: 10.1016/j.optlaseng.2020.106193
[6]	张宗华, 于瑾, 高楠, 等. 高反光表面三维形貌测量技术[J]. 红外与激光工程,2020,49(3):0303006. doi: 10.3788/IRLA202049.0303006 ZHANG Z H, YU J, GAO N, et al. Three-dimensional shape measurement techniques of shiny surfaces[J]. Infrared and Laser Engineering, 2020, 49(3): 0303006. (in Chinese) doi: 10.3788/IRLA202049.0303006
[7]	FENG S J, ZHANG L, ZUO CH, et al. High dynamic range 3D measurements with fringe projection profilometry: a review[J]. Measurement Science and Technology, 2018, 29(12): 122001. doi: 10.1088/1361-6501/aae4fb
[8]	ZHANG P, ZHONG K, LI ZH W, et al. Hybrid-quality-guided phase fusion model for high dynamic range 3D surface measurement by structured light technology[J]. Optics Express, 2022, 30(9): 14600-14614. doi: 10.1364/OE.457305
[9]	Zhang S. Rapid and automatic optimal exposure control for digital fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106029. doi: 10.1016/j.optlaseng.2020.106029
[10]	马泽龙, 高慧斌, 余毅, 等. 采用图像直方图特征函数的高速相机自动曝光方法[J]. 光学精密工程,2017,25(4):1026-1035. doi: 10.3788/OPE.20172504.1026 MA Z L, GAO H B, YU Y, et al. Auto exposure control for high frame rate camera using image histogram feature function[J]. Optics and Precision Engineering, 2017, 25(4): 1026-1035. (in Chinese) doi: 10.3788/OPE.20172504.1026
[11]	雷经发, 陆宗胜, 李永玲, 等. 基于投影栅相位法和多曝光图像融合技术的强反射表面轮廓检测[J]. 光学精密工程,2022,30(18):2195-2204. doi: 10.37188/OPE.20223018.2195 LEI J F, LU Z SH, LI Y L, et al. High reflection surface topography measurement based on fringe projection phase method and multi-exposure image fusion technology[J]. Optics and Precision Engineering, 2022, 30(18): 2195-2204. (in Chinese) doi: 10.37188/OPE.20223018.2195
[12]	ZHANG S, YAU S T. High dynamic range scanning technique[J]. Optical Engineering, 2009, 48(3): 033604. doi: 10.1117/1.3099720
[13]	SONG ZH, JIANG H L, LIN H B, et al. A high dynamic range structured light means for the 3D measurement of specular surface[J]. Optics and Lasers in Engineering, 2017, 95: 8-16. doi: 10.1016/j.optlaseng.2017.03.008
[14]	FENG SH J, ZHANG Y ZH, CHEN Q, et al. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique[J]. Optics and Lasers in Engineering, 2014, 59: 56-71. doi: 10.1016/j.optlaseng.2014.03.003
[15]	CUI H H, LI ZH J, TIAN W, et al. Multiple-exposure adaptive selection algorithm for high dynamic range 3D fringe projection measurement[J]. Proceedings of SPIE, 2019, 11053: 110530M.
[16]	RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi: 10.1016/j.jvcir.2017.12.003
[17]	WU K, TAN J, XIA H L, et al. An exposure fusion-based structured light approach for the 3D measurement of a specular surface[J]. IEEE Sensors Journal, 2021, 21(5): 6314-6324. doi: 10.1109/JSEN.2020.3027317
[18]	SALAHIEH B, CHEN ZH Y, RODRIGUEZ J J, et al. Multi-polarization fringe projection imaging for high dynamic range objects[J]. Optics Express, 2014, 22(8): 10064-10071. doi: 10.1364/OE.22.010064
[19]	平茜茜, 刘勇, 董欣明, 等. 基于偏振双目视觉的无纹理高反光目标三维重构[J]. 红外与毫米波学报,2017,36(4):432-438. PING X X, LIU Y, DONG X M, et al. 3-D reconstruction of textureless and high-reflective target by polarization and binocular stereo vision[J]. Journal of Infrared and Millimeter Waves, 2017, 36(4): 432-438. (in Chinese)
[20]	郝婧蕾, 赵永强, 赵海盟, 等. 偏振多光谱机器视觉的高反光无纹理目标三维重构方法[J]. 测绘学报,2018,47(6):816-824. HAO J L, ZHAO Y Q, ZHAO H M, et al. 3D Reconstruction of High-reflective and Textureless Targets Based on Multispectral Polarization and Machine Vision[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 816-824. (in Chinese)
[21]	WANG Y H, ZHANG Q, HU Y, et al. Rapid 3D measurement of high dynamic range surface based on multi-polarization fringe projection[J]. Optical Engineering, 2021, 60(8): 084107.
[22]	ZHU ZH M, ZHU W T, ZHOU F Q, et al. Three-dimensional measurement of fringe projection based on the camera response function of the polarization system[J]. Optical Engineering, 2021, 60(5): 055105.
[23]	MAEDA Y, SHIBATA S, HAGEN N, et al. Single shot 3D profilometry by polarization pattern projection[J]. Applied Optics, 2020, 59(6): 1654-1659. doi: 10.1364/AO.382690
[24]	XIANG G, ZHU H J, GUO H W. Spatial phase-shifting profilometry by use of polarization for measuring 3D shapes of metal objects[J]. Optics Express, 2021, 29(13): 20981-20994. doi: 10.1364/OE.427407
[25]	王月敏, 张宗华, 高楠. 基于全场条纹反射的镜面物体三维面形测量综述[J]. 光学精密工程,2018,26(5):1014-1027. doi: 10.3788/OPE.20182605.1014 WANG Y M, ZHANG Z H, GAO N. Review on three-dimensional surface measurements of specular objects based on full-field fringe reflection[J]. Optics and Precision Engineering, 2018, 26(5): 1014-1027. (in Chinese) doi: 10.3788/OPE.20182605.1014
[26]	ZHUANG Y CH, ZHENG Y M, LIN SH B, et al. Surface shape distortion online measurement method for compact laser cavities based on phase measuring Deflectometry[J]. Photonics, 2022, 9(3): 151. doi: 10.3390/photonics9030151
[27]	GAO F, XU Y J, JIANG X Q. Near optical coaxial phase measuring deflectometry for measuring structured specular surfaces[J]. Optics Express, 2022, 30(10): 17554-17566. doi: 10.1364/OE.457198
[28]	HAN H, WU SH Q, SONG ZH. Curved LCD based deflectometry method for specular surface measurement[J]. Optics and Lasers in Engineering, 2022, 151: 106909. doi: 10.1016/j.optlaseng.2021.106909
[29]	SU P, PARKS R E, WANG L R, et al. Software configurable optical test system: a computerized reverse Hartmann test[J]. Applied Optics, 2010, 49(23): 4404-4412. doi: 10.1364/AO.49.004404
[30]	邵山川, 陶小平, 王孝坤. 基于条纹反射的超精密车削反射镜的在位面形检测[J]. 激光与光电子学进展,2018,55(7):071203. SHAO SH CH, TAO X P, WANG X K. On-machine surface shape measurement of reflective mirrors by ultra-precision turning based on fringe reflection[J]. Laser & Optoelectronics Progress, 2018, 55(7): 071203. (in Chinese)
[31]	袁婷. 基于条纹反射法的大口径非球面反射镜面形检测技术研究[D]. 长春: 中国科学院研究生院(长春光学精密机械与物理研究所), 2016. YUAN T. Study on fringe-reflection optical surface shape measurement technology for large aspheric mirror[D]. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics, CAS, 2016. (in Chinese)
[32]	OH C J, LOWMAN A E, SMITH G A, et al. Fabrication and testing of 4.2m off-axis aspheric primary mirror of Daniel K. Inouye Solar Telescope[J]. Proceedings of SPIE, 2016, 9912: 99120O.
[33]	WOODHAM R J. Photometric method for determining surface orientation from multiple images[J]. Optical Engineering, 1980, 19(1): 191139.
[34]	LU L, QI L, LUO Y S, et al. Three-dimensional reconstruction from single image base on combination of CNN and multi-spectral photometric stereo[J]. Sensors, 2018, 18(3): 764. doi: 10.3390/s18030764
[35]	张颖, 李金龙, 黄趾维, 等. 基于BRDF模型的金属表面反射特性及相变特性研究[J]. 光电技术应用,2017,32(3):32-35. doi: 10.3969/j.issn.1673-1255.2017.03.008 ZHANG Y, LI J L, HUANG ZH W, et al. Research on reflection and phase shift characters of metal surface based on BRDF model[J]. Electro-Optic Technology Application, 2017, 32(3): 32-35. (in Chinese) doi: 10.3969/j.issn.1673-1255.2017.03.008
[36]	王金海, 李华, 魏力. 基于C-T模型的光学元件加工表面的光学特性研究[J]. 光学技术,2021,47(2):172-177. WANG J H, LI H, WEI L. Study on optical properties of machining surface of optical element based on C-T model[J]. Optical Technique, 2021, 47(2): 172-177.
[37]	PEI X H, REN M J, WANG X, et al. Profile measurement of non-Lambertian surfaces by integrating fringe projection profilometry with near-field photometric stereo[J]. Measurement, 2022, 187: 110277. doi: 10.1016/j.measurement.2021.110277
[38]	MENG L F, LU L Y, BEDARD N, et al. Single-shot specular surface reconstruction with gonio-plenoptic imaging[C]. IEEE International Conference on Computer Vision, IEEE, 2015: 3433-3441.
[39]	WADDINGTON C, KOFMAN J. Analysis of measurement sensitivity to illuminance and fringe-pattern gray levels for fringe-pattern projection adaptive to ambient lighting[J]. Optics and Lasers in Engineering, 2010, 48(2): 251-256. doi: 10.1016/j.optlaseng.2009.07.001
[40]	LI D, KOFMAN J. Adaptive fringe-pattern projection for image saturation avoidance in 3D surface-shape measurement[J]. Optics Express, 2014, 22(8): 9887-9901. doi: 10.1364/OE.22.009887
[41]	JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi: 10.1016/j.optlaseng.2011.11.021
[42]	WANG J H, YANG Y X, ZHOU Y G. 3-D shape reconstruction of non-uniform reflectance surface based on pixel intensity, pixel color and camera exposure time adaptive adjustment[J]. Scientific Reports, 2021, 11(1): 4700. doi: 10.1038/s41598-021-83779-9
[43]	SUN J H, ZHANG Q Y. A 3D shape measurement method for high-reflective surface based on accurate adaptive fringe projection[J]. Optics and Lasers in Engineering, 2022, 153: 106994. doi: 10.1016/j.optlaseng.2022.106994
[44]	BABAIE G, ABOLBASHARI M, FARAHI F. Dynamics range enhancement in digital fringe projection technique[J]. Precision Engineering, 2015, 39: 243-251. doi: 10.1016/j.precisioneng.2014.06.007
[45]	CHEN CH, GAO N, WANG X J, et al. Adaptive pixel-to-pixel projection intensity adjustment for measuring a shiny surface using orthogonal color fringe pattern projection[J]. Measurement Science and Technology, 2018, 29(5): 055203. doi: 10.1088/1361-6501/aab07a
[46]	冯维, 徐仕楠, 王恒辉, 等. 逐像素调制的高反光表面三维测量方法[J]. 中国光学,2022,15(3):488-497. doi: 10.37188/CO.2021-0220 FENG W, XU SH N, WANG H H, et al. Three-dimensional measurement method of highly reflective surface based on per-pixel modulation[J]. Chinese Optics, 2022, 15(3): 488-497. (in Chinese) doi: 10.37188/CO.2021-0220
[47]	李乾, 薛俊鹏, 张启灿, 等. 利用相机响应曲线实现高反光元件三维面形测量[J]. 光学学报,2022,42(7):0712001. doi: 10.3788/AOS202242.0712001 LI Q, XUE J P, ZHANG Q C, et al. Three dimensional shape measurement of high reflective elements using camera response curve[J]. Acta Optica Sinica, 2022, 42(7): 0712001. (in Chinese) doi: 10.3788/AOS202242.0712001
[48]	SHAFER S A. Using color to separate reflection components[J]. Color Research & Application, 1985, 10(4): 210-218.
[49]	WANG J H, YANG Y X. High-speed three-dimensional measurement technique for object surface with a large range of reflectivity variations[J]. Applied Optics, 2018, 57(30): 9172-9182. doi: 10.1364/AO.57.009172
[50]	CHUA S Y, LIM C C, ENG S K, et al. Improved high dynamic range for 3D shape measurement based on saturation of the coloured fringe[J]. Pertanika Journal of Science & Technology, 2021, 29(2): 759-770.
[51]	YIN Y K, CAI Z W, JIANG H, et al. High dynamic range imaging for fringe projection profilometry with single-shot raw data of the color camera[J]. Optics and Lasers in Engineering, 2017, 89: 138-144. doi: 10.1016/j.optlaseng.2016.08.019
[52]	ZHENG Y, WANG Y J, SURESH V, et al. Real-time high-dynamic-range fringe acquisition for 3D shape measurement with a RGB camera[J]. Measurement Science and Technology, 2019, 30(7): 075202. doi: 10.1088/1361-6501/ab0ced
[53]	LIU Y ZH, FU Y J, ZHUAN Y H, et al. High dynamic range real-time 3D measurement based on Fourier transform profilometry[J]. Optics & Laser Technology, 2021, 138: 106833.
[54]	CHEN Y M, HE Y M, HU E Y. Phase deviation analysis and phase retrieval for partial intensity saturation in phase-shifting projected fringe profilometry[J]. Optics Communications, 2008, 281(11): 3087-3090. doi: 10.1016/j.optcom.2008.01.070
[55]	JIANG C F, BELL T, ZHANG S. High dynamic range real-time 3D shape measurement[J]. Optics Express, 2016, 24(7): 7337-7346. doi: 10.1364/OE.24.007337
[56]	WANG M M, DU G L, ZHOU C L, et al. Enhanced high dynamic range 3D shape measurement based on generalized phase-shifting algorithm[J]. Optics Communications, 2017, 385: 43-53. doi: 10.1016/j.optcom.2016.10.023
[57]	ZUO CH, HUANG L, ZHANG M L, et al. Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review[J]. Optics and Lasers in Engineering, 2016, 85: 84-103. doi: 10.1016/j.optlaseng.2016.04.022
[58]	CHEN B, ZHANG S. High-quality 3D shape measurement using saturated fringe patterns[J]. Optics and Lasers in Engineering, 2016, 87: 83-89. doi: 10.1016/j.optlaseng.2016.04.012
[59]	HE Z X, LI P L, ZHAO X Y, et al. Chessboard-like high-frequency patterns for 3D measurement of reflective surface[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 5009712.
[60]	HU Y, CHEN Q, LIANG Y CH, et al. Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme[J]. Optics and Lasers in Engineering, 2019, 122: 1-7. doi: 10.1016/j.optlaseng.2019.05.019
[61]	张启灿, 吴周杰. 基于格雷码图案投影的结构光三维成像技术[J]. 红外与激光工程,2020,49(3):0303004. doi: 10.3788/IRLA202049.0303004 ZHANG Q C, WU ZH J. Three-dimensional imaging technique based on Gray-coded structured illumination[J]. Infrared and Laser Engineering, 2020, 49(3): 0303004. (in Chinese) doi: 10.3788/IRLA202049.0303004
[62]	SONG ZH, CHUNG R, ZHANG X T. An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3-D[J]. IEEE Transactions on Industrial Electronics, 2013, 60(3): 1023-1032. doi: 10.1109/TIE.2012.2188875
[63]	LU L L, WU ZH J, ZHANG Q C, et al. High-efficiency dynamic three-dimensional shape measurement based on misaligned Gray-code light[J]. Optics and Lasers in Engineering, 2022, 150: 106873. doi: 10.1016/j.optlaseng.2021.106873
[64]	ZUO CH, QIAN J M, FENG SH J, et al. Deep learning in optical metrology: a review[J]. Light:Science & Applications, 2022, 11(1): 39.
[65]	REYES-FIGUEROA A, FLORES V H, RIVERA M. Deep neural network for fringe pattern filtering and normalization[J]. Applied Optics, 2021, 60(7): 2022-2036. doi: 10.1364/AO.413404
[66]	JEON W, JEONG W, SON K, et al. Speckle noise reduction for digital holographic images using multi-scale convolutional neural networks[J]. Optics Letters, 2018, 43(17): 4240-4243. doi: 10.1364/OL.43.004240
[67]	ZHANG B W, LIN SH N, LIN J Y, et al. Single-shot high-precision 3D reconstruction with color fringe projection profilometry based BP neural network[J]. Optics Communications, 2022, 517: 128323. doi: 10.1016/j.optcom.2022.128323
[68]	NGUYEN H, WANG ZH Y. Accurate 3D shape reconstruction from single structured-light image via fringe-to-fringe network[J]. Photonics, 2021, 8(11): 459. doi: 10.3390/photonics8110459
[69]	FENG SH J, ZUO CH, YIN W, et al. Micro deep learning profilometry for high-speed 3D surface imaging[J]. Optics and Lasers in Engineering, 2019, 121: 416-427. doi: 10.1016/j.optlaseng.2019.04.020
[70]	LIU Y, BLUNT L, GAO F, et al. High-dynamic-range 3D measurement for E-beam fusion additive manufacturing based on SVM intelligent fringe projection system[J]. Surface Topography:Metrology and Properties, 2021, 9(3): 034002. doi: 10.1088/2051-672X/ac0c62
[71]	彭广泽, 陈文静. 基于卷积神经网络去噪正则化的条纹图修复[J]. 光学学报,2020,40(18):1810002. doi: 10.3788/AOS202040.1810002 PENG G Z, CHEN W J. Fringe pattern inpainting based on convolutional neural network denoising regularization[J]. Acta Optica Sinica, 2020, 40(18): 1810002. (in Chinese) doi: 10.3788/AOS202040.1810002
[72]	YANG G W, YANG M, ZHOU N, et al. High dynamic range fringe pattern acquisition based on deep neural network[J]. Optics Communications, 2022, 512: 127765. doi: 10.1016/j.optcom.2021.127765
[73]	QIAO G, HUANG Y Y, SONG Y P, et al. A single-shot phase retrieval method for phase measuring deflectometry based on deep learning[J]. Optics Communications, 2020, 476: 126303. doi: 10.1016/j.optcom.2020.126303
[74]	ZHANG L, CHEN Q, ZUO CH, et al. High-speed high dynamic range 3D shape measurement based on deep learning[J]. Optics and Lasers in Engineering, 2020, 134: 106245. doi: 10.1016/j.optlaseng.2020.106245
[75]	HU Y, CHEN Q, TAO T Y, et al. Absolute three-dimensional micro surface profile measurement based on a Greenough-type stereomicroscope[J]. Measurement Science and Technology, 2017, 28(4): 045004. doi: 10.1088/1361-6501/aa5a2d
[76]	陈龙, 王文聪, 张峰峰, 等. 基于双目结构光的术中肝脏表面局部亮度饱和分区投影[J]. 光学精密工程,2021,29(11):2590-2602. doi: 10.37188/OPE.20212911.2590 CHEN L, WANG W C, ZHANG F F, et al. Zonal projection based on binocular structured light for localized luminance saturation of intraoperative liver surface[J]. Optics and Precision Engineering, 2021, 29(11): 2590-2602. (in Chinese) doi: 10.37188/OPE.20212911.2590