Volume 14 Issue 3
May  2021
Turn off MathJax
Article Contents
Chinese Optics  > 2021  >  14(3): 447-457.
WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry：a review[J]. Chinese Optics, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199
Citation: WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry：a review[J]. Chinese Optics, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199
shu

3D small-field surface imaging based on microscopic fringe projection profilometry：a review

  • School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology, Hefei 230009, China

Funds:  Supported by National Key Research and Development Program of China (No. 2016YFF0101803); National Natural Science Foundation of China (No. 51805137)
More Information
  • Corresponding author: yhwang@hfut.edu.cn
  • Received Date: 10 Nov 2020
  • Rev Recd Date: 07 Jan 2021
    • Available Online: 27 Mar 2021
  • Publish Date: 14 May 2021
    Abstract
  • Intelligent manufacturing has become more precise, miniaturized and integrated. Representative integrated circuit technology and its derived miniature sensors such as Micro-Electro-Mechanical System (MEMS) have become widely used. Therefore, it is important for intelligent manufacturing development to accurately obtain the surface morphology information of micro-devices and implement rapid detection of device surface defects. Fringe Projection Profilometry (FPP) based on structural light projection has the advantages of being non-contact, highly precise, highly efficient and having full-field measurement, which plays an important role in the field of precision measurement. Microscopic Fringe Projection Profilometry (MFPP) has been developed rapidly during recent decades. In recent years, MFPP has made great progress in many aspects, including its optical system structures, corresponding system calibration methods, phase extraction algorithms, and 3D coordinate reconstruction methods. In this paper, the structure and principle of a three-dimensional measurement system of microscopic fringe projection are reviewed, and the calibration problem of a small field-of-view system that is different from the traditional projection model is analyzed. After that, the development and improvement process of the micro-projection system structure is introduced, and the reflection in the measurment caused by the system structure and metal material is analyzed. On this basis, the prospects of the development of microscopic fringe projection of 3D measurement system are discussed.

     

    • FullText(HTML)
  • loading
    • References(60)
  • [1]
    WANG J H, YANG Y X, SHAO M W, et al. Three-dimensional measurement for rigid moving objects based on multi-fringe projection[J]. IEEE Photonics Journal, 2020, 12(4): 6802114.
    [2]
    XIA P, WANG Q H, RI SH E. Random phase-shifting digital holography based on a self-calibrated system[J]. Optics Express, 2020, 28(14): 19988-19996. doi: 10.1364/OE.395819
    [3]
    屈铭, 郑俊杰, 李敏, 等. 基于扫描白光干涉法的LCOS芯片像素级相位分析[J]. 光子学报,2019,48(9):0911004. doi: 10.3788/gzxb20194809.0911004

    QU M, ZHENG J J, LI M, et al. Pixel-level observation of phase profile in liquid crystal on silicon device by white light scanning interferometry[J]. Acta Photonica Sinica, 2019, 48(9): 0911004. (in Chinese) doi: 10.3788/gzxb20194809.0911004
    [4]
    MURAKAMI H, KATSUKI A, SAJIMA T, et al. Investigation of factors affecting sensitivity enhancement of an optical fiber probe for microstructure measurement using oblique incident light[J]. Applied Sciences, 2020, 10(9): 3191. doi: 10.3390/app10093191
    [5]
    李成辉, 田云飞, 闫曙光. 激光扫描共聚焦显微成像技术与应用[J]. 实验科学与技术,2020,18(4):33-38. doi: 10.12179/1672-4550.20190257

    LI CH H, TIAN Y F, YAN SH G. Laser scanning confocal microscopy and its application[J]. Experiment Science and Technology, 2020, 18(4): 33-38. (in Chinese) doi: 10.12179/1672-4550.20190257
    [6]
    HU Y, CHEN Q, FENG SH J, et al. Microscopic fringe projection profilometry: a review[J]. Optics and Lasers in Engineering, 2020, 135: 106192. doi: 10.1016/j.optlaseng.2020.106192
    [7]
    LEONHARDT K, DROSTE U, TIZIANI H J. Microshape and rough-surface analysis by fringe projection[J]. Applied Optics, 1994, 33(31): 7477-7488. doi: 10.1364/AO.33.007477
    [8]
    QUAN C, HE X Y, WANG C F, et al. Shape measurement of small objects using LCD fringe projection with phase shifting[J]. Optics Communications, 2001, 189(1-3): 21-29. doi: 10.1016/S0030-4018(01)01038-0
    [9]
    PROLL K P, NIVET J M, KÖRNER K, et al. Microscopic three-dimensional topometry with ferroelectric liquid-crystal-on-silicon displays[J]. Applied Optics, 2003, 42(10): 1773-1778. doi: 10.1364/AO.42.001773
    [10]
    NOTNI G, RIEHEMANN S, KUEHMSTEDT P, et al. OLED microdisplays: a new key element for fringe projection setups[J]. Proceedings of SPIE, 2004, 5532: 170-177. doi: 10.1117/12.560433
    [11]
    ZHANG SH F, LI B, REN F J, et al. High-precision measurement of binocular telecentric vision system with novel calibration and matching methods[J]. IEEE Access, 2019, 7: 54682-54692. doi: 10.1109/ACCESS.2019.2913181
    [12]
    ZHANG CH P, HUANG P S, CHIANG F P. Microscopic phase-shifting profilometry based on digital micromirror device technology[J]. Applied Optics, 2002, 41(28): 5896-5904. doi: 10.1364/AO.41.005896
    [13]
    LIU Y H, ZHANG Q C, ZHANG H H, et al. Improve temporal Fourier transform profilometry for complex dynamic three-dimensional shape measurement[J]. Sensors, 2020, 20(7): 1808. doi: 10.3390/s20071808
    [14]
    ZHANG H H, ZHANG Q C, LI Y, et al. High speed 3D shape measurement with temporal Fourier transform profilometry[J]. Applied Sciences, 2019, 9(19): 4123. doi: 10.3390/app9194123
    [15]
    史耀群, 邓林嘉, 王朝旭, 等. 一种基于结构光条纹投影的微小物体测量系统[J]. 应用光学,2019,40(6):1120-1125. doi: 10.5768/JAO201940.0603007

    SHI Y Q, DENG L J, WANG ZH X, et al. Micro-objects measurement system based on structured light fringe projection[J]. Journal of Applied Optics, 2019, 40(6): 1120-1125. (in Chinese) doi: 10.5768/JAO201940.0603007
    [16]
    ZHONG M, CUI J, HYUN J S, et al. Uniaxial three-dimensional phase-shifting profilometry using a dual-telecentric structured light system in micro-scale devices[J]. Measurement Science and Technology, 2020, 31(8): 085003. doi: 10.1088/1361-6501/ab63b2
    [17]
    殷永凯, 张宗华, 刘晓利, 等. 条纹投影轮廓术系统模型与标定综述[J]. 红外与激光工程,2020,49(3):0303008. doi: 10.3788/IRLA202049.0303008

    YIN Y K, ZHANG Z H, LIU X L, et al. Review of the system model and calibration for fringe projection profilometry[J]. Infrared and Laser Engineering, 2020, 49(3): 0303008. (in Chinese) doi: 10.3788/IRLA202049.0303008
    [18]
    ZHANG ZH Y. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334. doi: 10.1109/34.888718
    [19]
    LI B W, KARPINSKY N, ZHANG S. Novel calibration method for structured-light system with an out-of-focus projector[J]. Applied Optics, 2014, 53(16): 3415-3426. doi: 10.1364/AO.53.003415
    [20]
    LI B W, ZHANG S. Flexible calibration method for microscopic structured light system using telecentric lens[J]. Optics Express, 2015, 23(20): 25795-25803. doi: 10.1364/OE.23.025795
    [21]
    LIU H B, LIN H J, YAO L SH. Calibration method for projector-camera-based telecentric fringe projection profilometry system[J]. Optics Express, 2017, 25(25): 31492-31508. doi: 10.1364/OE.25.031492
    [22]
    安东, 陈李, 丁一飞, 等. 光栅投影相位法系统模型及标定方法[J]. 中国光学,2015,8(2):248-254. doi: 10.3788/co.20150802.0248

    AN D, CHEN L, DING Y F, et al. Optical system model and calibration of grating projection phase method[J]. Chinese Optics, 2015, 8(2): 248-254. (in Chinese) doi: 10.3788/co.20150802.0248
    [23]
    丁一飞, 王永红, 胡悦, 等. 样本块匹配光栅投影阶梯标定方法[J]. 中国测试,2016,42(8):7-12. doi: 10.11857/j.issn.1674-5124.2016.08.002

    DING Y F, WANG Y H, HU Y, et al. Step calibration method of grating projection based on exemplar matching[J]. China Measurement &Test, 2016, 42(8): 7-12. (in Chinese) doi: 10.11857/j.issn.1674-5124.2016.08.002
    [24]
    LU P, SUN CH K, LIU B, et al. Accurate and robust calibration method based on pattern geometric constraints for fringe projection profilometry[J]. Applied Optics, 2017, 56(4): 784-794. doi: 10.1364/AO.56.000784
    [25]
    CHEN Z, LIAO H Y, ZHANG X M. Telecentric stereo micro-vision system: calibration method and experiments[J]. Optics and Lasers in Engineering, 2014, 57: 82-92. doi: 10.1016/j.optlaseng.2014.01.021
    [26]
    HU Y, CHEN Q, LI H Y, et al. Absolute three-dimensional micro surface profile measurement based on a Greenough-type stereomicroscope[J]. Measurement Science and Technology, 2017, 28(4): 045004. doi: 10.1088/1361-6501/aa5a2d
    [27]
    Overview: DLP products[EB/OL]. [2020-10-18]. http://www.ti.com/dlp-chip/overview.html.
    [28]
    肖萍萍. 基于光栅投射的小尺寸物体三维形状测量系统研究[D]. 武汉: 华中科技大学, 2019.

    XIAO P P. Research on 3D shape measurement system of small scale object based on grating projection[D]. Wuhan: Huazhong University of Science and Technology, 2019. (in Chinese).
    [29]
    VAN DER JEUGHT S, SOONS J A M, DIRCKX J J J. Real-time microscopic phase-shifting profilometry[J]. Applied Optics, 2015, 54(15): 4953-4959.
    [30]
    CHEN L C, LIAO CH CH, LAI M J. Full-field micro surface profilometry using digital fringe projection with spatial encoding principle[J]. Journal of Physics:Conference Series, 2005, 13: 147-150. doi: 10.1088/1742-6596/13/1/034
    [31]
    LI A M, PENG X, YIN Y K, et al. Fringe projection based quantitative 3D microscopy[J]. Optik, 2013, 124(21): 5052-5056. doi: 10.1016/j.ijleo.2013.03.070
    [32]
    LI D, LIU CH Y, TIAN J D. Telecentric 3D profilometry based on phase-shifting fringe projection[J]. Optics Express, 2014, 22(26): 31826-31835. doi: 10.1364/OE.22.031826
    [33]
    PENG J ZH, WANG M, DENG N N, et al. Distortion correction for microscopic fringe projection system with Scheimpflug telecentric lens[J]. Applied Optics, 2015, 54(34): 10055-10062. doi: 10.1364/AO.54.010055
    [34]
    YIN Y K, WANG M, GAO B Z, et al. Fringe projection 3D microscopy with the general imaging model[J]. Optics Express, 2015, 23(5): 6846-6857. doi: 10.1364/OE.23.006846
    [35]
    WANG M, YIN Y K, DENG D N, et al. Improved performance of multi-view fringe projection 3D microscopy[J]. Optics Express, 2017, 25(16): 19408-19421. doi: 10.1364/OE.25.019408
    [36]
    HU Y, CHEN Q, FENG SH J, et al. A new microscopic telecentric stereo vision system-calibration, rectification, and three-dimensional reconstruction[J]. Optics and Lasers in Engineering, 2019, 113: 14-22. doi: 10.1016/j.optlaseng.2018.09.011
    [37]
    HU Y, LIANG Y CH, TAO T Y, et al. Dynamic 3D measurement of thermal deformation based on geometric-constrained stereo-matching with a stereo microscopic system[J]. Measurement Science and Technology, 2019, 30(12): 125007. doi: 10.1088/1361-6501/ab35a1
    [38]
    QUAN C, TAY C J, HE X Y, et al. Microscopic surface contouring by fringe projection method[J]. Optics &Laser Technology, 2002, 34(7): 547-552.
    [39]
    WANG W H, WONG Y S, HONG G S. 3D measurement of crater wear by phase shifting method[J]. Wear, 2006, 261(2): 164-171. doi: 10.1016/j.wear.2005.09.036
    [40]
    张莲涛, 卢荣胜, 程子怡. 基于相移偏折法的高反射表面面形测量技术[J]. 光子学报,2020,49(1):0112002. doi: 10.3788/gzxb20204901.0112002

    ZHANG L T, LU R SH, CHENG Z Y. Measurement technique of high-reflected surface based on phase measuring deflectometry[J]. Acta Photonica Sinica, 2020, 49(1): 0112002. (in Chinese) doi: 10.3788/gzxb20204901.0112002
    [41]
    LIU X H, ZHANG Z H, GAO N, et al. 3D shape measurement of diffused/specular surface by combining fringe projection and direct phase measuring deflectometry[J]. Optics Express, 2020, 28(19): 27561-27574. doi: 10.1364/OE.402432
    [42]
    陶迁, 周志峰, 吴明晖, 等. 基于相位测量偏折术的反射表面缺陷检测[J]. 液晶与显示,2020,35(12):1315-1322. doi: 10.37188/YJYXS20203512.1315

    TAO Q, ZHOU ZH F, WU M H, et al. Detection of reflective surface defects based on phase measuring deflectometry[J]. Chinese Journal of Liquid Crystals and Displays, 2020, 35(12): 1315-1322. (in Chinese) doi: 10.37188/YJYXS20203512.1315
    [43]
    王月敏, 张宗华, 高楠. 基于全场条纹反射的镜面物体三维面形测量综述[J]. 光学 精密工程,2018,26(5):1014-1027. doi: 10.3788/OPE.20182605.1014

    WANG Y M, ZHANG Z H, GAO N. Review on three-dimensional surface measurements of specular objects based on full-field fringe reflection[J]. Optics and Precision Engineering, 2018, 26(5): 1014-1027. (in Chinese) doi: 10.3788/OPE.20182605.1014
    [44]
    ZHANG L, CHEN Q, ZUO CH, et al. High-speed high dynamic range 3D shape measurement based on deep learning[J]. Optics and Lasers in Engineering, 2020, 134: 106245. doi: 10.1016/j.optlaseng.2020.106245
    [45]
    JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi: 10.1016/j.optlaseng.2011.11.021
    [46]
    RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi: 10.1016/j.jvcir.2017.12.003
    [47]
    ZHANG S. Rapid and automatic optimal exposure control for digital fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106029. doi: 10.1016/j.optlaseng.2020.106029
    [48]
    CHEN CH, GAO N, WANG X J, et al. Adaptive projection intensity adjustment for avoiding saturation in three-dimensional shape measurement[J]. Optics Communications, 2018, 410: 694-702. doi: 10.1016/j.optcom.2017.11.009
    [49]
    WANG J H, YANG Y X. High-speed three-dimensional measurement technique for object surface with a large range of reflectivity variations[J]. Applied Optics, 2018, 57(30): 9172-9182. doi: 10.1364/AO.57.009172
    [50]
    SONG ZH, JIANG H L, LIN H B, et al. A high dynamic range structured light means for the 3D measurement of specular surface[J]. Optics and Lasers in Engineering, 2017, 95: 8-16. doi: 10.1016/j.optlaseng.2017.03.008
    [51]
    LIU Y ZH, FU Y J, CAI X Q, et al. A novel high dynamic range 3D measurement method based on adaptive fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106004. doi: 10.1016/j.optlaseng.2020.106004
    [52]
    万钇良, 王建立, 张楠. 一种基于相位相关与子图像的偏振图像配准方法[J]. 液晶与显示,2019,34(5):530-536. doi: 10.3788/YJYXS20193405.0530

    WAN Y L, WANG J L, ZHANG N. Polarized image registration method based on phase correlation and sub-graph[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(5): 530-536. (in Chinese) doi: 10.3788/YJYXS20193405.0530
    [53]
    RIVIERE J, RESHETOUSKI I, FILIPI L, et al. Polarization imaging reflectometry in the wild[J]. ACM Transactions on Graphics, 2017, 36(6): 206.
    [54]
    FENG SH J, ZHANG Y ZH, CHEN Q, et al. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique[J]. Optics and Lasers in Engineering, 2014, 59: 56-71. doi: 10.1016/j.optlaseng.2014.03.003
    [55]
    BENVENISTE R, ÜNSALAN C. Binary and ternary coded structured light 3D scanner for shiny objects[M]//GELENBE E, LENT R, SAKELLARI G, et al. . Computer and Information Sciences. Dordrecht: Springer, 2011: 241-244.
    [56]
    BENVENISTE R, ÜNSALAN C. A color invariant for line stripe-based range scanners[J]. The Computer Journal, 2011, 54(5): 738-753. doi: 10.1093/comjnl/bxq014
    [57]
    BENVENISTE R, ÜNSALAN C. Nary coded structured light-based range scanners using color invariants[J]. Journal of Real-Time Image Processing, 2014, 9(2): 359-377. doi: 10.1007/s11554-011-0235-4
    [58]
    MENG L F, LU L Y, BEDARD N, et al. . Single-shot specular surface reconstruction with gonio-plenoptic imaging[C]. Proceedings of 2015 IEEE International Conference on Computer Vision, IEEE, 2015.
    [59]
    ZHANG L, CHEN Q, ZUO CH, et al. High dynamic range and real-time 3D measurement based on a multi-view system[J]. Proceedings of SPIE, 2019, 11427: 1142715.
    [60]
    HU Y, CHEN Q, LIANG Y CH, et al. Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme[J]. Optics and Lasers in Engineering, 2019, 122: 1-7. doi: 10.1016/j.optlaseng.2019.05.019
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(6)  / Tables(4)

    Get Citation
    PDF
    XML
    Article views(3692) PDF downloads(591) Cited by()
    Proportional views
    Related

    Copyright© 2012 Editorial Office of Chinese Optics    吉ICP备11002662号

    Address: No. 3888 Southeast Lake Road, Changchun City, Jilin ProvinceTel: 投稿问题：0431-84627061（李老师）；财务问题：0431-86176852（王老师）；邮寄及发行：0431-86176862（张老师）

    Email: chineseoptics@ciomp.ac.cnChina Pos: 130033

    Supported by: Beijing Renhe Information Technology Co. Ltd  support: info@rhhz.net

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return