Improving terahertz imaging by light field processing
doi: 10.37188/CO.EN.2022-0005
-
Abstract:
Terahertz (THz) technology becomes increasingly important nowadays, especially in testing and security applications. Extending the field of view and increasing the imaging quality are both vital challenges for THz imaging. To address these problems, we build a THz light field imaging system based on a single-camera scanning configuration, which utilizes the 4D information of the spatial and angular distribution of THz waves. Based on the 4D plenoptic function and the parameterization method with two parallel planes, the intensity consistency of THz propagation is used for refocusing calculation, then a series of refocused images can be obtained by integrating original light field images corresponding to different imaging distances and views. Compared with the original THz imaging, the field of view and the imaging quality of the THz light field imaging are effectively improved. In our experiment, the field of view was enlarged by a factor of 1.84 and the resolution increased from 1.3 mm to 0.7 mm. Furthermore, information on some obscured targets could also be retrieved by defocusing the obstructions. This method could improve the imaging quality of THz imaging as well as expand its functions, which inspires a new way for THz nondestructive testing (NDT) and security inspection.
-
Key words:
- terahertz imaging /
- light field imaging /
- THz nondestructive testing
摘要:近年来，太赫兹技术在检测、安保等诸多领域变得日益重要。如何扩大视场和提高成像质量是太赫兹成像的关键。针对以上问题，本文基于单相机扫描方式搭建了一个太赫兹光场成像系统以同时实现对太赫兹波空间和角度分布信息的利用。基于四维全光函数和双平面参数化方法，利用太赫兹光场传播过程中的强度一致性进行了重聚焦运算，通过对光场进行积分即可得到一系列对应不同视角、不同成像距离的结果。与初始的太赫兹成像结果相比，所搭建成像系统的视场和成像质量都得到了有效提升。在本文实验中，视场扩大了1.84倍，分辨率从1.3 mm提高到了0.7 mm。此外，部分被遮挡的目标信息也可通过使遮挡物离焦得到恢复。结果证明所搭系统能够提高太赫兹成像质量并拓展其功能，从而为太赫兹无损测量和安全检测提供了新的思路。
-
1. Introduction
Terahertz (THz) waves are located between the millimeter-wave band and the infrared band, with advantages that include low energy consumption, strong penetration and no ionizing effect on organisms[1]. It can transmit through some objects opaque in visible wavelengths such as paper and cloth, thus providing great potential for nondestructive testing (NDT)[2]. With tremendous progress of sources and detectors[3-6], THz technology has been applied to security apparatus based on millimeter waves in airports[1,7-8] and imaging systems for medical detection[9-10]. However, compared with x-ray imaging, the wavelength of the THz band is relatively long, which limits the performance of THz imaging[11] and induces many problems such as a narrow field of view and a low resolution.
Light field imaging is a computational imaging method that could record the spatial and angular distribution of light at the same time[12]. Acquisition approaches of the light field can be divided into three categories[13]: time-sequential capture, multi-sensor capture and multiplexed imaging. Current research of light field imaging is mainly focused on the visible light band[14-15] and has made plentiful achievements, such as light field imaging theories[16-18], numerical measurement[19], denoising methods[20], parallax estimation, depth estimation[21] and imaging of obscured objects[12]. Some researchers have studied the light field imaging of the THz band, and the results show that direction-based light field mapping is feasible for THz frequencies[22-24].
In this paper, we construct a THz light field imaging system using a movable THz camera. According to our experiment’s results, the field of view was enlarged by a factor of 1.84 time, and the resolution was increased from 1.3 mm to 0.7 mm. In addition, by refocusing with proper parameters, we got clear images corresponding to different imaging distances and views, and restored the information of obscured parts of the target. Therefore, we have shown that this method improved the imaging quality, indicating good application potential in nondestructive testing (NDT), non-contact measurement and security inspection.
2. Experiment
2.1 THz light field imaging system
We implement the time-sequential capture mode for a THz light field with a single-camera scanning imaging system. The camera is mounted on a two-dimensional translation stage. The principal planes of the translation platform, the lens and the detector of the camera are parallel with each other. The controller can precisely locate the exact position of the camera to take photos at different sampling points. During this acquisition process, the settings of the camera are fixed. Thus, a series of THz sub-images with their 2D positions can be taken by the camera. The schematic diagram of the system is shown in Fig. 1, which consists of a continuous wave (CW) terahertz source (SIFIR-50 THz), a beam expander collimation system, a two-dimensional translation platform, a filter plate (designed in-house) and a camera (INO IRXCAM-THz-384i).
Figure 1. Schematic diagram of the system. (a) The THz camera and the two-dimensional translation stage; (b) the light filter.2.2 Light field imaging and refocusing
Similar to the visible light band, the spatial and angular distribution of the intensity of THz waves through the samples can be parametrized as L(u,v,x,y) based on two arbitrary reference planes parallel with each other, as shown in Fig. 2(a), where L represents the light intensity, and (u,v) and (s,t) represent the coordinates of the intersections of the light ray and the two planes, respectively. To parameterize our light field, we choose two virtual planes: the original object plane and the principal plane of the camera lens. All sub-images of our system are conjugate with the object plane but captured separately and correspond to different imaging coordinate systems. We choose a reference sub-image (u0,v0) and stipulate the coordinates of points in the object plane to be identical to their corresponding pixels on the reference sub-image. The sampling position of the optical center of the camera is represented by angular coordinates (u,v), while the coordinates of points in the object plane are represented by (s,t), as shown in Fig. 2(b). Without loss of generality, we use the 2D simplification L(u,s) for our demonstration.
Figure 2. Parametrization of the light field. (a) Pairs of points on two parallel planes; (b) parameterization of our systemAs the intensity of the THz waves along every light ray in the light field is consistent, we can calculate the imaging result of an arbitrary virtual object plane parallel with the original object plane by refocusing based on a chosen reference sub-image (u0,v0). During the refocusing process, for a pixel of the reference sub-image with coordinates (u0,v0,s,t), we calculate the spatial coordinate (sα,tα) of its corresponding pixel on the sub-image with spatial coordinate (u,v) following equation (1), while the angular coordinates of all points remain unchanged, as shown in Fig.3.
Figure 3. The resampling diagram of refocusing process. When changing the object distance, the spatial coordinate S of the reference sub-image u0 are constant, while the corresponding object point moves from P to Pα. According to the geometric principles, we can calculate the new coordinate Sα of sub-image u, which is different from the coordinate S1 corresponding to PIn the refocusing process, the coordinates calculated by equation (1) for resampling are generally not integers, so the corresponding intensity value L(u,v,sα,tα) is generated by the inverse distance-weighted linear interpolation of four adjacent pixels’ values.
sα=s−DuldsL⋅1α(u−u0)tα=t−DvldtL⋅1α(v−v0), (1) where, Du, Dv, ds and dt are the unit lengths of the angular coordinates and the spatial coordinates respectively. L and l are the original object distance and imaging distance respectively. α is the ratio of the new object’s distance from the original one. For a specific light field, Dul/dsL and Dvl/dtL are constant parameters, thus the resampling is determined by α, and the imaging result Lα can be represented as the integration of u and v after resampling:
Lα(s,t)=∬L(u,v,s−DuldsL⋅1α(u−u0),t−DvldtL⋅1α(v−v0))dudv. (2) The magnification Mα of the refocused result can be calculated by:
Mα=l/αL. (3) 3. Results and discussion
We choose a series of samples to verify the performance of our system. The focal length f of the camera is 44 mm, the original object distance L is 750 mm, and the pixel size ds×dt is 35 µm×35 µm. In our experiments, the sampling positions of the THz camera are distributed on a grid on the u-o-v plane. The size of the grid and the distances of adjacent points in the u-direction and v-direction are arbitrary chosen. The camera was moved to every position sequentially by the translation stage and recorded a series of sub-images to get the light field information of the target.
Due to the size and intensity of the THz source, the sub images are only partially visible and unclear. During the refocusing process, all sub-images are mapped on the new imaging plane, thus improving the field of view and the resolution of the synthetic result, as shown in Figs.4~6 (color online).
Figure 4. A rubber ring in an envelope (grid size 9×3, Du=Dv=10 mm). (a) The rubber ring (outer diameter is 25 mm; inner diameter is 17 mm) and the envelope; (b) the reference sub-image (u=4, v=2); (c) the refocused image (αL=720 mm); (d) normalized gradient image of the labelled regions of (b) and (c), respectively and their gradient values of labelled horizontal lines corresponding to t=197
Figure 5. A steel board with hollowed parts in a plastic box (grid size 9×1, Du=10 mm). (a) The steel board (length is 68 mm; width is 45 mm) and the plastic box; (b) the reference sub-image (u=5, v=1); (c) the refocused image (αL=730 mm); (d) intensity of labelled horizontal lines corresponding to t=174
Figure 6. Metal wrenches fixed on a polyvinyl chloride(PVC) board by tape. (grid size 9×7, Du=Dv=10 mm) (a) the wrenches (from left to right the diameters are 0.7 mm, 0.9 mm, 1.3 mm and 1.5 mm) and the PVC board; (b) the reference sub-image (u=3, v=4); (c) the refocused image (αL=680 mm); (d) the refocused image after histogram equalization.According to Fig.4 (d), the gradient values of the refocused image show four prominent peaks corresponding to the edges of the rubber ring while the central sub-image shows an undistinguished result, of which only the peak of s=208 is clear for identification. The outer diameter and the inner diameter of the rubber ring are 47 pixels and 32 pixels respectively. Based on the refocusing parameters, the estimated diameters are 25.3 mm and 17.2 mm, which shows the potential of using refocused images for measurement. The ratio of the outer and inner diameters is 1.469, which is quite close to the reference truth-value 1.471. The deviation of size may be caused by the transformation of the rubber ring as well as the inaccurate object distance estimation, which can be effectively eliminated by precise calibration and reliable algorithm for focusing evaluation.
In Fig. 5 (b) and Fig. 5 (c), the ratio of the widths of their field of views (intensity thresh: 0.25) is 106 pixels to 195 pixels, which is an expansion by a factor of 1.84. Fig.6(b) and Fig.6(c) show the improvement of resolution from 1.3 mm to 0.7 mm, which are the diameters of the finest wrenches that could be recognized.
To verify the flexibility of our method, we perform an experiment of the same sample with Fig.6, in which we move the sample rather than the camera for sub-image acquisition. The refocusing process shows similar results, as shown in Fig.7. This mode could be more applicable to some cases where the camera needs to be fixed. We can see the difference of noise in Fig.6 and Fig.7. There may be two reasons: first, the relative sample positions of two experiments are not identical; second, the movement of the target of Fig.7 also changes its relative position with the THz source, while the target of Fig.6 is relatively still with the THz source.
Figure 7. Metal wrenches (fixed camera and moving target, grid size 9×5, Du=Dv=10 mm). (a) The wrenches and the PVC board; (b) the reference sub-image (u=5, v=3); (c) the refocused image (αL=700 mm); (d) the refocused image after histogram equalizationWith different sub-image as the reference view, the imaging position of the sample correspondingly changes, as shown in Fig.8(color online). According to Table.1, the mean parallax values per mm the camera moves are 1.825 pixel and 1.850 pixel for u and v directions, respectively. Thus the corresponding estimations of the object distance are 731.8 mm and 721.9 mm, which conform well with the refocusing data. The parallax values generated from these different views could be used for specific 3D measurement with further calibration of the system.
Figure 8. Refocusing images of different sub-views (two PVC bottles with liquid fixed by tape in a high-density polyethylene plastic box, grid size 9×2, Du=10 mm, Dv=20 mm). (a) The bottles (length is 50 mm; diameter is 13 mm; thickness is 2 mm) and the plastic box; (b) one primitive sub-image; (c)(d)(e)(f) are the refocused images with angular coordinates (4,1), (4,2), (1,1) and (9,1) respectively (αL=720 mm).Table 1. Coordinates of sample pixels in different sub-viewsSub-view (4,1) (4,2) (1,1) (9,1) Pixel 1 (192,175) (192,138) (248,175) (99,175) Pixel 2 (238,193) (238,156) (295,193) (145,193) | Show Table
DownLoad:
CSV
Fig.9 shows two imaging results with different refocusing depths. In Fig. 9 (a), the plastic bottles are in focus and the mark on the plastic box are unclear due to defocusing. While in Fig. 9 (b), the mark on the plastic box is in focus and the bottles become blurry. The refocusing results of different object distances expanded the depth of field and could also be used for depth estimation.
Figure 9. Refocusing images of different depths. (a) The image focused on the bottles, αL=720 mm. (b) The image focused on the tape and the plastic box, αL=670 mmFor some situations where the obstructions could not be penetrated by THz waves, the synthetic process also works. As shown in Fig.10, by defocusing the wire mesh, we can restore the information of our target. In Fig.10 (c), the wire mesh is blurred and the steel holed cube is clearer than Fig.10 (b), especially the hole at the center.
Figure 10. A steel holed cube blocked by a piece of wire mesh placed between it and the THz camera. (a) and (b) are two primitive sub-images showing the wire mesh and the obscured target respectively; (c) the refocused image4. Conclusion
Terahertz imaging has many particular advantages for multiple applications. To exceed the limitation of the THz camera’s resolution and field of view as well as expand its functions, we build a THz light field imaging system of time sequential acquisition mode. Experiments show that compared with the original camera, the resolution, the field of view and the depth of field of the system are effectively improved. Besides, the system shows potential in restoration of obscured information, 3D measurement and depth estimation. Moreover, the system implementation and the image acquisition scheme are flexible, which can be modified to adapt different scenarios. Following points can be considered for further improvements based on this system and method: first, the refocusing process can be promoted by optimizing the interpolation and integration algorithm; second, the image acquisition scheme can be more efficient with adaptive design for different targets; third, the measurement function can be effectively developed by applicable calibration approaches.
-
Figure 1. Schematic diagram of the system. (a) The THz camera and the two-dimensional translation stage; (b) the light filter.
Figure 2. Parametrization of the light field. (a) Pairs of points on two parallel planes; (b) parameterization of our system
Figure 3. The resampling diagram of refocusing process. When changing the object distance, the spatial coordinate S of the reference sub-image u0 are constant, while the corresponding object point moves from P to Pα. According to the geometric principles, we can calculate the new coordinate Sα of sub-image u, which is different from the coordinate S1 corresponding to P
Figure 4. A rubber ring in an envelope (grid size 9×3, Du=Dv=10 mm). (a) The rubber ring (outer diameter is 25 mm; inner diameter is 17 mm) and the envelope; (b) the reference sub-image (u=4, v=2); (c) the refocused image (αL=720 mm); (d) normalized gradient image of the labelled regions of (b) and (c), respectively and their gradient values of labelled horizontal lines corresponding to t=197
Figure 5. A steel board with hollowed parts in a plastic box (grid size 9×1, Du=10 mm). (a) The steel board (length is 68 mm; width is 45 mm) and the plastic box; (b) the reference sub-image (u=5, v=1); (c) the refocused image (αL=730 mm); (d) intensity of labelled horizontal lines corresponding to t=174
Figure 6. Metal wrenches fixed on a polyvinyl chloride(PVC) board by tape. (grid size 9×7, Du=Dv=10 mm) (a) the wrenches (from left to right the diameters are 0.7 mm, 0.9 mm, 1.3 mm and 1.5 mm) and the PVC board; (b) the reference sub-image (u=3, v=4); (c) the refocused image (αL=680 mm); (d) the refocused image after histogram equalization.
Figure 7. Metal wrenches (fixed camera and moving target, grid size 9×5, Du=Dv=10 mm). (a) The wrenches and the PVC board; (b) the reference sub-image (u=5, v=3); (c) the refocused image (αL=700 mm); (d) the refocused image after histogram equalization
Figure 8. Refocusing images of different sub-views (two PVC bottles with liquid fixed by tape in a high-density polyethylene plastic box, grid size 9×2, Du=10 mm, Dv=20 mm). (a) The bottles (length is 50 mm; diameter is 13 mm; thickness is 2 mm) and the plastic box; (b) one primitive sub-image; (c)(d)(e)(f) are the refocused images with angular coordinates (4,1), (4,2), (1,1) and (9,1) respectively (αL=720 mm).
Figure 9. Refocusing images of different depths. (a) The image focused on the bottles, αL=720 mm. (b) The image focused on the tape and the plastic box, αL=670 mm
Figure 10. A steel holed cube blocked by a piece of wire mesh placed between it and the THz camera. (a) and (b) are two primitive sub-images showing the wire mesh and the obscured target respectively; (c) the refocused image
Table 1. Coordinates of sample pixels in different sub-views
Sub-view (4,1) (4,2) (1,1) (9,1) Pixel 1 (192,175) (192,138) (248,175) (99,175) Pixel 2 (238,193) (238,156) (295,193) (145,193) 
下载: 导出CSV
-
[1] ZHANG Y D, DENG C, SUN W F, et al. Terahertz continuous-wave transmission imaging system and its application in security inspections[J]. Proceedings of SPIE, 2008, 6840: 684010. [2] MITTLEMAN D M. Twenty years of terahertz imaging [Invited][J]. Optics Express, 2018, 26(8): 9417-9431. doi: 10.1364/OE.26.009417 [3] YAO J Q, LU Y, ZHANG B G, et al. New research progress of THz radiation[J]. Journal of Optoelectronics·Laser, 2005, 16(4): 503-510. [4] LEE A W, HU Q. Real-time, continuous-wave terahertz imaging by use of a microbolometer focal-plane array[J]. Optics Letters, 2005, 30(19): 2563-2565. doi: 10.1364/OL.30.002563 [5] KARPOWICZ N, ZHONG H, ZHANG C L, et al. Compact continuous-wave subterahertz system for inspection applications[J]. Applied Physics Letters, 2005, 86(5): 054105. doi: 10.1063/1.1856701 [6] VALUŠIS G, LISAUSKAS A, YUAN H, et al. Roadmap of terahertz imaging 2021[J]. Sensors, 2021, 21(12): 4092. doi: 10.3390/s21124092 [7] YANG J, RUAN SH CH, ZHANG M, et al. Real-time continuous-wave imaging with a 1.63THz OPTL and a pyroelectric camera[J]. Optoelectronics Letters, 2008, 4(4): 295-298. doi: 10.1007/s11801-008-8036-0 [8] XIAO H, ZHU F. Identification of dangerous goods in human THZ images[C]. Proceedings of the 2018 International Conference on Network, Communication, Computer Engineering (NCCE 2018), Atlantis Press, 2018: 899-903. [9] WOODWARD R M, WALLACE V P, ARNONE D D, et al. Terahertz pulsed imaging of skin cancer in the time and frequency domain[J]. Journal of Biological Physics, 2003, 29(2-3): 257-259. [10] TAO Y H, FITZGERALD A J, WALLACE V P. Non-contact, non-destructive testing in various industrial sectors with terahertz technology[J]. Sensors, 2020, 20(3): 712. doi: 10.3390/s20030712 [11] WONG T M, KAHL M, BOLIVAR P H, et al. . Frequency Modulated Continuous Wave (FMCW) THz Image 3D Superresolution[Z]. arXiv: 1802.05457v1, 2018. [12] LEVOY M. Light fields and computational imaging[J]. Computer, 2006, 39(8): 46-55. doi: 10.1109/MC.2006.270 [13] WU G CH, MASIA B, JARABO A, et al. Light field image processing: an overview[J]. IEEE Journal of Selected Topics in Signal Processing, 2017, 11(7): 926-954. doi: 10.1109/JSTSP.2017.2747126 [14] FANG L, DAI Q H. Computational light field imaging[J]. Acta Optica Sinica, 2020, 40(1): 0111001. doi: 10.3788/AOS202040.0111001 [15] YIN Y K, YU K, YU CH ZH, et al. 3D imaging using geometric light field: a review[J]. Chinese Journal of Lasers, 2021, 48(12): 1209001. (in Chinese) [16] LI Y, WANG X, ZHOU G, et al. Overview of 4D Light Field Representation[J]. Laser &Optoelectronics Progress, 2021, 58(18): 1811012. (in Chinese) [17] TIAN Y, ZENG H Q, HOU J H, et al. Light field image quality assessment via the light field coherence[J]. IEEE Transactions on Image Processing, 2020, 29: 7945-7956. doi: 10.1109/TIP.2020.3008856 [18] COUILLAUD J, ZIOU D. Light field variational estimation using a light field formation model[J]. The Visual Computer, 2020, 36(2): 237-251. doi: 10.1007/s00371-018-1599-2 [19] YAO T, SANG X ZH, WANG P, et al. Depth reconstruction for 3D light-field display based on axially distributed light field[J]. Optical Engineering, 2021, 60(5): 053103. [20] ZHOU G, LI S, LAM E Y. Light field image restoration in low-light environment[J]. Proceedings of SPIE, 2020, 11525: 115251H. [21] GUO C L, JIN J, HOU J H, et al. . Accurate light field depth estimation via an occlusion-aware network[C]. IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2020. [22] JAIN R, GRZYB J, PFEIFFER U R. Terahertz light-field imaging[J]. IEEE Transactions on Terahertz Science and Technology, 2016, 6(5): 649-657. [23] LYU N F, ZUO J, ZHAO Y M, et al. Terahertz synthetic aperture imaging with a light field imaging system[J]. Electronics, 2020, 9(5): 830. doi: 10.3390/electronics9050830 [24] LYU N F, ZUO J, ZHAO Y M, et al. Layer-resolving terahertz light-field imaging based on angular intensity filtering method[J]. Sensors, 2021, 21(22): 7451. doi: 10.3390/s21227451 期刊类型引用(0)
其他类型引用(1)
-
DownLoad:
下载:
计量
- 文章访问数: 583
- HTML全文浏览量: 406
- PDF下载量: 185
- 被引次数: 1
