基于航空单目相机的目标定位技术

王鑫; 李玉芳; 任航; 韩松伟; 刘立刚; 孙明超; 宋策

doi:10.37188/CO.2022-0147

基于航空单目相机的目标定位技术

doi: 10.37188/CO.2022-0147

王鑫^1,,
李玉芳^2, ,,
任航¹,
韩松伟¹,
刘立刚¹,
孙明超¹,
宋策¹

1.
中国科学院长春光学精密机械与物理研究所，吉林长春 130033
2.
中国第一汽车集团有限公司研发总院，吉林长春 130013

基金项目: 国家自然科学基金面上项目（No. 62175233）；吉林省科技发展计划（No. 20200403057SF)

详细信息

作者简介:
王　鑫（1993—），男，黑龙江肇东人，硕士，助理研究员，2016年于哈尔滨工业大学获得控制工程专业硕士学位。现就职于中国科学院长春光学精密机械与物理研究所，主要从事图像处理方面的研究。E-mail：14S104020@hit.edu.cn

李玉芳（1994—），女，黑龙江肇东人，硕士，中级工程师，2017年于哈尔滨工业大学获得机械设计及理论专业硕士学位。现就职于中国第一汽车集团有限公司，主要从事汽车设计方面的研究。E-mail：910341666@qq.com

中图分类号: TP752.2
计量
- 文章访问数: 606
- HTML全文浏览量: 146
- PDF下载量: 251
- 被引次数: 0
出版历程
- 收稿日期: 2022-07-04
- 修回日期: 2022-07-25
- 网络出版日期: 2023-02-22

Targeting technology based on aerial monocular camera

1.
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
2.
Research and Development Institute, China FAW Group Corporation, Changchun 130013, China

Funds: Supported by National Natural Science Foundation of China (No. 62175233); Science and Technology Development Program of Jilin Providence (No. 20200403057SF)

More Information

Corresponding author: 910341666@qq.com

摘要

摘要:
为了提升无人机机载光电吊舱在特殊工况条件下的自主侦察能力，本文结合实际工程项目研发了适用于机载光电吊舱的目标定位技术，并在嵌入式GPU（Graphics Processing Unit，型号为Jetson-TX2i）上实现了功能验证。首先，提出一种改进的SURF（Speeded Up Robust Features）算法与GPU加速数字图像处理方案，对不同焦距下获取的两幅图像进行实时特征检测与匹配。接着，利用几何交比不变性校正图像边缘畸变特征点的位置信息。最后，使用最小二乘法估计目标深度信息并结合四元数空间模型确定目标姿态信息，从而实现目标定位。实验结果表明，改进的SURF算法在特征匹配方面，精度和速度都优于经典的SURF算法。若角点特征位置误差控制在一个像素以内，深度误差不超过2%，方位角、俯仰角和滚转角的角度误差分别小于4°，5°和2°，这一误差满足机载光电吊舱的目标定位准确性需求。此外，处理一组（两帧）分辨率为1080P的图像，利用GPU加速可以将处理时间提升至74 ms，这一速度满足机载光电吊舱数据处理的实时性需求。
- 特征提取 /
- 特征匹配 /
- 目标定位技术 /
- 姿态
Abstract:
In order to improve the autonomous detection ability of the airborne optronics pod of a UAV under special working conditions, this paper developed a targeting technology suitable for the airborne optronics pod in an actual engineering project, and realized the functional verification on the embedded GPU (Graphics Processing Unit, Jetson-TX2i platform model). Firstly, we proposed an improved SURF (Speeded Up Robust Features) algorithm and GPU-accelerated digital image processing scheme to detect and match the real-time features of two images acquired at different focal lengths. Secondly, geometric cross-ratio invariance was used to correct the position information of distorted feature points at image edges. Finally, we used the least square method to estimate the depth information of the target and combines the quaternion space model to determine the attitude information of the target to determine its position. Experimental results show that the improved SURF algorithm is superior to the classical SURF algorithm in feature matching accuracy and speed. If the corner characteristic position error is controlled within one pixel, the depth error is no more than 2% and the angle errors of azimuth, pitch and roll angles are less than 4°, 5° and 2°, respectively. This error meets the target positioning accuracy requirements of the airborne optronics pod. In addition, when processing a set of images (two frames) at 1080 P resolution, the processing time can be increased to 74 ms through GPU acceleration, which meets the real-time demand for data processing in the airborne optronics pod.
- feature extraction /
- feature matching /
- targeting technology /
- attitude

HTML全文

图 1 获取目标位姿流程图

Figure 1. Flowchart of obtaining target depth and attitude

下载: 全尺寸图片幻灯片

图 2 特征实时匹配流程图

Figure 2. Flowchart of real-time feature matching

下载: 全尺寸图片幻灯片

图 3 特征描述示意图

Figure 3. Schematic diagram of feature description

下载: 全尺寸图片幻灯片

图 4 射影变换示意图

Figure 4. Schematic diagram of projective transformation

下载: 全尺寸图片幻灯片

图 5 棋盘格投影示意图

Figure 5. Schematic diagram of checkerboard projection

下载: 全尺寸图片幻灯片

图 6 变焦距系统深度估计原理图

Figure 6. Schematic of zoom system depth estimation

下载: 全尺寸图片幻灯片

图 7 三种坐标系示意图

Figure 7. Schematic diagram of three coordinate systems

下载: 全尺寸图片幻灯片

图 8 本文算法在不同条件下的特征提取结果

Figure 8. Feature extraction results obtained by the proposed algorithm under different conditions

下载: 全尺寸图片幻灯片

图 9 3种算法在不同条件下的角点检测重复率

Figure 9. Corner detection repetition rates obtained by three algorithms under different conditions

下载: 全尺寸图片幻灯片

图 10 实验用的靶标与坐标系

Figure 10. The target and coordinate system used in the experiment

下载: 全尺寸图片幻灯片

图 11 特征点偏差引起的深度误差

Figure 11. Depth error caused by feature point deviation

下载: 全尺寸图片幻灯片

图 12 特征点定位精度与解算姿态角的关系

Figure 12. The relation between position accuracy of feature point and calculated attitude angle

下载: 全尺寸图片幻灯片

表 1 3种算法的正确匹配率和处理速度

Table 1. Correct matching rates and processing speeds of the three algorithms (Unit: ms)

图像	本文算法		SURF		Harris-SIFT
图像	${r_{\rm{c}}}$	$ t $	${r_{\rm{c}}}$	$ t $	${r_{\rm{c}}}$	$ t $
1	6.17	7.17	3.01	11.83	6.17	10.60
2	74.85	9.28	76.89	30.26	51.33	39.25
3	50.15	8.87	51.50	20.27	34.39	26.30
4	28.44	7.82	29.21	11.49	19.51	14.92
5	62.13	9.01	63.82	25.67	46.26	35.27
6	39.67	8.32	40.75	13.67	27.94	20.82

下载: 导出CSV

表 2 两种相机标定算法的结果对比

Table 2. Comparison of the results obtained by two camera calibration algorithms

参数	本文			张方法
使用图像	$ {{\boldsymbol{I}}_1} $	$ {{\boldsymbol{I}}_{25}} $	$ {{\boldsymbol{I}}_{49}} $	$ {{\boldsymbol{I}}_1},{{\boldsymbol{I}}_2}, \ldots ,{{\boldsymbol{I}}_{50}} $
$ {{{k_1}}/ { \times {{10}^{ - 3}}}} $	3.132	3.131	3.132	3.129
$ {{{k_2}}/ { \times {{10}^{ - 3}}}} $	3.227	3.225	3.226	3.225
$ {{{k_3}} /{ \times {{10}^{ - 3}}}} $	0.002	0.001	0.001	0.002
$ {p_1} $	−0.0001	−0.00009	−0.00008	−0.0001
$ {p_2} $	0.0001	0.0001	0.0003	0.0001

下载: 导出CSV

表 3 两种方案在不同分辨率条件下的处理速度

Table 3. The processing speeds of the two algorithms at different resolutions (Unit: ms)

图像分辨率	处理时间
图像分辨率	CPU方案	GPU方案
640×320	84	17
640×512	161	32
1024×768	304	52
1920×1080	596	74

下载: 导出CSV

参考文献(18)

[1]	张金隆, 吴珊, 龚业明. 中国智能机械制造评价及发展研究[J]. 中国机械工程,2020,31(4):451-458. doi: 10.3969/j.issn.1004-132X.2020.04.012 ZHANG J L, WU SH, GONG Y M. Research on evaluation and development of intelligent machinery manufacturing in China[J]. China Mechanical Engineering, 2020, 31(4): 451-458. (in Chinese) doi: 10.3969/j.issn.1004-132X.2020.04.012
[2]	WEI M S, XING F, YOU ZH. A real-time detection and positioning method for small and weak targets using a 1D morphology-based approach in 2D images[J]. Light:Science &Applications, 2018, 7: 18006.
[3]	YUAN X Y, JI M Q, WU J M, et al. A modular hierarchical array camera[J]. Light:Science &Applications, 2021, 10(1): 37.
[4]	PINIES P, LUPTON T, SUKKARIEH S, et al. . Inertial aiding of inverse depth SLAM using a monocular camera[C]. IEEE International Conference on Robotics & Automation, IEEE, 2007: 2797-2802.
[5]	张中, 黄俊杰, 汪明明. 一种基于实时视频的位姿估计方法: 中国, 112419409A[P]. 2021-02-26. ZHANG ZH, HUANG J J, WANG M M. A pose estimation method based on real-time video: CN, 112419409A[P]. 2021-02-26. (in Chinese)
[6]	丁国绅, 乔延利, 易维宁, 等. 基于光谱图像空间的改进SIFT特征提取与匹配[J]. 北京理工大学学报,2022,42(2):192-199. DING G SH, QIAO Y L, YI W N, et al. Improved SIFT feature extraction and matching based on spectral image space[J]. Transactions of Beijing Institute of Technology, 2022, 42(2): 192-199. (in Chinese)
[7]	LESHCHENKO V E, KESSEL A, JAHN O, et al. On-target temporal characterization of optical pulses at relativistic intensity[J]. Light:Science &Applications, 2019, 8: 96.
[8]	左威健, 胡立华, 刘爱琴, 等. 基于动态拓展的特征匹配方法[J]. 计算机工程与设计,2022,43(3):778-785. ZUO W J, HU L H, LIU A Q, et al. Feature matching method based on dynamic expansion[J]. Computer Engineering and Design, 2022, 43(3): 778-785. (in Chinese)
[9]	ZHANG ZH Y. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334. doi: 10.1109/34.888718
[10]	ZHANG Y, DU SH Q, HIRAKAWA K. Deep-nanometer-scale terahertz spectroscopy using a transistor geometry with metal nanogap electrodes[J]. Light:Advanced Manufacturing, 2021, 2(4): 460-472.
[11]	王月, 李阳, 金映谷, 等. 拼接图像的径向畸变矫正技术[J]. 大连民族大学学报,2021,23(3):228-231. doi: 10.3969/j.issn.1009-315X.2021.03.009 WANG Y, LI Y, JIN Y G, et al. Radial distortion correction technology of image mosaic[J]. Journal of Dalian Minzu University, 2021, 23(3): 228-231. (in Chinese) doi: 10.3969/j.issn.1009-315X.2021.03.009
[12]	HAJANO S, NAZ B, TALPUR S. Area and feature based image registration using template matching and SURF algorithm[C]. INCCST'20 Second International Conference, 2021: 109-113.
[13]	雒福佐. 基于小波分析与RANSAC匹配的雷达图像拼接研究[J]. 自动化与仪器仪表,2022(2):199-203,208. LUO F Z. Research on radar image mosaic based on wavelet analysis[J]. Automation &Instrumentation, 2022(2): 199-203,208. (in Chinese)
[14]	杨建柏, 赵建, 孙强. 基于交比不变性的投影仪标定[J]. 中国光学,2021,14(2):320-328. doi: 10.37188/CO.2020-0111 YANG J B, ZHAO J, SUN Q. Projector calibration based on cross ratio invariance[J]. Chinese Optics, 2021, 14(2): 320-328. (in Chinese) doi: 10.37188/CO.2020-0111
[15]	LI Q, ZENG Y A, JIU ZH X, et al. Distortion correction of optical system based on neural network and automated generation of reference points[J]. Proceedings of SPIE, 2009, 7283: 72831R. doi: 10.1117/12.828680
[16]	HARTLEY R, KANG S B. Parameter-free radial distortion correction with center of distortion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(8): 1309-1321. doi: 10.1109/TPAMI.2007.1147
[17]	陈天择, 葛宝臻, 罗其俊. 重投影优化的自由双目相机位姿估计方法[J]. 中国光学,2021,14(6):1400-1409. doi: 10.37188/CO.2021-0105 CHEN T Z, GE B ZH, LUO Q J. Pose estimation for free binocular cameras based on reprojection error optimization[J]. Chinese Optics, 2021, 14(6): 1400-1409. (in Chinese) doi: 10.37188/CO.2021-0105
[18]	杜明鑫, 闫钰锋, 张燃, 等. 基于透镜阵列的三维姿态角度测量[J]. 中国光学,2022,15(1):45-55. DU M X, YAN Y F, ZHANG R, et al. 3D position angle measurement based on a lens array[J]. Chinese Optics, 2022, 15(1): 45-55. (in Chinese)