重投影优化的自由双目相机位姿估计方法

陈天择; 葛宝臻; 罗其俊

doi:10.37188/CO.2021-0105

重投影优化的自由双目相机位姿估计方法

doi: 10.37188/CO.2021-0105

陈天择^{1, 2,},
葛宝臻^{1, 2, ,},
罗其俊^{1, 2, 3,}

1.
天津大学精密仪器与光电子工程学院，天津 300072
2.
光电信息技术教育部重点实验室，天津 300072
3.
中国民航大学电子信息与自动化学院，天津 300300

基金项目: 国家自然科学基金项目（No. 61535008）

详细信息

作者简介:
陈天择（1995—），男，黑龙江哈尔滨人，硕士研究生，2018年于天津大学获得学士学位，现就读于天津大学精密仪器与光电子工程学院，攻读光学工程专业硕士学位，主要研究方向为计算机视觉。E-mail：chentianze@tju.edu.cn

葛宝臻（1964—），男，内蒙古卓资人，博士，教授，博士生导师，主要从事三维彩色数字成像技术、光电检测技术、激光粒子测量方面的研究。E-mail：gebz@tju.edu.cn

罗其俊（1982—），男，湖北孝感人，博士，2008年于中国民航大学获得硕士学位，现为中国民航大学电子信息与自动化学院讲师，主要从事机器视觉和只能控制系统方面的研究。E-mail：qjluo@cauc.edu.cn

中图分类号: TP391.4
计量
- 文章访问数: 1509
- HTML全文浏览量: 609
- PDF下载量: 200
- 被引次数: 0
出版历程
- 收稿日期: 2021-05-14
- 修回日期: 2021-06-02
- 网络出版日期: 2021-08-11
- 刊出日期: 2021-11-19

Pose estimation for free binocular cameras based on reprojection error optimization

CHEN Tian-ze^{1, 2
,},
GE Bao-zhen^{1, 2
, ,},
LUO Qi-jun^{1, 2, 3
,}

1.
School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, China
2.
Key Laboratory of Opto-Electronics Information and Technical Science for Ministry of Education, Tianjin 300072, China
3.
College of Electronic Information and Automation, Civil Aviation University of China, Tianjin 300300, China

Funds: Supported by National Natural Science Foundation of China (No. 61535008)

More Information

Corresponding author: gebz@tju.edu.cn

摘要

摘要: 在自由双目立体视觉系统中，左右相机自由旋转，导致相机位姿发生实时变化。针对这一问题，提出一种重投影优化的自由双目相机位姿估计方法。通过分解相邻两幅图像间的单应矩阵估计相机的运动参数；将其作为初值计算重叠拍摄区域内特征点云的重投影误差，构建目标函数；利用非线性优化算法优化目标函数，得到最优的相机运动参数，结合相机旋转前的位姿参数，计算当前位置的相机位姿。仿真实验表明，位姿估计误差随重投影误差的减小而减小，所提算法能快速稳定收敛至全局最优值；水泥模型的三维重建实验表明，利用所提的相机位姿估计算法能够有效生成模型的三维点云，并可实现相邻点云的高精度拼接，拼接点云模型上两点距离的平均误差为1.68%。
- 位姿估计 /
- 立体视觉 /
- 重投影误差 /
- 非线性优化 /
- 三维重建
Abstract: To solve the real-time change of the camera poses caused by the rotation of cameras in free binocular stereo vision, a method for estimating the poses of free binocular cameras based on reprojection error optimization is proposed. The movement paraments of cameras are estimated by decomposing the homography matrix between two adjacent images. Then, the reprojection error of feature points in the overlapping area is calculated, and the objective function is constructed by using the movement parameters as initial values. Finally, the objective function is optimized by the nonlinear optimization algorithm, and the current camera poses are calculated by combining with the optimal movement parameters and the camera poses before rotation. Simulations indicate that the pose estimation error declines with a decrease in reprojection error and the proposed method can converge to a globally optimal solution both rapidly and stably. An experiment of 3D reconstruction of cement models indicates that 3D point clouds of models are generated effectively with the proposed method, the adjacent point clouds are stitched accurately, and the average error of distance between any two points on the stitched point clouds is 1.68%.
- pose estimation /
- stereo vision /
- reprojection error /
- nonlinear optimization /
- 3D reconstruction

HTML全文

图 1 自由双目立体视觉系统中相邻两次测量

Figure 1. Two adjacent measurements of the free binocular vision system

下载: 全尺寸图片幻灯片

图 2 位姿估计误差随重投影误差的变化

Figure 2. Changes in pose estimation error with reprojection error

下载: 全尺寸图片幻灯片

图 3 不同条件下算法的收敛情况

Figure 3. Convergence of the algorithm under various conditions

下载: 全尺寸图片幻灯片

图 4 不同参考点数目对算法的影响

Figure 4. Changes in post estimation error to the algorithm with different numbers of reference points

下载: 全尺寸图片幻灯片

图 5 不同噪声对算法的影响

Figure 5. Changes in post estimation error to the algorithm under different noise

下载: 全尺寸图片幻灯片

图 6 自由双目立体视觉装置图

Figure 6. Diagram of free binocular stereo vision device

下载: 全尺寸图片幻灯片

图 7 双目相机采集图像

Figure 7. Images obtained by binocular cameras

下载: 全尺寸图片幻灯片

图 8 三维重建结果

Figure 8. 3D reconstruction results

下载: 全尺寸图片幻灯片

表 1 标志点间距

Table 1. Distance between mark points （mm）

实验次数	AB间距	CD间距	EF间距
1	498.073	199.762	501.328
2	501.137	197.037	501.670
3	500.569	198.053	502.404
4	502.842	200.459	502.634

下载: 导出CSV

参考文献(22)

[1]	郑圣子, 李湘旭, 孙志超. 新型移动机器人激光测距雷达的研究[J]. 计算机测量与控制,2011,19(5):1094-1097. ZHENG SH Z, LI X X, SUN ZH CH. Novel design of low cost mobile robot lazer scanner[J]. Computer Measurement &Control, 2011, 19(5): 1094-1097. (in Chinese)
[2]	李肖, 葛宝臻, 罗其俊, 等. 自由双目立体视觉摄像机动态外参数的获取[J]. 计算机应用,2017,37(10):2888-2894. doi: 10.11772/j.issn.1001-9081.2017.10.2888 LI X, GE B ZH, LUO Q J, et al. Acquisition of camera dynamic extrinsic parameters in free binocular stereo vision system[J]. Journal of Computer Applications, 2017, 37(10): 2888-2894. (in Chinese) doi: 10.11772/j.issn.1001-9081.2017.10.2888
[3]	于海, 万秋华, 孙莹, 等. 一种自适应安装的高精度图像式角位移测量装置[J]. 中国光学,2020,13(3):510-516. YU H, WAN Q H, SUN Y, et al. A high precision image angular displacement measurement device with self-adaptive installation[J]. Chinese Optics, 2020, 13(3): 510-516. (in Chinese)
[4]	SWEENEY C, SATTLER T, HÖLLERER T, et al.. Optimizing the viewing graph for structure-from-motion[C]. 2015 IEEE International Conference on Computer Vision (ICCV), IEEE, 2015: 801-809.
[5]	MOULON P, MONASSE P, MARLET R. Global fusion of relative motions for robust, accurate and scalable structure from motion[C]. 2013 IEEE International Conference on Computer Vision, IEEE, 2013: 3248-3255.
[6]	ZHU S Y, ZHANG R Z, ZHOU L, et al.. Very large-scale global SfM by distributed motion averaging[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018: 4568-4577.
[7]	ZHUANG B B, CHEONG L F, LEE G H. Baseline desensitizing in translation averaging[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018: 4539-4547.
[8]	NISTER D. An efficient solution to the five-point relative pose problem[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(6): 756-770. doi: 10.1109/TPAMI.2004.17
[9]	杜瑞建, 葛宝臻, 陈雷. 多视高分辨率纹理图像与双目三维点云的映射方法[J]. 中国光学,2020,13(5):1055-1064. doi: 10.37188/CO.2020-0034 DU R J, GE B ZH, CHEN L. Texture mapping of multi-view high-resolution images and binocular 3D point clouds[J]. Chinese Optics, 2020, 13(5): 1055-1064. (in Chinese) doi: 10.37188/CO.2020-0034
[10]	SNAVELY N, SEITZ S M, SZELISKI R. Photo tourism: exploring photo collections in 3D[J]. ACM Transactions on Graphics, 2006, 25(3): 835-846. doi: 10.1145/1141911.1141964
[11]	WU CH CH. Towards linear-time incremental structure from motion[C]. 2013 International Conference on 3D Vision - 3DV 2013, IEEE, 2013: 127-134.
[12]	SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016: 4104-4113.
[13]	LEPETIT V, MORENO-NOGUER F, FUA P. EPnP: an accurate O(n) solution to the PnP problem[J]. International Journal of Computer Vision, 2009, 81(2): 155-166. doi: 10.1007/s11263-008-0152-6
[14]	PENATE-SANCHEZ A, ANDRADE-CETTO J, MORENO-NOGUER F. Exhaustive linearization for robust camera pose and focal length estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(10): 2387-2400. doi: 10.1109/TPAMI.2013.36
[15]	李正炜, 王建立, 吴元昊, 等. 基于单站地基望远镜的空间目标姿态估计方法[J]. 中国光学,2016,9(3):371-378. doi: 10.3788/co.20160903.0371 LI ZH W, WANG J L, WU Y H, et al. Method of attitude estimation for space object based on single ground-based telescope[J]. Chinese Optics, 2016, 9(3): 371-378. (in Chinese) doi: 10.3788/co.20160903.0371
[16]	ENGEL J, SCHÖPS T, CREMERS D. LSD-SLAM: large-scale direct monocular SLAM[C]. Proceedings of the 13th European Conference, Springer, 2014: 834-849.
[17]	张可, 杨灿坤, 周春平, 等. 无人机视频图像运动目标检测算法综述[J]. 液晶与显示,2019,34(1):98-109. doi: 10.3788/YJYXS20193401.0098 ZHANG K, YANG C K, ZHOU CH P, et al. Review of moving target detection algorithms for UAV video images[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(1): 98-109. (in Chinese) doi: 10.3788/YJYXS20193401.0098
[18]	LI Y Y, BRASCH N, WANG Y D, et al. Structure-SLAM: low-drift monocular SLAM in indoor environments[J]. IEEE Robotics and Automation Letters, 2020, 5(4): 6583-6590. doi: 10.1109/LRA.2020.3015456
[19]	CUI H N, SHEN SH H, GAO X, et al.. Batched incremental structure-from-motion[C]. 2017 International Conference on 3D Vision (3DV), IEEE, 2017: 205-214.
[20]	周单, 董秀成, 张帆, 等. 基于自适应重投影误差单目位姿优化算法[J]. 激光与光电子学进展,2019,56(2):021204. ZHOU D, DONG X CH, ZHANG F, et al. Monocular pose optimization algorithm based on adaptive reprojection error[J]. Laser &Optoelectronics Progress, 2019, 56(2): 021204. (in Chinese)
[21]	赵亚凤, 胡峻峰. 一种双正交消隐点的双目相机标定方法[J]. 液晶与显示,2016,31(10):958-966. doi: 10.3788/YJYXS20163110.0958 ZHAO Y F, HU J F. Binocular self calibration using two pairs of orthogonal vanishing points[J]. Chinese Journal of Liquid Crystals and Displays, 2016, 31(10): 958-966. (in Chinese) doi: 10.3788/YJYXS20163110.0958
[22]	HARTLEY R, ZISSERMAN A. Multiple View Geometry in Computer Vision[M]. 2nd ed. Cambridge: Cambridge University Press, 2003.