| Citation: | CHENG Bo-yang, LI Ting, WANG Yu-lin. Fusion of infrared and visible light images based on visual saliency weighting and maximum gradient singular value[J]. Chinese Optics, 2022, 15(4): 675-688. doi: 10.37188/CO.2022-0124
|
In order to effectively integrate the spectral saliency information of infrared and visible light images and improve the visual contrast of the fused images, a fusion method of infrared and visible light images based on weighted visual saliency and maximum gradient singular value is proposed in this paper. Firstly, the new algorithm uses the rolling guidance shearlet transform as a multi-scale analysis tool to obtain the approximate layer components and multi-directional detail layer components of the image. Secondly, for the approximate layer components that reflect the energy characteristics of the image subject, visual saliency weighted fusion is used as its fusion rule. This method uses the saliency weighted coefficient matrix to guide the effective fusion of spectral saliency information in the image, and improves the visual observation of the fused image. In addition, the principle of maximum gradient singular value is used to guide the fusion of detail layer components. This method can restore the gradient features hidden in the two source images to the fused image to a great extent, so that the fused image has clearer edge details. In order to verify the effectiveness of this algorithm, we have adopted five groups of independent fusion experiments. The final experimental results show that this algorithm has higher contrast and richer edge details. Compared with the existing typical methods, the objective parameters such as AVG, IE, QE, SF, SD and SCD are improved by 16.4%, 3.9%, 11.8%, 17.1%, 21.4% and 10.1%, respectively, so it has better visual effect.
| [1] |
陈清江, 张彦博, 柴昱洲, 等. 有限离散剪切波域的红外可见光图像融合[J]. 中国光学,2016,9(5):523-531. doi: 10.3788/co.20160905.0523
CHEN Q J, ZHANG Y B, CHAI Y ZH, et al. Fusion of infrared and visible images based on finite discrete shearlet domain[J]. Chinese Optics, 2016, 9(5): 523-531. (in Chinese) doi: 10.3788/co.20160905.0523
|
| [2] |
王成, 张艳超. 像素级自适应融合的夜间图像增强[J]. 液晶与显示,2019,34(9):888-896. doi: 10.3788/YJYXS20193409.0888
WANG CH, ZHANG Y CH. Night image enhancement based on pixel level adaptive image fusion[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(9): 888-896. (in Chinese) doi: 10.3788/YJYXS20193409.0888
|
| [3] |
陈广秋, 高印寒, 才华, 等. 局部化NSST与PCNN相结合的图像融合[J]. 液晶与显示,2015,30(4):701-712. doi: 10.3788/YJYXS20153004.0701
CHEN G Q, GAO Y H, CAI H, et al. Image fusion algorithm based on local NSST and PCNN[J]. Chinese Journal of Liquid Crystals and Display, 2015, 30(4): 701-712. (in Chinese) doi: 10.3788/YJYXS20153004.0701
|
| [4] |
陈广秋, 陈昱存, 李佳悦, 等. 基于DNST和卷积稀疏表示的红外与可见光图像融合[J]. 吉林大学学报(工学版),2021,51(3):996-1010.
CHEN G Q, CHEN Y C, LI J Y, et al. Infrared and visible image fusion based on discrete nonseparable shearlet transform and convolutional sparse representation[J]. Journal of Jilin University (Engineering and Technology Edition)
|
| [5] |
PRAKASH O, PARK C M, KHARE A, et al. Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform[J]. Optik, 2019, 182: 995-1014. doi: 10.1016/j.ijleo.2018.12.028
|
| [6] |
TAO T W, LIU M X, HOU Y K, et al. Latent low-rank representation with sparse consistency constraint for infrared and visible image fusion[J]. Optik, 2022, 261: 169102. doi: 10.1016/j.ijleo.2022.169102
|
| [7] |
LIU Y Y, HE K J, XU D, et al. Infrared and visible image fusion based on visibility enhancement and hybrid multiscale decomposition[J]. Optik, 2022, 258: 168914. doi: 10.1016/j.ijleo.2022.168914
|
| [8] |
ANOOP SURAJ A, FRANCIS M, KAVYA T S, et al. Discrete wavelet transform based image fusion and de-noising in FPGA[J]. Journal of Electrical Systems and Information Technology, 2014, 1(1): 72-81. doi: 10.1016/j.jesit.2014.03.006
|
| [9] |
DONOHO D L, DUNCAN M R. Digital curvelet transform: strategy, implementation, and experiments[J]. Proceedings of SPIE, 2000, 4056: 12-30. doi: 10.1117/12.381679
|
| [10] |
CUNHA A L D, ZHOU J, DO M N. The nonsubsampled contourlet transform: theory, design, and applications[J]. IEEE Transactions on Image Processing, 2006, 15(10): 3089-3101. doi: 10.1109/TIP.2006.877507
|
| [11] |
GUO K H, LABATE D. Optimally sparse multidimensional representation using shearlets[J]. SIAM Journal on Mathematical Analysis, 2007, 39(1): 298-318. doi: 10.1137/060649781
|
| [12] |
KONG W W, MIAO Q G, LEI Y, et al. Guided filter random walk and improved spiking cortical model based image fusion method in NSST domain[J]. Neurocomputing, 2022, 488: 509-527. doi: 10.1016/j.neucom.2021.11.060
|
| [13] |
陈广秋, 梁小伟, 段锦, 等. 多级方向引导滤波器及其在多传感器图像融合中的应用[J]. 吉林大学学报(理学版),2019,57(1):129-138. doi: 10.13413/j.cnki.jdxblxb.2017447
CHEN G Q, LIANG X W, DUAN J, et al. Multistage directional guided filter and its application in multi-sensor image fusion[J]. Journal of Jilin University (Science Edition)
|
| [14] |
ZHANG Q, SHEN X Y, XU L, et al.. Rolling guidance filter[C]. Proceedings of the 13th European Conference on Computer Vision, Springer, 2014: 815-830.
|
| [15] |
程博阳. 基于滚动引导剪切波变换的红外与可见光图像融合研究[D]. 长春: 中国科学院大学(中国科学院长春光学精密机械与物理研究所), 2020.
CHENG B Y. Research on fusion of infrared and visible light image based on rolling guidance shearlet transform[D]. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, 2020.
|
| [16] |
GUO ZH Y, YU X T, DU Q L. Infrared and visible image fusion based on saliency and fast guided filtering[J]. Infrared Physics &Technology, 2022, 123: 104178.
|
| [17] |
LI W SH, LI R Y, FU J, et al. MSENet: a multi-scale enhanced network based on unique features guidance for medical image fusion[J]. Biomedical Signal Processing and Control, 2022, 74: 103534. doi: 10.1016/j.bspc.2022.103534
|
| [18] |
CHAO ZH, DUAN X G, JIA SH F, et al. Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network[J]. Applied Soft Computing, 2022, 118: 108542. doi: 10.1016/j.asoc.2022.108542
|
| [19] |
GUO Y N, LI X, GAO A, et al.. A scale-aware pansharpening method with rolling guidance filter[C]. Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 2017: 5121-5124.
|
| [20] |
刘博, 韩广良, 罗惠元. 基于多尺度细节的孪生卷积神经网络图像融合算法[J]. 液晶与显示,2021,36(9):1283-1293. doi: 10.37188/CJLCD.2020-0339
LIU B, HAN G L, LUO H Y. Image fusion algorithm based on multi-scale detail Siamese convolutional neural network[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(9): 1283-1293. (in Chinese) doi: 10.37188/CJLCD.2020-0339
|
| [21] |
XIANG I B, YU Z, FU G Z. Quadtree-based multi-focus image fusion using a weighted focus-measure[J]. Inform. Fusion, 2015, 22: 105-118. doi: 10.1016/j.bspc.2021.102852
|
| [22] |
JOSE J, GAUTAM N, TIWARI M, et al. An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion[J]. Biomedical Signal Processing and Control, 2021, 66: 102480. doi: 10.1016/j.bspc.2021.102480
|
| [23] |
ACHANTA R, HEMAMI S, ESTRADA F, et al. . Frequency-tuned salient region detection[C]. Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009: 1597-1604.
|
| [24] |
CHENG B Y, JIN L X, LI G N. Adaptive fusion framework of infrared and visual image using saliency detection and improved dual-channel PCNN in the LNSST domain[J]. Infrared Physics &Technology, 2018, 79: 30-43.
|
| [25] |
CHENG B Y, JIN L X, LI G N. Infrared and visual image fusion using LNSST and an adaptive dual-channel PCNN with triple-linking strength[J]. Neurocomputing, 2018, 310: 135-147. doi: 10.1016/j.neucom.2018.05.028
|
| [26] |
陈广秋, 高印寒, 段锦, 等. 基于奇异值分解的PCNN红外与可见光图像融合[J]. 液晶与显示,2015,30(1):126-136. doi: 10.3788/YJYXS20153001.0126
CHEN G Q, GAO Y H, DUAN J, et al. Fusion algorithm of infrared and visible images based on singular value decomposition and PCNN[J]. Chinese Journal of Liquid Crystals and Displays, 2015, 30(1): 126-136. (in Chinese) doi: 10.3788/YJYXS20153001.0126
|
| [27] |
NENCINI F, GARZELLI A, BARONTI S, et al. Remote sensing image fusion using the curvelet transform[J]. Information Fusion, 2007, 8(2): 143-156. doi: 10.1016/j.inffus.2006.02.001
|
| [28] |
LIU Y, LIU SH P, WANG Z F. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164. doi: 10.1016/j.inffus.2014.09.004
|
| [29] |
BAVIRISETTI D P, DHULI R. Fusion of infrared and visible sensor images based on anisotropic diffusion and karhunen-loeve transform[J]. IEEE Sensors Journal, 2016, 16(1): 203-209. doi: 10.1109/JSEN.2015.2478655
|
| [30] |
JIN L M, ZHI Q Z, BO W. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics &Technology, 2017, 82: 8-17.
|
| [31] |
NAIDU V P S. Image fusion technique using multi-resolution singular value decomposition[J]. Defence Science Journal, 2011, 61(5): 479-484. doi: 10.14429/dsj.61.705
|
| [32] |
BAVIRISETTI D P, DHULI R. Two-scale image fusion of visible and infrared images using saliency detection[J]. Infrared Physics &Technology, 2016, 76: 52-64.
|
| [33] |
D. P. B, R. D Two-scale image fusion of visible and infrared images using saliency detection[J]. Infrared Physics &Technology, 2016, 76: 52-64. doi: 10.1016/j.cmpb.2019.04.010
|
| [34] |
LIN Y C, CAO D X, ZHOU X C. Adaptive infrared and visible image fusion method by using rolling guidance filter and saliency detection[J]. Optik, 2022, 262: 169218.
|
| [35] |
ZHE L, YU Q S, VICTOR S. MRI and PET image fusion using the nonparametric density model and the theory of variable-weight[J]. Computer Methods and Programs in Biomedicine, 2019, 175: 73-82.
|
| [36] |
BAI X ZH, ZHANG Y, ZHOU F G, et al. Quadtree-based multi-focus image fusion using a weighted focus-measure[J]. Information Fusion, 2015, 22: 105-118. doi: 10.1016/j.inffus.2014.05.003
|
| [37] |
FARID M S, MAHMOOD A, AL-MAADEED S A. Multi-focus image fusion using Content Adaptive Blurring[J]. Information Fusion, 2019, 45: 96-112. doi: 10.1016/j.inffus.2018.01.009
|
| [38] |
YIN M, DUAN P H, LIU W, et al. A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation[J]. Neurocomputing, 2017, 226: 182-191. doi: 10.1016/j.neucom.2016.11.051
|
| [39] |
KONG X Y, LIU L, QIAN Y SH, et al. . Infrared and visible image fusion using structure-transferring fusion method[J]. Infrared Physics & Technology, 2019, 98: 161-173. ASLANTAS V, BENDES E. A new image quality metric for image fusion: the sum of the correlations of differences[J]. AEU - International Journal of Electronics and Communications, 2015, 69(12): 1890-1896.
|
| [40] |
ASLANTAS V, BENDES E. A new image quality metric for image fusion: the sum of the correlations of differences[J]. AEU - International Journal of Electronics and Communications, 2015, 69(12): 1890-1896.
|
Figures(14) / Tables(5)
DownLoad:
