- ISSN 0258-2724
- CN 51-1277/U
- EI Compendex
- Scopus
- Indexed by Core Journals of China, Chinese S&T Journal Citation Reports
- Chinese S&T Journal Citation Reports
- Chinese Science Citation Database
| Citation: | YANG Yanchun, YAN Yan, WANG Ke. Infrared and Visible Image Fusion Based on Attention Mechanism and Illumination-Aware Network[J]. Journal of Southwest Jiaotong University, 2024, 59(5): 1204-1214. doi: 10.3969/j.issn.0258-2724.20230529
|
Some image fusion methods do not fully consider the illumination conditions in the image environment, resulting in insufficient brightness of infrared targets and overall low brightness of the image in the fused image, thereby affecting the clarity of texture details. To address these issues, an infrared and visible image fusion algorithm based on attention mechanism and illumination-aware network was proposed. Firstly, before training the fusion network, the illumination-aware network was used to calculate the probability that the current scene was daytime or nighttime and apply it to the loss function of the fusion network, so as to guide the training of the fusion network. Then, in the feature extraction part of the network, spatial attention mechanism and depthwise separable convolution were used to extract features from the source image. After obtaining spatial salient information, it was input into a convolutional neural network (CNN) to extract deep features. Finally, the deep feature information was concatenated for image reconstruction to obtain the final fused image. The experimental results show that the method proposed in this paper improves mutual information (MI), visual fidelity (VIF), average gradient (AG), fusion quality (Qabf), and spatial frequency (SF) by an average of 39.33%, 11.29%, 26.27%, 47.11%, and 39.01%, respectively. At the same time, it can effectively preserve the brightness of infrared targets in the fused images, including rich texture detail information.
| [1] |
LIU X W, WANG R H, HUO H T, et al. An attention-guided and wavelet-constrained generative adversarial network for infrared and visible image fusion[J]. Infrared Physics and Technology, 2023, 129: 104570.1-104570.16.
|
| [2] |
唐霖峰,张浩,徐涵,等. 基于深度学习的图像融合方法综述[J]. 中国图象图形学报,2023,28(1): 3-36. doi: 10.11834/jig.220422
TANG Linfeng, ZHANG Hao, XU Han, et al. Deep learning-based image fusion: a survey[J]. Journal of Image and Graphics, 2023, 28(1): 3-36. doi: 10.11834/jig.220422
|
| [3] |
程博阳,李婷,王喻林. 基于视觉显著性加权与梯度奇异值最大的红外与可见光图像融合[J]. 中国光学(中英文),2022,15(4): 675-688. doi: 10.37188/CO.2022-0124
CHENG Boyang, LI Ting, WANG Yulin. Fusion of infrared and visible light images based on visual saliency weighting and maximum gradient singular value[J]. Chinese Optics, 2022, 15(4): 675-688. doi: 10.37188/CO.2022-0124
|
| [4] |
杨莘,田立凡,梁佳明,等. 改进双路径生成对抗网络的红外与可见光图像融合[J]. 电子与信息学报,2023,45(8): 3012-3021. doi: 10.11999/JEIT220819
YANG Shen, TIAN Lifan, LIANG Jiaming, et al. Infrared and visible image fusion based on improved dual path generation adversarial network[J]. Journal of Electronics & Information Technology, 2023, 45(8): 3012-3021. doi: 10.11999/JEIT220819
|
| [5] |
沈英,黄春红,黄峰,等. 红外与可见光图像融合技术的研究进展[J]. 红外与激光工程,2021,50(9): 152-169.
SHEN Ying, HUANG Chunhong, HUANG Feng, et al. Research progress of infrared and visible image fusion technology[J]. Infrared and Laser Engineering, 2021, 50(9): 152-169.
|
| [6] |
胡建平,郝梦云,杜影,等. 结构和纹理感知的Retinex融合红外与可见光图像[J]. 光学 精密工程,2022,30(24): 3225-3238. doi: 10.37188/OPE.20223024.3225
HU Jianping, HAO Mengyun, DU Ying, et al. Fusion of infrared and visible images via structure and texture-aware retinex[J]. Optics and Precision Engineering, 2022, 30(24): 3225-3238. doi: 10.37188/OPE.20223024.3225
|
| [7] |
李霖,王红梅,李辰凯. 红外与可见光图像深度学习融合方法综述[J]. 红外与激光工程,2022,51(12): 337-356.
LI Lin, WANG Hongmei, LI Chenkai. A review of deep learning fusion methods for infrared and visible images[J]. Infrared and Laser Engineering, 2022, 51(12): 337-356.
|
| [8] |
王银,王立德,邱霁. 基于DenseNet结构的轨道暗光环境实时增强算法[J]. 西南交通大学学报,2022,57(6): 1349-1357. doi: 10.3969/j.issn.0258-2724.20210199
WANG Yin, WANG Lide, QIU Ji. Real-time enhancement algorithm based on DenseNet structure for railroad low-light environment[J]. Journal of Southwest Jiaotong University, 2022, 57(6): 1349-1357. doi: 10.3969/j.issn.0258-2724.20210199
|
| [9] |
朱雯青,汤心溢,张瑞,等. 基于边缘保持和注意力生成对抗网络的红外与可见光图像融合[J]. 红外与毫米波学报,2021,40(5): 696-708. doi: 10.11972/j.issn.1001-9014.2021.05.017
ZHU Wenqing, TANG Xinyi, ZHANG Rui, et al. Infrared and visible image fusion based on edge-preserving and attention generative adversarial network[J]. Journal of Infrared and Millimeter Waves, 2021, 40(5): 696-708. doi: 10.11972/j.issn.1001-9014.2021.05.017
|
| [10] |
TANG L F, XIANG X Y, ZHANG H, et al. DIVFusion: darkness-free infrared and visible image fusion[J]. Information Fusion, 2023, 91: 477-493. doi: 10.1016/j.inffus.2022.10.034
|
| [11] |
李泽琛,李恒超,胡文帅,等. 多尺度注意力学习的Faster R-CNN口罩人脸检测模型[J]. 西南交通大学学报,2021,56(5): 1002-1010. doi: 10.3969/j.issn.0258-2724.20210017
LI Zechen, LI Hengchao, HU Wenshuai, et al. Masked face detection model based on multi-scale attention-driven faster R-CNN[J]. Journal of Southwest Jiaotong University, 2021, 56(5): 1002-1010. doi: 10.3969/j.issn.0258-2724.20210017
|
| [12] |
王满利,王晓龙,张长森. 基于动态范围压缩增强和NSST的红外与可见光图像融合算法[J]. 光子学报,2022,51(9): 277-291.
WANG Manli, WANG Xiaolong, ZHANG Changsen. Infrared and visible image fusion algorithm based on dynamic range compression enhancement and NSST[J]. Acta Photonica Sinica, 2022, 51(9): 277-291.
|
| [13] |
朱文鹏,陈莉,张永新. 基于引导滤波和快速共现滤波的红外和可见光图像融合[J]. 计算机应用研究,2021,38(2): 600-604,610.
ZHU Wenpeng, CHEN Li, ZHANG Yongxin. Infrared and visible image fusion based on guided filter and fast co-occurrence filter[J]. Application Research of Computers, 2021, 38(2): 600-604,610.
|
| [14] |
MA J L, ZHOU Z Q, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics and Technology, 2017, 82: 8-17. doi: 10.1016/j.infrared.2017.02.005
|
| [15] |
MA J Y, YU W, LIANG P W, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. doi: 10.1016/j.inffus.2018.09.004
|
| [16] |
ZHANG H, MA J Y. SDNet: a versatile squeeze-and-decomposition network for real-time image fusion[J]. International Journal of Computer Vision, 2021, 129(10): 2761-2785. doi: 10.1007/s11263-021-01501-8
|
| [17] |
LI H, WU X J, KITTLER J. RFN-Nest: an end-to-end residual fusion network for infrared and visible images[J]. Information Fusion, 2021, 73(10): 72-86.
|
| [18] |
XU H, MA J Y, JIANG J J, et al. U2Fusion: a unified unsupervised image fusion network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(1): 502-518. doi: 10.1109/TPAMI.2020.3012548
|
| [19] |
LI H, WU X J. DenseFuse: a fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2019, 28(5): 2614-2623. doi: 10.1109/TIP.2018.2887342
|
| [20] |
JIAN L H, YANG X M, LIU Z, et al. SEDRFuse: a symmetric encoder–decoder with residual block network for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 5002215.1-5002215.15.
|
| [21] |
WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
|
| [22] |
CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 1800-1807.
|
| [23] |
TANG L F, YUAN J T, ZHANG H, et al. PIAFusion: a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83/84: 79-92. doi: 10.1016/j.inffus.2022.03.007
|
| [24] |
ZHANG H, XU H, XIAO Y, et al. Rethinking the image fusion: a fast unified image fusion network based on proportional maintenance of gradient and intensity[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12797-12804. doi: 10.1609/aaai.v34i07.6975
|
| [25] |
TANG L F, ZHANG H, XU H, et al. Rethinking the necessity of image fusion in high-level vision tasks: a practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity[J]. Information Fusion, 2023, 99: 101870.1-101870.16.
|
| [1] | ZHANG Hong, JIANG Xiaogang, ZHU Zhiwei, XIA Runchuan, ZHOU Jianting. Review on Intelligent Image Recognition of Apparent Diseases of Stay Cable[J]. Journal of Southwest Jiaotong University, 2025, 60(1): 10-26. doi: 10.3969/j.issn.0258-2724.20220647 |
| [2] | LI Linchao, ZHONG Liangjian, SU Qing, REN Lu, DU Bowen. Fine Urban Land Use Identification Based on Fusion of Multi-source Data[J]. Journal of Southwest Jiaotong University, 2025, 60(2): 326-335. doi: 10.3969/j.issn.0258-2724.20230296 |
| [3] | BAO Yanyan, YANG Guangze, CHEN Wei, FENG Tingna. Voiceprint Recognition of Discharge Aliasing Signals from 750 kV Transformer and Pin-Plate Based on Sparse Representation Theory and Convolutional Neural Network[J]. Journal of Southwest Jiaotong University, 2025, 60(3): 781-792. doi: 10.3969/j.issn.0258-2724.20230177 |
| [4] | WANG Huiqin, GUO Ruili, HE Yongqiang, LIU Bincan. NGO-Based CNN-BiLSTM-AM Model for Landslide Displacement Prediction[J]. Journal of Southwest Jiaotong University. doi: 10.3969/j.issn.0258-2724.20240550 |
| [5] | YANG Jun, GAO Zhiming, LI Jintai, ZHANG Chen. Correspondence Calculation of Three-Dimensional Point Cloud Model Based on Attention Mechanism[J]. Journal of Southwest Jiaotong University, 2024, 59(5): 1184-1193. doi: 10.3969/j.issn.0258-2724.20220682 |
| [6] | YUE Chuan, WANG Lide, YAN Haipeng. Attack-Sample Generation Method for Train Communication Network Under Few-Shot Condition[J]. Journal of Southwest Jiaotong University, 2023, 58(6): 1277-1285. doi: 10.3969/j.issn.0258-2724.20210557 |
| [7] | XIA Ying, LIU Min. Traffic Flow Prediction Based on Spatial-Temporal Attention Convolutional Neural Network[J]. Journal of Southwest Jiaotong University, 2023, 58(2): 340-347. doi: 10.3969/j.issn.0258-2724.20210526 |
| [8] | YANG Yueyi, WANG Lide, WANG Chong, WANG Huizhen, LI Ye. Fault Diagnosis Method Based on Deep Active Learning For MVB Network[J]. Journal of Southwest Jiaotong University, 2022, 57(6): 1342-1348, 1385. doi: 10.3969/j.issn.0258-2724.20210195 |
| [9] | ZHANG Chunxiang, TANG Libo, GAO Xueyao. Word Sense Disambiguation Based on Semi-Supervised Convolutional Neural Networks[J]. Journal of Southwest Jiaotong University, 2022, 57(1): 11-17, 27. doi: 10.3969/j.issn.0258-2724.20200105 |
| [10] | CUI Hongtao, CAO Ke, ZHANG Hu, CUI Xiao. Weather Classification in Traffic Scene Based on Joint Voting Network[J]. Journal of Southwest Jiaotong University, 2021, 56(3): 579-586. doi: 10.3969/j.issn.0258-2724.20200084 |
| [11] | GUO Liang, LI Changgen, GAO Hongli, DONG Xun, XIANG Shoubing. Residual Life Prediction of Mechanical Equipment Based on Feature Learning in Big Data Background[J]. Journal of Southwest Jiaotong University, 2021, 56(4): 730-735, 768. doi: 10.3969/j.issn.0258-2724.20190528 |
| [12] | GUO Lie, GE Pingshu, WANG Xiao, WANG Dongxing. Visual Simultaneous Localization and Mapping Algorithm Based on Convolutional Neural Network to Optimize Loop Detection[J]. Journal of Southwest Jiaotong University, 2021, 56(4): 706-712, 768. doi: 10.3969/j.issn.0258-2724.20190723 |
| [13] | LI Zechen, LI Hengchao, HU Wenshuai, YANG Jinyu, HUA Zexi. Masked Face Detection Model Based on Multi-scale Attention-Driven Faster R-CNN[J]. Journal of Southwest Jiaotong University, 2021, 56(5): 1002-1010. doi: 10.3969/j.issn.0258-2724.20210017 |
| [14] | TIAN Sheng, ZHANG Jianfeng, ZHANG Yutian, XU Kai. Lane Detection Algorithm Based on Dilated Convolution Pyramid Network[J]. Journal of Southwest Jiaotong University, 2020, 55(2): 386-392, 416. doi: 10.3969/j.issn.0258-2724.20181026 |
| [15] | YUAN Fei, ZHAO Xuyan, WANG Yige, ZHAO Zhisheng. Smoke Recognition Algorithm Based on Lightweight Convolutional Neural Network[J]. Journal of Southwest Jiaotong University, 2020, 55(5): 1111-1116, 1132. doi: 10.3969/j.issn.0258-2724.20190777 |
| [16] | XIANG Yu, CONG Deming, ZHANG Yang, YUAN Fei. Two-Stream Neural Network Fusion Model for Highway Fog Detection[J]. Journal of Southwest Jiaotong University, 2019, 54(1): 173-179. doi: 10.3969/j.issn.0258-2724.20180205 |
| [17] | HOU Jin, LÜ Zhiliang, XU Mao, WU Peijun, LIU Yuling, ZHANG Xiaoyu, CHENG Zeng. Combined Neural Networks Based on Deep Learning for Signal Detection in Aeronautical Communications[J]. Journal of Southwest Jiaotong University, 2019, 54(4): 863-869, 878. doi: 10.3969/j.issn.0258-2724.20180164 |
| [18] | YE Li-sheng, HE Feng-dao. The Learning of BP Neural Network Based on Evolutionary Programming[J]. Journal of Southwest Jiaotong University, 2001, 14(5): 545-548. |
| [19] | He Zhengyou, Qian Qingquan. An Improved Wavelet Neural Network Structure and Its Learning Algorithm[J]. Journal of Southwest Jiaotong University, 1999, 12(5): 436-440. |
| [20] | HE Zheng-You, Jian-Qing-Quan. An Improved Wavelet Neural Network Structure and Its Learning Algorithm[J]. Journal of Southwest Jiaotong University, 1999, 12(5): 436-440. |
| 1. | 王丹阳，侯米娜. 结合注意力机制与改进UNet的低照度图像增强算法研究. 智能物联技术. 2025(02): 102-105 .  |
Figures(5) / Tables(6)
YANG Yanchun, YAN Yan, WANG Ke. Infrared and Visible Image Fusion Based on Attention Mechanism and Illumination-Aware Network[J]. Journal of Southwest Jiaotong University, 2024, 59(5): 1204-1214. doi: 10.3969/j.issn.0258-2724.20230529
DownLoad:
