基于交互融合的高精度图像融合算法的研究

Research on high-precision image fusion algorithm based on interactive fusion

摘要:

针对可见光和SAR图像融合，提出了一种基于跨模态差分感知和注意力机制的交互融合（TDPAM Fusion）算法，能有效保留可见光图像中的纹理结构和SAR图像的细节信息。首先，采用跨模态差分感知融合（Cross-Modal Differential Perception Fusion，CMDAF）模块提取图像的互补信息，避免真值缺失并提高融合精度。其次，通过坐标注意力机制（Coordinate Attention，CA）提高特征提取的准确性和效率，增强语义信息的集成。最后通过交互融合算法（Interactive Fusion Module，IFM）将特征自适应融合。设计了相应的大型基准数据集，用于网络模型的训练和测试。实验结果表明：TDPAM Fusion融合算法可以获得包含清晰SAR信息的高质量可见光图像。此外，融合算法将互信息（Mutual Information，MI）、空间频率（Spatial Frequency，SF）、视觉保真度（Visual Fidelity，VIF）和相关系数（Correlation Coefficient，CC）等关键指标，分别提高了约6.41％、10.36％、14.25％和4.74％。

Abstract:

Aiming at the fusion of visible and SAR images, a novel interactive fusion algorithm based on Transmembrane Differential Perception and Attention Mechanism (TDPAM Fusion) is proposed, which can effectively preserve the texture structure of visible images and detail information of SAR images. Firstly, the Cross-Modal Differential Perception Fusion module is utilized to extract complementary information from images, which can avoid the missing of true values and improve the accuracy of fusion. Secondly, the coordinate attention mechanism is employed to enhance the accuracy and efficiency of feature extraction, and improve the integration of semantic information. Finally, a feature interaction fusion algorithm is used to adaptively fuse features from SAR and visible images. A corresponding large benchmark dataset is designed for model training and testing. Experimental results demonstrate that the fusion algorithm can obtain high-quality visible images with clear SAR information. In addition, the algorithm can improve key indicators such as mutual information, spatial frequency, visual fidelity, and correlation coefficient by approximately 6.41%, 10.36%, 14.25%, and 4.74%, respectively.