IVJDN: AN END-TO-END NETWORK FOR JOINT INFRARED AND VISIBLE IMAGE FUSION AND DETECTION

Fusing infrared and visible images has been an active research topic within the remote sensing community since these two kinds of images can provide complementary information. Though different methods have been proposed, most of the existing infrared and visible image fusion methods only focus on obtaining visually pleasing results without considering if the fusion results fit well for the subsequential object detection task. To obtain better detection results from infrared and visible image fusion, we propose an end-to-end network that incorporates an image fusion module and object detection module into a unified framework. Within the network architecture constructed, attention mechanisms as well as intensity loss and gradient loss are utilized to effectively preserve the distinguishing characteristics of both infrared and visible modalities for object detection, yielding advantageous attributes for detection purposes. By jointly training the image fusion module and the object detection module, our proposed method achieves improved object detection performance. Experimental results corroborate the effectiveness of the proposed approach.