Target recognition with fusion of visible and infrared images based on mutual learning

Multi-source fusion is an important research in image target recognition. Different image sources usually can provide complementary knowledge for improving the classification performance. Current methods generally extract features or recognize each source separately before performing fusion, and this cannot well exploit the correlation of different sources. We propose a multi-source image (i.e., visible and infrared images) fusion target recognition method based on mutual learning (MIF-ML). In this paper, an end-to-end visible-infrared image fusion model is constructed. Firstly, two networks are built for the visible and infrared images, respectively, and jointly trained based on mutual learning. The generalization performance of the networks can be efficiently enhanced because the information of different images is transferred between the two networks. Secondly, a weighted decision-level fusion method is developed to combine the classification results of visible and infrared images for achieving as good as possible recognition performance. In the training process, the weight of each image is automatically optimized in the networks. Finally, the performance of the MIF-ML method has been tested by comparing with other related methods, and the experimental results show that the proposed MIF-ML can efficiently improve the classification accuracy.