Heterogeneous Image Fusion for Target Recognition Based on Evidence Reasoning

Multi-source fusion is an efficient strategy in complex image target recognition since it can exploit the complementary knowledge in different sources to improve the classification performance. In this paper, we propose a new end-to-end framework for heterogeneous (i.e. visible & infrared) image fusion target recognition (HIFTR). Firstly, two networks are built for the visible and infrared images respectively and jointly trained based on mutual learning. It aims to transfer heterogeneous information mutually and improve the generalization performance of the networks. Secondly, a weighted decision-level fusion method based on evidence reasoning is developed to combine the classification results of visible and infrared images for the final target recognition. In the training process, the weight of each image is automatically optimized in the networks. Finally, the performance of the proposed HIFTR has been evaluated by comparing with other related methods, and the experimental results show that the HIFTR method can efficiently improve the classification accuracy.