Interactive visible and infrared image fusion and segmentation

The primary objective of visible and infrared image fusion (VIF) methods is to integrate the complementary information from visible and infrared images to generate high-quality fused outputs that satisfy both human visual perception and machine vision performance. However, most existing VIF approaches still face several critical limitations. They rely on predefined and static fusion strategies that primarily emphasize the preservation of salient content, but fail to adapt to diverse imaging conditions or varying task requirements. Consequently, they lack the flexibility to generalize across different scenarios, provide no mechanisms for user intervention to guide the fusion process, and offer only limited fine-grained interaction for semantic-level integration with high-level tasks. To overcome these limitations, we propose an Interactive Visible and Infrared Fusion and Segmentation (IVIFS) framework. It introduces user-driven interaction to enable dynamic and fine-grained control over both fusion and segmentation outputs. The IVIFS framework offers two main contributions. First, it significantly enhances the adaptability of VIF methods. This allows it to effectively respond to diverse fusion scenarios as well as individual user preferences during the testing phase. Second, the proposed framework jointly optimizes for human visual quality and machine vision performance. In this way, it mitigates the traditional trade-off between these two objectives. As a result, it improves performance on downstream tasks (e.g., segmentation) across a wide range of scenarios. Specifically, IVIFS is composed of two core modules: the Interactive Reinforcement Module (IRM) and the Controllable Visible and Infrared Fusion and Segmentation Module (CVIFSM). The IRM translates user requirements into intermediate control variables. These variables are then leveraged by the CVIFSM to generate fusion and segmentation results tailored to user preferences. Extensive experiments conducted on benchmark datasets demonstrate that IVIFS enables effective user interaction. Moreover, it achieves superior fusion and segmentation performance. The source codes are available at https://github.com/JiangtaoNie/IVIFS.git