TY - GEN
T1 - Degradation-Robust and Temporally Consistent Infrared–Visible Video Fusion via One-Step Diffusion Framework
AU - Du, Songcheng
AU - Xu, Haoyuan
AU - Li, Xingyuan
AU - Zou, Yang
AU - Liu, Jinyuan
AU - Bai, Yunpeng
AU - Li, Ying
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.
PY - 2026
Y1 - 2026
N2 - Infrared–visible video fusion aims to integrate complementary thermal and visual information to produce stable and informative fused videos for real-world applications. While recent advances have achieved remarkable progress in image-level fusion, extending these methods to dynamic video scenarios remains challenging. Existing video fusion approaches typically adopt a ‘temporal-first’ processing strategy, where temporal modeling is performed within each modality before cross-modal fusion. However, such designs may become vulnerable to modality-specific degradations, including low illumination in visible videos and noise or flicker in infrared sequences, which often lead to unreliable motion estimation and temporally inconsistent fusion results. To address this limitation, we advocate a ‘fusion-first’ strategy and propose DRT-VF, a unified degradation-robust and temporally consistent infrared–visible video fusion method built upon a one-step diffusion architecture with a two-stage progressive training strategy. In Stage I, a Modality-Collaborative Attention module integrates complementary infrared and visible cues to establish robust intra-frame fusion representations. In Stage II, temporal consistency is modeled through a hierarchical design consisting of a Global Temporal Consistency module and a Local Motion Compensation module, which jointly enhance global temporal consistency and refine motion-sensitive regions. Extensive experiments demonstrate that DRT-VF improves temporal stability while preserving fine structural details.
AB - Infrared–visible video fusion aims to integrate complementary thermal and visual information to produce stable and informative fused videos for real-world applications. While recent advances have achieved remarkable progress in image-level fusion, extending these methods to dynamic video scenarios remains challenging. Existing video fusion approaches typically adopt a ‘temporal-first’ processing strategy, where temporal modeling is performed within each modality before cross-modal fusion. However, such designs may become vulnerable to modality-specific degradations, including low illumination in visible videos and noise or flicker in infrared sequences, which often lead to unreliable motion estimation and temporally inconsistent fusion results. To address this limitation, we advocate a ‘fusion-first’ strategy and propose DRT-VF, a unified degradation-robust and temporally consistent infrared–visible video fusion method built upon a one-step diffusion architecture with a two-stage progressive training strategy. In Stage I, a Modality-Collaborative Attention module integrates complementary infrared and visible cues to establish robust intra-frame fusion representations. In Stage II, temporal consistency is modeled through a hierarchical design consisting of a Global Temporal Consistency module and a Local Motion Compensation module, which jointly enhance global temporal consistency and refine motion-sensitive regions. Extensive experiments demonstrate that DRT-VF improves temporal stability while preserving fine structural details.
KW - Infrared–Visible Video Fusion
KW - Low-Level Vision
KW - Multimodal Image Fusion
UR - https://www.scopus.com/pages/publications/105052640772
U2 - 10.1007/978-3-032-37383-0_17
DO - 10.1007/978-3-032-37383-0_17
M3 - 会议稿件
AN - SCOPUS:105052640772
SN - 9783032373823
T3 - Lecture Notes in Computer Science
SP - 279
EP - 298
BT - Computer Vision – ECCV 2026 - 19th European Conference, 2026, Proceedings
A2 - Favaro, Paolo
A2 - Kukelova, Zuzana
A2 - Maki, Atsuto
A2 - Rohrbach, Anna
A2 - Schindler, Konrad
A2 - Tombari, Federico
PB - Springer Science and Business Media Deutschland GmbH
T2 - 19th European Conference on Computer Vision, ECCV 2026
Y2 - 8 September 2026 through 12 September 2026
ER -