TY - JOUR
T1 - Navigating Uncertainty
T2 - Semantic-Powered Image Enhancement and Fusion
AU - Yao, Jiaxin
AU - Zhao, Yongqiang
AU - Kong, Seong G.
AU - Zhang, Xun
N1 - Publisher Copyright:
© 1994-2012 IEEE.
PY - 2024
Y1 - 2024
N2 - The fusion of infrared and visible imagery plays a crucial role in environmental monitoring. Existing approaches aim to achieve high-quality perceptual results for human observers and robust outcomes for machine-based high-level tasks by adopting a joint design of fusion and segmentation. However, design constraints imposed by predetermined fusion rules limit the precision of high-level tasks, and drawbacks in the image domains to be fused are often overlooked. This letter presents a novel semantic-powered infrared and visible image fusion framework to address these issues. The key feature of our approach is the utilization of trainable gains and weights in enhancement and fusion processes, influenced solely by segmentation and serving as uncertainty parameters. We propose a two-stage training strategy: initially, training a combined enhancement and fusion network with random uncertainty parameters, followed by the estimation of semantic-driven uncertainty parameters. The enhancement and fusion process is optimized within the Laplacian pyramid framework to ensure efficient computation. Experimental results highlight the significance of modeling the fusion process with uncertainty for achieving satisfactory fusion and segmentation outcomes.
AB - The fusion of infrared and visible imagery plays a crucial role in environmental monitoring. Existing approaches aim to achieve high-quality perceptual results for human observers and robust outcomes for machine-based high-level tasks by adopting a joint design of fusion and segmentation. However, design constraints imposed by predetermined fusion rules limit the precision of high-level tasks, and drawbacks in the image domains to be fused are often overlooked. This letter presents a novel semantic-powered infrared and visible image fusion framework to address these issues. The key feature of our approach is the utilization of trainable gains and weights in enhancement and fusion processes, influenced solely by segmentation and serving as uncertainty parameters. We propose a two-stage training strategy: initially, training a combined enhancement and fusion network with random uncertainty parameters, followed by the estimation of semantic-driven uncertainty parameters. The enhancement and fusion process is optimized within the Laplacian pyramid framework to ensure efficient computation. Experimental results highlight the significance of modeling the fusion process with uncertainty for achieving satisfactory fusion and segmentation outcomes.
KW - Infrared and visible images fusion
KW - Laplacian pyramid
KW - task-oriented fusion
UR - http://www.scopus.com/inward/record.url?scp=85190725419&partnerID=8YFLogxK
U2 - 10.1109/LSP.2024.3389463
DO - 10.1109/LSP.2024.3389463
M3 - 文章
AN - SCOPUS:85190725419
SN - 1070-9908
VL - 31
SP - 1164
EP - 1168
JO - IEEE Signal Processing Letters
JF - IEEE Signal Processing Letters
ER -