跳到主要导航 跳到搜索 跳到主要内容

Focus on the Hard: SAM-Based Prompt-Guided Refinement for RGB-T Semantic Segmentation

  • Northwestern Polytechnical University Xian
  • Peking University

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

RGB-Thermal semantic segmentation is essential for robust visual understanding in autonomous driving, surveillance, and search-and-rescue applications. While RGB and thermal modalities provide complementary information, cross-modal difficult regions with ambiguous boundaries and conflicting signals represent the primary performance bottleneck. Existing methods treat all regions uniformly and lack principled approaches for targeting these critical challenging areas. To address this limitation, we propose FocusSeg, a prompt-guided refinement framework embodying “Focus on the Hard” to systematically identify and refine difficult cross-modal regions. Our approach employs a two-stage framework: first establishing multi-modal understanding, then introducing intelligent point-based correction. Specifically, the framework integrates a Difficult Point Sampling Strategy (DPSS) for dynamic challenging region identification, a Multimodal Correction Module (MMCM) for training specialized correction heads, and a Point Correction Strategy (PCS) combining correction-head predictions with historical priors to generate reliable corrective prompts. These refined prompts guide a class-aware SAM framework toward superior targeted refinement. Extensive experiments on multiple RGB-T datasets demonstrate substantial improvements over state-of-the-art methods, with particularly notable gains in challenging scenarios involving cross-modal conflicts and boundary ambiguities. Our work establishes a new paradigm for intelligent resource allocation in multimodal segmentation tasks. The source code will be released at https://github.com/Wanda36/FocusSeg.

源语言英语
页(从-至)10559-10572
页数14
期刊IEEE Transactions on Image Processing
卷35
DOI
出版状态已出版 - 2026

学术指纹

探究 'Focus on the Hard: SAM-Based Prompt-Guided Refinement for RGB-T Semantic Segmentation' 的科研主题。它们共同构成独一无二的学术指纹。

引用此