TY - JOUR
T1 - Integrating SAM With Feature Interaction for Remote Sensing Change Detection
AU - Zhang, Da
AU - Wang, Feiyu
AU - Ning, Lichen
AU - Zhao, Zhiyuan
AU - Gao, Junyu
AU - Li, Xuelong
N1 - Publisher Copyright:
© 1980-2012 IEEE.
PY - 2024
Y1 - 2024
N2 - Vision foundation models (VFMs) have rapidly gained application across various visual scenarios due to their robust universality and generalization capabilities. However, when directly applied to remote sensing images (RSIs), their performance often falls short owing to the unique inherent imaging characteristics. Moreover, these models typically suffer from inadequate feature extraction capabilities and unclear boundary detection because of the lack of specialized knowledge in the remote sensing (RS) field. To ameliorate these issues, we propose SFCD-Net, a novel network integrating SAM with feature interaction for RS change detection. To be specific, we first introduce a parameter-efficient fine-tuning (PEFT) method that allows the model to learn domain-specific knowledge, thereby enhancing its fine-grained feature extraction capability. Second, an innovative bitemporal feature interaction (BFI) module is designed to improve the model's sensitivity to changes. Finally, we use the boundary loss function (BLF) to enhance the model's ability to process boundary details, thereby improving its performance in recognizing boundaries and small targets. Through a series of ablation studies and comparative experiments, we demonstrate that the proposed SFCD-Net significantly improves model adaptability in RS tasks under limited computational resources, outperforming existing models.
AB - Vision foundation models (VFMs) have rapidly gained application across various visual scenarios due to their robust universality and generalization capabilities. However, when directly applied to remote sensing images (RSIs), their performance often falls short owing to the unique inherent imaging characteristics. Moreover, these models typically suffer from inadequate feature extraction capabilities and unclear boundary detection because of the lack of specialized knowledge in the remote sensing (RS) field. To ameliorate these issues, we propose SFCD-Net, a novel network integrating SAM with feature interaction for RS change detection. To be specific, we first introduce a parameter-efficient fine-tuning (PEFT) method that allows the model to learn domain-specific knowledge, thereby enhancing its fine-grained feature extraction capability. Second, an innovative bitemporal feature interaction (BFI) module is designed to improve the model's sensitivity to changes. Finally, we use the boundary loss function (BLF) to enhance the model's ability to process boundary details, thereby improving its performance in recognizing boundaries and small targets. Through a series of ablation studies and comparative experiments, we demonstrate that the proposed SFCD-Net significantly improves model adaptability in RS tasks under limited computational resources, outperforming existing models.
KW - Change detection (CD)
KW - remote sensing (RS)
KW - vision foundation model (VFM)
UR - http://www.scopus.com/inward/record.url?scp=85208254351&partnerID=8YFLogxK
U2 - 10.1109/TGRS.2024.3483775
DO - 10.1109/TGRS.2024.3483775
M3 - 文章
AN - SCOPUS:85208254351
SN - 0196-2892
VL - 62
JO - IEEE Transactions on Geoscience and Remote Sensing
JF - IEEE Transactions on Geoscience and Remote Sensing
M1 - 4513011
ER -