TY - JOUR
T1 - Cross-Modality Domain Adaptation Based on Semantic Graph Learning
T2 - From Optical to SAR Images
AU - Zhang, Xiufei
AU - Huang, Zhongling
AU - Yao, Xiwen
AU - Feng, Xiaoxu
AU - Cheng, Gong
AU - Han, Junwei
N1 - Publisher Copyright:
© 1980-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - Synthetic aperture radar (SAR) imaging provides a distinct advantage in scene understanding due to its capability for all-weather data acquisition. However, in comparison to easily annotated optical remote sensing images, the lower imaging quality of SAR images presents significant challenges in obtaining manually annotated training data, which poses substantial issues for SAR image analysis. In this paper, we employ the domain adaptation (DA) that leverages labeled optical images to better understand unlabeled SAR images. Global feature alignment as a method for DA has demonstrated effectiveness in transferring knowledge, yet it faces challenges in cross-modality adaptation from optical remote sensing to SAR images due to their differing imaging mechanisms. With distinct visual features between optical and SAR images, the semantic dependency is difficult to construct, which results in low-quality pseudo-label assignment for SAR images. To address the above issue, we propose a semantic graph learning framework to comprehensively align the global features of optical remote sensing and SAR images by modeling the cross-modality semantics and generating high-quality pseudo-labels. It can be applied for SAR scene classification and object detection when only optical remote sensing images are labeled. Specifically, a cross-modality semantic graph alignment (CSGA) module is constructed to model and align the second-order semantic dependencies by aggregating cross-modality visual semantic information. Then, an uncertainty-based robust pseudo-label generation (URPG) module is designed to generate pseudo-labels for effective semantic alignment and self-training by modeling the uncertainty of pseudo-labels for each SAR image. Comprehensive experiments show that our proposed method outperforms the state-of-the-art methods on scene classification (NWPU-RESISC45→WHU-SAR6, MLRSNet→NWPU-SAR6, MLRSNet→NWPU-SAR6, and NWPU-RESISC45→NWPU-SAR6) and object detection (MASATI-ship→SSDD, MVSRD→SARDet-vehicle, and DIOR-airplane→SAR-airplane) tasks.
AB - Synthetic aperture radar (SAR) imaging provides a distinct advantage in scene understanding due to its capability for all-weather data acquisition. However, in comparison to easily annotated optical remote sensing images, the lower imaging quality of SAR images presents significant challenges in obtaining manually annotated training data, which poses substantial issues for SAR image analysis. In this paper, we employ the domain adaptation (DA) that leverages labeled optical images to better understand unlabeled SAR images. Global feature alignment as a method for DA has demonstrated effectiveness in transferring knowledge, yet it faces challenges in cross-modality adaptation from optical remote sensing to SAR images due to their differing imaging mechanisms. With distinct visual features between optical and SAR images, the semantic dependency is difficult to construct, which results in low-quality pseudo-label assignment for SAR images. To address the above issue, we propose a semantic graph learning framework to comprehensively align the global features of optical remote sensing and SAR images by modeling the cross-modality semantics and generating high-quality pseudo-labels. It can be applied for SAR scene classification and object detection when only optical remote sensing images are labeled. Specifically, a cross-modality semantic graph alignment (CSGA) module is constructed to model and align the second-order semantic dependencies by aggregating cross-modality visual semantic information. Then, an uncertainty-based robust pseudo-label generation (URPG) module is designed to generate pseudo-labels for effective semantic alignment and self-training by modeling the uncertainty of pseudo-labels for each SAR image. Comprehensive experiments show that our proposed method outperforms the state-of-the-art methods on scene classification (NWPU-RESISC45→WHU-SAR6, MLRSNet→NWPU-SAR6, MLRSNet→NWPU-SAR6, and NWPU-RESISC45→NWPU-SAR6) and object detection (MASATI-ship→SSDD, MVSRD→SARDet-vehicle, and DIOR-airplane→SAR-airplane) tasks.
KW - Cross-modality semantic graph alignment
KW - Domain adaptation
KW - SAR images
KW - Semantic graph learning
KW - Uncertainty-based robust pseudo-label generation
UR - http://www.scopus.com/inward/record.url?scp=105002576863&partnerID=8YFLogxK
U2 - 10.1109/TGRS.2025.3559915
DO - 10.1109/TGRS.2025.3559915
M3 - 文章
AN - SCOPUS:105002576863
SN - 0196-2892
JO - IEEE Transactions on Geoscience and Remote Sensing
JF - IEEE Transactions on Geoscience and Remote Sensing
ER -