MSCANet: Multi-scale Cross-Attention Fusion Network for Multimodal Remote Sensing Image Semantic Segmentation

Semantic segmentation of remote sensing images is critical for land cover classification and disaster monitoring, but single-modal sources like optical or SAR imagery suffer from inherent limitations. To address this, we propose MSCANet, a multi-scale cross-attention fusion network for multimodal segmentation. MSCANet uses a dual-branch architecture, where optical and SAR images are processed by lightweight InceptionNext-Tiny encoders. At each stage, a Residual-Calibrated Three-Stage Attention (RCTA) module sequentially applies multi-head cross-attention to model inter-modal dependencies, followed by channel and spatial attention to recalibrate features and suppress modality-specific noise within a residual learning framework. This design ensures semantic alignment and robust information flow with linear complexity. Fused multi-scale features are then aggregated in a Multi-Scale Dilated-Skip Fusion (MSDSF) decoder, which uses serial dilated convolutions and dynamic skip connections to capture rich context and recover fine boundaries. Experiments on WHU-OPT-SAR and UBCv2 datasets show that MSCANet achieves state-of-the-art performance with low computational cost. Specifically, on WHU-OPT-SAR, MSCANet reaches 59.19% mIoU, outperforming the second-best method (CEN) by 2.35%, while requiring only 15.94 GFLOPs. These results demonstrate the effectiveness of MSCANet for robust and efficient multimodal remote sensing segmentation.