Detail-Aware Local and Context-Aware Global Dual-Branch Upsampling for Pan-Sharpening

While deep learning methods have demonstrated significant success in pan-sharpening, most existing approaches suffer from insufficient high-frequency guidance from panchromatic (PAN) images during the upsampling of low-resolution multispectral (LRMS) images, often failing to reconstruct fine-grained local details and maintain global contextual consistency. To alleviate these limitations, we propose a novel detail-aware local and context-aware global dual-branch upsampling (DCDU) method for pan-sharpening. The core of DCDU is a dual-branch architecture specifically designed to decouple and leverage both local structural details and global semantic information from both modalities: the detail-aware local branch calculates feature-space correlations between LRMS images and PAN images to derive adaptive kernels, guiding the reconstruction of local neighborhoods and facilitating high-fidelity injection of fine-grained spatial structures; the context-aware global branch mainly employs a cross-modality channel-specific spatial attention mechanism to holistically model long-range dependencies, dynamically recalibrating spectral-spatial representations based on cross-modal context and capturing macrolevel semantic coherence across the entire scene. The distinct information streams from the two branches are then integrated via an adaptive and learnable mapping function. We also provide a theoretical analysis under the minimum mean square error criterion to motivate our dual-stream strategy, the efficacy of which is robustly validated by our experimental results. As a generic and lightweight upsampling module, the collaborative design can be seamlessly integrated into various backbone networks, enabling superior performance across multiple datasets.