A Semantic-Guided Framework for Few-Shot Remote Sensing Object Detection

Few-shot object detection (FSOD) aims to recognize novel class targets using limited annotated data. Conventional approaches rely on extensive base class training, followed by fine-tuning where few instances from both base and novel classes are sampled for each category. Although they demonstrate remarkable performance in natural image domains, the specificity of remote sensing scenarios poses two critical challenges for FSOD: 1) the morphological differences between remote sensing images and natural images are significant, leading to a loss of structural priors in the region proposal network (RPN). This makes it difficult for structural priors pretrained on natural images to generalize remote sensing images, especially for novel class with scarce data and 2) differences in imaging conditions lead to appearance variations among similar objects, leading to sparse visual features that are insufficient to represent the common semantic structure of the entire class. To solve the problems above, we introduce an innovative framework named ST-FSOD. Primarily, we introduce the SAM-augmented region proposal network (SA-RPN) module, which leverages efficient pixel association capability to generate high-quality foreground object proposals. Subsequently, through a text-guiding learner (TGL) module, we use textual labels of each category to generate image-agnostic text-guided prototypes. The enhanced text prototypes are fused with visual features to complement the sparse visual features. Extensive experiments conducted on the DIOR, NWPU VHR-10, and RSOD benchmarks demonstrate that the proposed method consistently surpasses strong baselines and achieves superior performance compared to previous state-of-the-art (SOTA) approaches. This article will be open-sourced soon on https://github.com/wdcjhyy/ST-FSOD