Visual Prompting Unsupervised Domain Adaptation for Medical Image Segmentation

Unsupervised domain adaptation (UDA), aimed at improving the segmentation performance of deep models on un-labeled data, has attracted considerable attention. Recently, the Segment Anything Model (SAM) has gained widespread attention in various scenarios. In this work, we propose a Visual Prompting UDA (VP-UDA) framework for medical image segmentation, which leverages SAM's ability to improve overall generalization performance. Specifically, we first present a Hybrid Prompting Strategy (HPS) to empower SAM with profound downstream task-specific knowledge. Moreover, a SAM-based Guidance Learning (SGL) scheme is proposed to enhance the learning process of the segmentation model. Then, we propose a Consistency-based Pseudo-label Selection (CPS) strategy to discern and exclude the outlier points in the target feature space. Furthermore, a Frequency Prior-induced Fusion (FPF) module is proposed to effectively integrate the results from SAM and the segmentation model. Experimental results show the effectiveness and superiority of our model over other state-of-the-art UDA methods.