Source-free domain adaptation using prompt learning for medical image segmentation

The domain discrepancy in medical images acquired under different situations renders a major hurdle in deploying pre-trained medical image segmentation models for clinical use. Since it is less possible to distribute training data with the pre-trained model due to the huge data size and privacy concerns, source-free unsupervised domain adaptation (SFDA) has recently been increasingly studied based on either pseudo labels or prior knowledge. However, when the domain discrepancy is large, the image features and probability maps used by pseudo label-based SFDA and the consistent prior assumption and prior prediction network used by prior-guided SFDA may become less reliable. In this paper, we propose a Prompt learning based SFDA (ProSFDA) method for medical image segmentation, aiming to improve the quality of domain adaption by explicitly minimizing the domain discrepancy. The workflow of ProSFDA consists of a Prompt Learning Stage (PLS) and a Feature Alignment Stage (FAS). Specifically, in PLS, we estimate source-domain images via adding a domain-aware prompt to target-domain images. We then optimize the prompt by minimizing the statistic alignment loss, and thereby prompt the source model to generate reliable predictions on the altered target-domain images. In FAS, we also align the features of altered target-domain images and their styles-augmented counterparts to optimize the source model, and hence push the model to extract compact features. We have evaluated our ProSFDA on two multi-domain medical image segmentation benchmarks. Our results indicate that the proposed ProSFDA outperforms other SFDA methods substantially and is even comparable to UDA methods. Code is available at https://github.com/ShishuaiHu/ProSFDA.