UniSeg: A Prompt-Driven Universal Segmentation Model as Well as A Strong Representation Learner

Yiwen Ye, Yutong Xie, Jianpeng Zhang, Ziyang Chen, Yong Xia

科研成果: 书/报告/会议事项章节会议稿件同行评审

23 引用 (Scopus)

摘要

The universal model emerges as a promising trend for medical image segmentation, paving up the way to build medical imaging large model (MILM). One popular strategy to build universal models is to encode each task as a one-hot vector and generate dynamic convolutional layers at the end of the decoder to extract the interested target. Although successful, it ignores the correlations among tasks and meanwhile is too late to make the model ‘aware’ of the ongoing task. To address both issues, we propose a prompt-driven Universal Segmentation model (UniSeg) for multi-task medical image segmentation using diverse modalities and domains. We first devise a learnable universal prompt to describe the correlations among all tasks and then convert this prompt and image features into a task-specific prompt, which is fed to the decoder as a part of its input. Thus, we make the model ‘aware’ of the ongoing task early and boost the task-specific training of the whole decoder. Our results indicate that the proposed UniSeg outperforms other universal models and single-task models on 11 upstream tasks. Moreover, UniSeg also beats other pre-trained models on two downstream datasets, providing the community with a high-quality pre-trained model for 3D medical image segmentation. Code and model are available at https://github.com/yeerwen/UniSeg.

源语言英语
主期刊名Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings
编辑Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
出版商Springer Science and Business Media Deutschland GmbH
508-518
页数11
ISBN(印刷版)9783031438974
DOI
出版状态已出版 - 2023
活动26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 - Vancouver, 加拿大
期限: 8 10月 202312 10月 2023

出版系列

姓名Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
14222 LNCS
ISSN(印刷版)0302-9743
ISSN(电子版)1611-3349

会议

会议26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
国家/地区加拿大
Vancouver
时期8/10/2312/10/23

指纹

探究 'UniSeg: A Prompt-Driven Universal Segmentation Model as Well as A Strong Representation Learner' 的科研主题。它们共同构成独一无二的指纹。

引用此