DeSD: Self-Supervised Learning with Deep Self-Distillation for 3D Medical Image Segmentation

Yiwen Ye, Jianpeng Zhang, Ziyang Chen, Yong Xia

科研成果: 书/报告/会议事项章节会议稿件同行评审

17 引用 (Scopus)

摘要

Self-supervised learning (SSL), enabling advanced performance with few annotations, has demonstrated a proven successful in medical image segmentation. Usually, SSL relies on measuring the similarity of features obtained at the deepest layer to attract the features of positive pairs or repulse the features of negative pairs, and then may suffer from the weak supervision at shallow layers. To address this issue, we reformulate SSL in a Deep Self-Distillation (DeSD) manner to improve the representation quality of both shallow and deep layers. Specifically, the DeSD model is composed of an online student network and a momentum teacher network, both being stacked by multiple sub-encoders. The features produced by each sub-encoder in the student network are trained to match the features produced by the teacher network. Such a deep self-distillation supervision is able to improve the representation quality of all sub-encoders, including both shallow ones and deep ones. We pre-train the DeSD model on a large-scale unlabeled dataset and evaluate it on seven downstream segmentation tasks. Our results indicate that the proposed DeSD model achieves superior pre-training performance over existing SSL methods, setting the new state of the art. The code is available at https://github.com/yeerwen/DeSD.

源语言英语
主期刊名Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings
编辑Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li
出版商Springer Science and Business Media Deutschland GmbH
545-555
页数11
ISBN(印刷版)9783031164392
DOI
出版状态已出版 - 2022
活动25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 - Singapore, 新加坡
期限: 18 9月 202222 9月 2022

出版系列

姓名Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
13434 LNCS
ISSN(印刷版)0302-9743
ISSN(电子版)1611-3349

会议

会议25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022
国家/地区新加坡
Singapore
时期18/09/2222/09/22

指纹

探究 'DeSD: Self-Supervised Learning with Deep Self-Distillation for 3D Medical Image Segmentation' 的科研主题。它们共同构成独一无二的指纹。

引用此