Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation

Yuanwei Liu, Nian Liu, Xiwen Yao, Junwei Han

科研成果: 书/报告/会议事项章节会议稿件同行评审

59 引用 (Scopus)

摘要

Few-shot semantic segmentation aims to segment the target objects in query under the condition of a few annotated support images. Most previous works strive to mine more effective category information from the support to match with the corresponding objects in query. However, they all ignored the category information gap between query and support images. If the objects in them show large intra-class diversity, forcibly migrating the category information from the support to the query is ineffective. To solve this problem, we are the first to introduce an intermediate prototype for mining both deterministic category information from the support and adaptive category knowledge from the query. Specifically, we design an Intermediate Prototype Mining Transformer (IPMT) to learn the prototype in an iterative way. In each IPMT layer, we propagate the object information in both support and query features to the prototype and then use it to activate the query feature map. By conducting this process iteratively, both the intermediate prototype and the query feature can be progressively improved. At last, the final query feature is used to yield precise segmentation prediction. Extensive experiments on both PASCAL-5i and COCO-20i datasets clearly verify the effectiveness of our IPMT and show that it outperforms previous state-of-the-art methods by a large margin. Code is available at https://github.com/LIUYUANWEI98/IPMT.

源语言英语
主期刊名Advances in Neural Information Processing Systems 35 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022
编辑S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
出版商Neural information processing systems foundation
ISBN(电子版)9781713871088
出版状态已出版 - 2022
活动36th Conference on Neural Information Processing Systems, NeurIPS 2022 - New Orleans, 美国
期限: 28 11月 20229 12月 2022

出版系列

姓名Advances in Neural Information Processing Systems
35
ISSN(印刷版)1049-5258

会议

会议36th Conference on Neural Information Processing Systems, NeurIPS 2022
国家/地区美国
New Orleans
时期28/11/229/12/22

指纹

探究 'Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation' 的科研主题。它们共同构成独一无二的指纹。

引用此