跳到主要导航 跳到搜索 跳到主要内容

What to Keep and How to Drop: Informative Token Pruning for Efficient Vision Transformers

  • Northwestern Polytechnical University Xian

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Vision Transformers (ViTs) are computationally demanding for resource-constrained deployment. Existing token-reduction methods mainly focus on identifying important tokens, while paying less attention to preserving useful information carried by discarded tokens. In this paper, we propose ToPro (Token Pruning with Informative Propagation), a lightweight token-pruning framework that derives token retention scores from attention statistics through a learnable mapping, avoiding complex auxiliary predictors. To mitigate information loss, ToPro propagates informative cues from discarded tokens to semantically related retained tokens. We further introduce non-linear augmentation to alleviate token-score homogenization in deeper layers and maintain token discriminability. Extensive experiments across six configurations from three representative ViT families (ViT, DeiT, and LV-ViT) show favorable accuracy–efficiency trade-offs on ImageNet-1K, NWPU-RESISC45, and AID. On the 19-task VTAB-1K benchmark, ToPro consistently achieves the highest Group Mean among pruning methods for all six backbones. ToPro reduces ViT-Base FLOPs by 43.55% with a 0.82-percentage-point accuracy drop on ImageNet-1K. Beyond classification, ToPro achieves the highest accuracy with the lowest FLOPs among the evaluated token-pruning methods on COCO object detection and ADE20K semantic segmentation.

学术指纹

探究 'What to Keep and How to Drop: Informative Token Pruning for Efficient Vision Transformers' 的科研主题。它们共同构成独一无二的学术指纹。

引用此