Residual Energy Metric for Efficient Filter Pruning

Yucheng Jiang, Xiwen Yao, Xuguang Yang, Shuai Wang, Yaxuan Jia, Gong Cheng, Junwei Han

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Filter pruning is a vital technique for effectively compressing the convolutional neural networks to comply with hardware constraints. Traditional methods measure the filter importance by averaging the importance of samples, which fails to properly decouple the influence of the unique characteristics of each sample on the filter, leading to inaccurate measurement of filter importance. Consequently, we propose a novel residual energy metric to measure the filer importance by comprehensively considering the characeristics across different samples. Specifically, the residual energy is obtained by performing a series of discrete cosine transform of feature maps, and followed by adding the original energy of feature maps to serve as the residual energy metric. Then, the filter importance is computed by its metric differences across samples. Finally, filers of small importance are pruned. Experimental results on two challenging benchmarks of various models clearly demostrate the superiority of our method. On both the CIFAR-10 and ImageNet datasets, we attained higher accuracy compared to the baseline network when the pruning rate approached 50%. Notably, on the ImageNet dataset, we achieved superior results with an extreme pruning rate, reducing the parameters of the ResNet-50 model by 68.6% and the FLOPs by 76.7%.

Original languageEnglish
Title of host publicationAdvances in Guidance, Navigation and Control - Proceedings of 2024 International Conference on Guidance, Navigation and Control Volume 15
EditorsLiang Yan, Haibin Duan, Yimin Deng
PublisherSpringer Science and Business Media Deutschland GmbH
Pages256-264
Number of pages9
ISBN (Print)9789819622559
DOIs
StatePublished - 2025
EventInternational Conference on Guidance, Navigation and Control, ICGNC 2024 - Changsha, China
Duration: 9 Aug 202411 Aug 2024

Publication series

NameLecture Notes in Electrical Engineering
Volume1351 LNEE
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119

Conference

ConferenceInternational Conference on Guidance, Navigation and Control, ICGNC 2024
Country/TerritoryChina
CityChangsha
Period9/08/2411/08/24

Keywords

  • Convolutional Neural Networks
  • Filter Pruning
  • Residual Energy Metric

Fingerprint

Dive into the research topics of 'Residual Energy Metric for Efficient Filter Pruning'. Together they form a unique fingerprint.

Cite this