TY - GEN
T1 - Residual Energy Metric for Efficient Filter Pruning
AU - Jiang, Yucheng
AU - Yao, Xiwen
AU - Yang, Xuguang
AU - Wang, Shuai
AU - Jia, Yaxuan
AU - Cheng, Gong
AU - Han, Junwei
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
PY - 2025
Y1 - 2025
N2 - Filter pruning is a vital technique for effectively compressing the convolutional neural networks to comply with hardware constraints. Traditional methods measure the filter importance by averaging the importance of samples, which fails to properly decouple the influence of the unique characteristics of each sample on the filter, leading to inaccurate measurement of filter importance. Consequently, we propose a novel residual energy metric to measure the filer importance by comprehensively considering the characeristics across different samples. Specifically, the residual energy is obtained by performing a series of discrete cosine transform of feature maps, and followed by adding the original energy of feature maps to serve as the residual energy metric. Then, the filter importance is computed by its metric differences across samples. Finally, filers of small importance are pruned. Experimental results on two challenging benchmarks of various models clearly demostrate the superiority of our method. On both the CIFAR-10 and ImageNet datasets, we attained higher accuracy compared to the baseline network when the pruning rate approached 50%. Notably, on the ImageNet dataset, we achieved superior results with an extreme pruning rate, reducing the parameters of the ResNet-50 model by 68.6% and the FLOPs by 76.7%.
AB - Filter pruning is a vital technique for effectively compressing the convolutional neural networks to comply with hardware constraints. Traditional methods measure the filter importance by averaging the importance of samples, which fails to properly decouple the influence of the unique characteristics of each sample on the filter, leading to inaccurate measurement of filter importance. Consequently, we propose a novel residual energy metric to measure the filer importance by comprehensively considering the characeristics across different samples. Specifically, the residual energy is obtained by performing a series of discrete cosine transform of feature maps, and followed by adding the original energy of feature maps to serve as the residual energy metric. Then, the filter importance is computed by its metric differences across samples. Finally, filers of small importance are pruned. Experimental results on two challenging benchmarks of various models clearly demostrate the superiority of our method. On both the CIFAR-10 and ImageNet datasets, we attained higher accuracy compared to the baseline network when the pruning rate approached 50%. Notably, on the ImageNet dataset, we achieved superior results with an extreme pruning rate, reducing the parameters of the ResNet-50 model by 68.6% and the FLOPs by 76.7%.
KW - Convolutional Neural Networks
KW - Filter Pruning
KW - Residual Energy Metric
UR - http://www.scopus.com/inward/record.url?scp=105000821378&partnerID=8YFLogxK
U2 - 10.1007/978-981-96-2256-6_27
DO - 10.1007/978-981-96-2256-6_27
M3 - 会议稿件
AN - SCOPUS:105000821378
SN - 9789819622559
T3 - Lecture Notes in Electrical Engineering
SP - 256
EP - 264
BT - Advances in Guidance, Navigation and Control - Proceedings of 2024 International Conference on Guidance, Navigation and Control Volume 15
A2 - Yan, Liang
A2 - Duan, Haibin
A2 - Deng, Yimin
PB - Springer Science and Business Media Deutschland GmbH
T2 - International Conference on Guidance, Navigation and Control, ICGNC 2024
Y2 - 9 August 2024 through 11 August 2024
ER -