当前位置: X-MOL 学术Neurocomputing › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Filter pruning via separation of sparsity search and model training
Neurocomputing ( IF 5.5 ) Pub Date : 2021-08-02 , DOI: 10.1016/j.neucom.2021.07.083
Youzao Lian 1 , Peng Peng 1 , Weisheng Xu 1
Affiliation  

Network pruning has been widely used in the field of model compression and inference acceleration for convolutional neural networks(CNN). Existing methods generally follow a “training-pruning-retraining” paradigm, known as a three-stage pipeline. However, it cannot play an effective role in a pre-trained model with a larger pruning rate. In addition, prevailing methods usually set pruning rates as super parameters, which fail to consider the sensitivity of different convolution layers. In this paper, a novel pruning approach, based on the separation of sparsity search and model training(SST), is proposed to solve the above problems. Specifically, an evolutionary algorithm is introduced into the process of searching for the most suitable number of pruned filters for every layer. After obtaining the best sparsity structure, a new pruning strategy, called the one-pruning pipeline, is utilized to prune the pre-trained model. Experiments on multiple advanced CNN architectures show that SST can greatly improve the pruning rate with a slight loss of accuracy, which is found to universally reduce more than 60% FLOPs on CIFAR-10. Notably, on ILSVRC-2012, pruning based on ResNet18 reduces FLOPs by 42.8%, while top-1 and top-5 accuracy only lose 1.19% and 0.62%, respectively.



中文翻译:

通过分离稀疏搜索和模型训练进行过滤器修剪

网络剪枝已广泛应用于卷积神经网络(CNN)的模型压缩和推理加速领域。现有方法通常遵循“训练-修剪-再训练”范式,称为三阶段管道。但是,它不能在具有较大剪枝率的预训练模型中发挥有效作用。此外,主流方法通常将剪枝率设置为超参数,没有考虑不同卷积层的敏感性。在本文中,提出了一种基于稀疏搜索和模型训练(SST)分离的新型剪枝方法来解决上述问题。具体来说,将进化算法引入到为每一层搜索最合适数量的剪枝过滤器的过程中。获得最佳稀疏结构后,新的剪枝策略,称为单剪枝管道,用于剪枝预训练模型。在多个高级 CNN 架构上的实验表明,SST 可以大大提高剪枝率,但精度略有下降,发现在 CIFAR-10 上普遍减少了 60% 以上的 FLOP。值得注意的是,在 ILSVRC-2012 上,基于 ResNet18 的剪枝将 FLOPs 减少了 42.8%,而 top-1 和 top-5 的准确率分别仅下降了 1.19% 和 0.62%。

更新日期:2021-08-13
down
wechat
bug