網絡加速和壓縮技術論文整理

從加速和壓縮本身來說，兩者不是同一件事，但通常情況下我們往往會同時做加速和壓縮，兩者都會給網絡的計算帶來收益，是以我們習慣将它們放在一起來講。

低秩近似（low-rank Approximation），網絡剪枝（network pruning），網絡量化（network quantization），知識蒸餾（knowledge distillation）和緊湊網絡設計（compact Network design）

低秩近似（low-rank Approximation）

網絡剪枝（network pruning）

網絡量化（network quantization）

2011-JMLR-Learning with Structured Sparsity
2013-NIPS-Predicting Parameters in Deep Learning
2014-BMVC-Speeding up convolutional neural networks with low rank expansions
2014-NIPS-Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
2014-NIPS-Do deep neural nets really need to be deep
2015-ICML-Compressing neural networks with the hashing trick
2015-INTERSPEECH-A Diversity-Penalizing Ensemble Training Method for Deep Learning
2015-BMVC-Data-free parameter pruning for deep neural networks
2015-NIPS-Learning both Weights and Connections for Efficient Neural Network
2015_NIPSw-Distilling Intractable Generative Models
2015-CVPR-Learning to generate chairs with convolutional neural networks
2015-CVPR-Understanding deep image representations by inverting them [2016 IJCV version:Visualizing deep convolutional neural networks using natural pre-images]
2015-CVPR-Efficient and Accurate Approximations of Nonlinear Convolutional Networks [2016 TPAMI version:Accelerating Very Deep Convolutional Networks for Classification and Detection]
2016-ICLRb-Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
2016-ICLR-All you need is a good init [Code]
2016-ICLR-Convolutional neural networks with low-rank regularization
2016-ICLR-Diversity networks
2016-EMNLP-Sequence-Level Knowledge Distillation
2016-CVPR-Inverting Visual Representations with Convolutional Networks
2016-NIPS-Learning Structured Sparsity in Deep Neural Networks
2016-NIPS-Dynamic Network Surgery for Efficient DNNs
2016.10-Deep model compression: Distilling knowledge from noisy teachers
2017-ICLR-Pruning Convolutional Neural Networks for Resource Efficient Inference
2017-ICLR-Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
2017-ICLR-Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
2017-ICML-Variational dropout sparsifies deep neural networks
2017-CVPR-Learning deep CNN denoiser prior for image restoration
2017-CVPR-Deep roots: Improving cnn efficiency with hierarchical filter groups
2017-CVPR-All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation
2017-CVPR-ResNeXt-Aggregated Residual Transformations for Deep Neural Networks
2017-CVPR-Xception: Deep learning with depthwise separable convolutions
2017-ICCV-Channel pruning for accelerating very deep neural networks [Code]
2017-ICCV-Learning efficient convolutional networks through network slimming [Code]
2017-ICCV-ThiNet: A filter level pruning method for deep neural network compression [Project]
2017-ICCV-Interleaved group convolutions
2017-NIPS-Net-trim: Convex pruning of deep neural networks with performance guarantee
2017-NIPS-Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon
2017-NNs-Nonredundant sparse feature extraction using autoencoders with receptive fields clustering
2017.02-The Power of Sparsity in Convolutional Neural Networks
2018-AAAI-Auto-balanced Filter Pruning for Efficient Convolutional Neural Networks
2018-AAAI-Deep Neural Network Compression with Single and Multiple Level Quantization
2018-ICML-On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
2018-ICMLw-Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
2018-ICLRo-Training and Inference with Integers in Deep Neural Networks
2018-ICLR-Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers
2018-ICLR-N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning
2018-ICLR-Model compression via distillation and quantization
2018-ICLR-Towards Image Understanding from Deep Compression Without Decoding
2018-ICLR-Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
2018-ICLR-Mixed Precision Training of Convolutional Neural Networks using Integer Operations
2018-ICLR-Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
2018-ICLR-Loss-aware Weight Quantization of Deep Networks
2018-ICLR-Alternating Multi-bit Quantization for Recurrent Neural Networks
2018-ICLR-Adaptive Quantization of Neural Networks
2018-ICLR-Variational Network Quantization
2018-ICLR-Learning Sparse Neural Networks through L0 Regularization
2018-ICLRw-To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model Compression (Similar topic:2018-NIPSw-nip in the bud,2018-NIPSw-rethink)
2018-ICLRw-Systematic Weight Pruning of DNNs using Alternating Direction Method of Multipliers
2018-ICLRw-Weightless: Lossy weight encoding for deep neural network compression
2018-ICLRw-Variance-based Gradient Compression for Efficient Distributed Deep Learning
2018-ICLRw-Stacked Filters Stationary Flow For Hardware-Oriented Acceleration Of Deep Convolutional Neural Networks
2018-ICLRw-Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks
2018-ICLRw-Accelerating Neural Architecture Search using Performance Prediction
2018-ICLRw-Nonlinear Acceleration of CNNs
2018-CVPR-Context-Aware Deep Feature Compression for High-Speed Visual Tracking
2018-CVPR-NISP: Pruning Networks using Neuron Importance Score Propagation
2018-CVPR-“Learning-Compression” Algorithms for Neural Net Pruning
2018-CVPR-Deep Image Prior [Code]
2018-CVPR-Condensenet: An efficient densenet using learned group convolutions
2018-CVPR-Shift: A zero flop, zero parameter alternative to spatial convolutions
2018-CVPR-Interleaved structured sparse convolutional neural networks
2018-IJCAI-Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error
2018-IJCAI-Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
2018-IJCAI-Where to Prune: Using LSTM to Guide End-to-end Pruning
2018-IJCAI-Accelerating Convolutional Networks via Global & Dynamic Filter Pruning
2018-IJCAI-Optimization based Layer-wise Magnitude-based Pruning for DNN Compression
2018-IJCAI-Progressive Blockwise Knowledge Distillation for Neural Network Acceleration
2018-IJCAI-Complementary Binary Quantization for Joint Multiple Indexing
2018-ECCV-A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
2018-ECCV-Coreset-Based Neural Network Compression
2018-ECCV-Data-Driven Sparse Structure Selection for Deep Neural Networks [Code]
2018-BMVCo-Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
2018-BMVC-Efficient Progressive Neural Architecture Search
2018-BMVC-Igcv3: Interleaved lowrank group convolutions for efficient deep neural networks
2018-NIPS-Discrimination-aware Channel Pruning for Deep Neural Networks
2018-NIPS-Frequency-Domain Dynamic Pruning for Convolutional Neural Networks
2018-NIPS-ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions
2018-NIPS-DropBlock: A regularization method for convolutional networks
2018-NIPS-Constructing fast network through deconstruction of convolution
2018-NIPS-Learning Versatile Filters for Efficient Convolutional Neural Networks [Code]
2018-NIPSw-Pruning neural networks: is it time to nip it in the bud?
2018-NIPSwb-Rethinking the Value of Network Pruning [2019 ICLR version]
2018-NIPSw-Structured Pruning for Efficient ConvNets via Incremental Regularization
2018.05-Compression of Deep Convolutional Neural Networks under Joint Sparsity Constraints
2018.05-AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference
2018.11-Second-order Optimization Method for Large Mini-batch: Training ResNet-50 on ImageNet in 35 Epochs
2018.11-Rethinking ImageNet Pre-training (Kaiming He)
2019-ICLRo-The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
2019-AAAIo-A layer decomposition-recomposition framework for neuron pruning towards accurate lightweight networks
2019-CVPR-All You Need is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification
2019-CVPR-HetConv Heterogeneous Kernel-Based Convolutions for Deep CNNs
2019-CVPR-Fully Learnable Group Convolution for Acceleration of Deep Neural Networks
2019-CVPR-Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
2019-CVPR-Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
2019-BigComp-Towards Robust Compressed Convolutional Neural Networks
2019-PR-Filter-in-Filter: Improve CNNs in a Low-cost Way by Sharing Parameters among the Sub-filters of a Filter
2019-PRL-BDNN: Binary Convolution Neural Networks for Fast Object Detection
2019.03-MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning (Face++)
2019.04-Data-Free Learning of Student Networks (Huawei)
2019.04-Resource Efficient 3D Convolutional Neural Networks
2019.04-Meta Filter Pruning to Accelerate Deep Convolutional Neural Networks
2019.04-Knowledge Squeezed Adversarial Network Compression- 2011-JMLR-Learning with Structured Sparsity
2013-NIPS-Predicting Parameters in Deep Learning
2014-BMVC-Speeding up convolutional neural networks with low rank expansions
2014-NIPS-Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
2014-NIPS-Do deep neural nets really need to be deep
2015-ICML-Compressing neural networks with the hashing trick
2015-INTERSPEECH-A Diversity-Penalizing Ensemble Training Method for Deep Learning
2015-BMVC-Data-free parameter pruning for deep neural networks
2015-NIPS-Learning both Weights and Connections for Efficient Neural Network
2015_NIPSw-Distilling Intractable Generative Models
2015-CVPR-Learning to generate chairs with convolutional neural networks
2015-CVPR-Understanding deep image representations by inverting them [2016 IJCV version:Visualizing deep convolutional neural networks using natural pre-images]
2015-CVPR-Efficient and Accurate Approximations of Nonlinear Convolutional Networks [2016 TPAMI version:Accelerating Very Deep Convolutional Networks for Classification and Detection]
2016-ICLRb-Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
2016-ICLR-All you need is a good init [Code]
2016-ICLR-Convolutional neural networks with low-rank regularization
2016-ICLR-Diversity networks
2016-EMNLP-Sequence-Level Knowledge Distillation
2016-CVPR-Inverting Visual Representations with Convolutional Networks
2016-NIPS-Learning Structured Sparsity in Deep Neural Networks
2016-NIPS-Dynamic Network Surgery for Efficient DNNs
2016.10-Deep model compression: Distilling knowledge from noisy teachers
2017-ICLR-Pruning Convolutional Neural Networks for Resource Efficient Inference
2017-ICLR-Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
2017-ICLR-Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
2017-ICML-Variational dropout sparsifies deep neural networks
2017-CVPR-Learning deep CNN denoiser prior for image restoration
2017-CVPR-Deep roots: Improving cnn efficiency with hierarchical filter groups
2017-CVPR-All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation
2017-CVPR-ResNeXt-Aggregated Residual Transformations for Deep Neural Networks
2017-CVPR-Xception: Deep learning with depthwise separable convolutions
2017-ICCV-Channel pruning for accelerating very deep neural networks [Code]
2017-ICCV-Learning efficient convolutional networks through network slimming [Code]
2017-ICCV-ThiNet: A filter level pruning method for deep neural network compression [Project]
2017-ICCV-Interleaved group convolutions
2017-NIPS-Net-trim: Convex pruning of deep neural networks with performance guarantee
2017-NIPS-Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon
2017-NNs-Nonredundant sparse feature extraction using autoencoders with receptive fields clustering
2017.02-The Power of Sparsity in Convolutional Neural Networks
2018-AAAI-Auto-balanced Filter Pruning for Efficient Convolutional Neural Networks
2018-AAAI-Deep Neural Network Compression with Single and Multiple Level Quantization
2018-ICML-On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
2018-ICMLw-Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
2018-ICLRo-Training and Inference with Integers in Deep Neural Networks
2018-ICLR-Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers
2018-ICLR-N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning
2018-ICLR-Model compression via distillation and quantization
2018-ICLR-Towards Image Understanding from Deep Compression Without Decoding
2018-ICLR-Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
2018-ICLR-Mixed Precision Training of Convolutional Neural Networks using Integer Operations
2018-ICLR-Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
2018-ICLR-Loss-aware Weight Quantization of Deep Networks
2018-ICLR-Alternating Multi-bit Quantization for Recurrent Neural Networks
2018-ICLR-Adaptive Quantization of Neural Networks
2018-ICLR-Variational Network Quantization
2018-ICLR-Learning Sparse Neural Networks through L0 Regularization
2018-ICLRw-To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model Compression (Similar topic:2018-NIPSw-nip in the bud,2018-NIPSw-rethink)
2018-ICLRw-Systematic Weight Pruning of DNNs using Alternating Direction Method of Multipliers
2018-ICLRw-Weightless: Lossy weight encoding for deep neural network compression
2018-ICLRw-Variance-based Gradient Compression for Efficient Distributed Deep Learning
2018-ICLRw-Stacked Filters Stationary Flow For Hardware-Oriented Acceleration Of Deep Convolutional Neural Networks
2018-ICLRw-Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks
2018-ICLRw-Accelerating Neural Architecture Search using Performance Prediction
2018-ICLRw-Nonlinear Acceleration of CNNs
2018-CVPR-Context-Aware Deep Feature Compression for High-Speed Visual Tracking
2018-CVPR-NISP: Pruning Networks using Neuron Importance Score Propagation
2018-CVPR-“Learning-Compression” Algorithms for Neural Net Pruning
2018-CVPR-Deep Image Prior [Code]
2018-CVPR-Condensenet: An efficient densenet using learned group convolutions
2018-CVPR-Shift: A zero flop, zero parameter alternative to spatial convolutions
2018-CVPR-Interleaved structured sparse convolutional neural networks
2018-IJCAI-Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error
2018-IJCAI-Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
2018-IJCAI-Where to Prune: Using LSTM to Guide End-to-end Pruning
2018-IJCAI-Accelerating Convolutional Networks via Global & Dynamic Filter Pruning
2018-IJCAI-Optimization based Layer-wise Magnitude-based Pruning for DNN Compression
2018-IJCAI-Progressive Blockwise Knowledge Distillation for Neural Network Acceleration
2018-IJCAI-Complementary Binary Quantization for Joint Multiple Indexing
2018-ECCV-A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
2018-ECCV-Coreset-Based Neural Network Compression
2018-ECCV-Data-Driven Sparse Structure Selection for Deep Neural Networks [Code]
2018-BMVCo-Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
2018-BMVC-Efficient Progressive Neural Architecture Search
2018-BMVC-Igcv3: Interleaved lowrank group convolutions for efficient deep neural networks
2018-NIPS-Discrimination-aware Channel Pruning for Deep Neural Networks
2018-NIPS-Frequency-Domain Dynamic Pruning for Convolutional Neural Networks
2018-NIPS-ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions
2018-NIPS-DropBlock: A regularization method for convolutional networks
2018-NIPS-Constructing fast network through deconstruction of convolution
2018-NIPS-Learning Versatile Filters for Efficient Convolutional Neural Networks [Code]
2018-NIPSw-Pruning neural networks: is it time to nip it in the bud?
2018-NIPSwb-Rethinking the Value of Network Pruning [2019 ICLR version]
2018-NIPSw-Structured Pruning for Efficient ConvNets via Incremental Regularization
2018.05-Compression of Deep Convolutional Neural Networks under Joint Sparsity Constraints
2018.05-AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference
2018.11-Second-order Optimization Method for Large Mini-batch: Training ResNet-50 on ImageNet in 35 Epochs
2018.11-Rethinking ImageNet Pre-training (Kaiming He)
2019-ICLRo-The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
2019-AAAIo-A layer decomposition-recomposition framework for neuron pruning towards accurate lightweight networks
2019-CVPR-All You Need is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification
2019-CVPR-HetConv Heterogeneous Kernel-Based Convolutions for Deep CNNs
2019-CVPR-Fully Learnable Group Convolution for Acceleration of Deep Neural Networks
2019-CVPR-Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
2019-CVPR-Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
2019-BigComp-Towards Robust Compressed Convolutional Neural Networks
2019-PR-Filter-in-Filter: Improve CNNs in a Low-cost Way by Sharing Parameters among the Sub-filters of a Filter
2019-PRL-BDNN: Binary Convolution Neural Networks for Fast Object Detection
2019.03-MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning (Face++)
2019.04-Data-Free Learning of Student Networks (Huawei)
2019.04-Resource Efficient 3D Convolutional Neural Networks
2019.04-Meta Filter Pruning to Accelerate Deep Convolutional Neural Networks
2019.04-Knowledge Squeezed Adversarial Network Compression

網絡加速和壓縮技術論文整理

繼續閱讀

Spring MVC+Ajax建立執行個體

【FPGA實作GA】基于FPGA的GA優化算法的設計與實作

解決方案之：DM relay 處理單元報錯

吃透MVC，馴服爛代碼吃透MVC，馴服爛代碼

Perl與網絡監控

Lua的預設metamethod清單

Cocos、Lua遊戲記憶體釋放之我見

Lua遊戲記憶體洩漏檢查

SSM整合02---Spring-SpringMVC層一、添加web架構支援二、web.xml三、spring-mvc.xml

UVA 442 Matrix Chain Multiplication

PAT (Advanced Level) Practise 1131 Subway Map (30)

ZOJ 3938 Defuse the Bomb

CSU 1565 Word Cloud

ZOJ 3700 Ever Dream

ZOJ 1199 Point of Intersection

CSU 1567 Reverse Rot