Model Compression

Learn-to-Share: A Hardware-friendly Transfer Learning Framework Exploiting Computation and Parameter Sharing ICML 2021 Leveraging Sparse Linear Layers for Debuggable Deep Networks ICML 2021 Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning ICLR 2021 Manifold Regularized Dynamic Network Pruning CVPR 2021 Memory-efficient Transformers via Top-k Attention EMNLP 2021 MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge NeurIPS 2021 Minimally Invasive Surgery for Sparse Neural Networks in Contrastive Manner CVPR 2021 MixKD: Towards Efficient Distillation of Large-scale Language Models ICLR 2021 Modality-specific Distillation NAACL 2021 Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! NAACL 2021 Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network ICLR 2021 Network Pruning via Performance Maximization CVPR 2021 Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects NAACL 2021 Neural Pruning via Growing Regularization ICLR 2021 NPAS: A Compiler-Aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration CVPR 2021 On Attention Redundancy: A Comprehensive Study NAACL 2021 Once Quantization-Aware Training: High Performance Extremely Low-Bit Architecture Search ICCV 2021 On-Device Text Representations Robust To Misspellings via Projections EACL 2021 Only Train Once: A One-Shot Neural Network Training And Pruning Framework NeurIPS 2021 On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation ACL 2021 On the Predictability of Pruning Across Scales ICML 2021 OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization AAAI 2021 OSOA: One-Shot Online Adaptation of Deep Generative Models for Lossless Compression NeurIPS 2021 Parameter-Efficient Transfer Learning with Diff Pruning ACL 2021 PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification Interspeech 2021