Model Compression

SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters EMNLP 2022 Sparse Double Descent: Where Network Pruning Aggravates Overfitting ICML 2022 Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT EMNLP 2022 Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm ACL 2022 Sparse Structure Search for Delta Tuning NeurIPS 2022 Sparse Teachers Can Be Dense with Knowledge EMNLP 2022 Sparsifying Transformer Models with Trainable Representation Pooling ACL 2022 Spartan: Differentiable Sparsity via Regularized Transportation NeurIPS 2022 Structural Pruning via Latency-Saliency Knapsack NeurIPS 2022 Structured Pruning Learns Compact and Accurate Models ACL 2022 Symbolic Distillation for Learned TCP Congestion Control NeurIPS 2022 Teach Less, Learn More: On the Undistillable Classes in Knowledge Distillation NeurIPS 2022 TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion EMNLP 2022 TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models ACL 2022 The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks ICML 2022 The Importance of Being Parameters: An Intra-Distillation Method for Serious Gains EMNLP 2022 The State of Sparse Training in Deep Reinforcement Learning ICML 2022 The trade-offs of model size in large recommendation models : 100GB to 10MB Criteo-tb DLRM model NeurIPS 2022 Thrifty Neural Architecture Search for Medical Image Segmentation (Student Abstract) AAAI 2022 Token and Head Adaptive Transformers for Efficient Natural Language Processing COLING 2022 Token Dropping for Efficient BERT Pretraining ACL 2022 Too Brittle to Touch: Comparing the Stability of Quantization and Distillation towards Developing Low-Resource MT Models EMNLP 2022 Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning ICML 2022 To update or not to update? Neurons at equilibrium in deep models NeurIPS 2022 Towards Disentangling Information Paths with Coded ResNeXt NeurIPS 2022