Model Compression

Specializing Pre-trained Language Models for Better Relational Reasoning via Network Pruning NAACL 2022 Structured Pruning Learns Compact and Accurate Models ACL 2022 Supervised Compression for Resource-Constrained Edge Computing Systems WACV 2022 Symbolic Distillation for Learned TCP Congestion Control NeurIPS 2022 Task Adaptive Parameter Sharing for Multi-Task Learning CVPR 2022 TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion EMNLP 2022 TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models ACL 2022 The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022 The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training ICLR 2022 THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption ACL 2022 To Fold or Not to Fold: a Necessary and Sufficient Condition on Batch-Normalization Layers Folding IJCAI 2022 Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning ICML 2022 Toward Efficient Low-Precision Training: Data Format Optimization and Hysteresis Quantization ICLR 2022 Towards Data-Free Model Stealing in a Hard Label Setting CVPR 2022 Towards Efficient Post-training Quantization of Pre-trained Language Models NeurIPS 2022 Towards Reasonable Budget Allocation in Untargeted Graph Structure Attacks via Gradient Debias NeurIPS 2022 Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models EMNLP 2022 Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network EMNLP 2022 Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systems Interspeech 2022 Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders EMNLP 2022 Unified Visual Transformer Compression ICLR 2022 Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space CVPR 2022 Weighted Mutual Learning with Diversity-Driven Model Compression NeurIPS 2022 When the Student Becomes the Master: Learning Better and Smaller Monolingual Models from mBERT COLING 2022 When To Prune? A Policy Towards Early Structural Pruning CVPR 2022