Model Compression

MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization NeurIPS 2024 Make RepVGG Greater Again: A Quantization-Aware Approach AAAI 2024 mALBERT: Is a Compact Multilingual BERT Model Still Worth It? COLING 2024 MAP: MAsk-Pruning for Source-Free Model Intellectual Property Protection CVPR 2024 MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models NeurIPS 2024 MathPile: A Billion-Token-Scale Pretraining Corpus for Math NeurIPS 2024 Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions EMNLP 2024 MaxQ: Multi-Axis Query for N:M Sparsity Network CVPR 2024 MediSwift: Efficient Sparse Pre-trained Biomedical Language Models ACL 2024 MEFT: Memory-Efficient Fine-Tuning through Sparse Adapter ACL 2024 Memory-Efficient LLM Training with Online Subspace Descent NeurIPS 2024 MEMORY-VQ: Compression for Tractable Internet-Scale Memory NAACL 2024 Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy ICLR 2024 MetaMix: Meta-State Precision Searcher for Mixed-Precision Activation Quantization AAAI 2024 Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing EMNLP 2024 Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices CVPR 2024 Mixture-of-Subspaces in Low-Rank Adaptation EMNLP 2024 Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts ACL 2024 MobileQuant: Mobile-friendly Quantization for On-device Language Models EMNLP 2024 MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning EMNLP 2024 MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks NeurIPS 2024 MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric CVPR 2024 Multi-Head Mixture-of-Experts NeurIPS 2024 Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization NeurIPS 2024 Multilingual Brain Surgeon: Large Language Models Can Be Compressed Leaving No Language behind COLING 2024