Model Compression

LRQuant: Learnable and Robust Post-Training Quantization for Large Language Models ACL 2024 MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization NeurIPS 2024 Make RepVGG Greater Again: A Quantization-Aware Approach AAAI 2024 MAP: MAsk-Pruning for Source-Free Model Intellectual Property Protection CVPR 2024 MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models NeurIPS 2024 Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions EMNLP 2024 MaxQ: Multi-Axis Query for N:M Sparsity Network CVPR 2024 MEFT: Memory-Efficient Fine-Tuning through Sparse Adapter ACL 2024 Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length NeurIPS 2024 MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning ACL 2024 Memory-Efficient Fine-Tuning of Transformers via Token Selection EMNLP 2024 Memory-Efficient Reversible Spiking Neural Networks AAAI 2024 Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging EMNLP 2024 Mixture of Diverse Size Experts EMNLP 2024 Mixture-of-Subspaces in Low-Rank Adaptation EMNLP 2024 Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue EMNLP 2024 MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning EMNLP 2024 MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks NeurIPS 2024 MSPE: Multi-Scale Patch Embedding Prompts Vision Transformers to Any Resolution NeurIPS 2024 Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization NeurIPS 2024 Multilingual Brain Surgeon: Large Language Models Can Be Compressed Leaving No Language behind COLING 2024 Multistep Distillation of Diffusion Models via Moment Matching NeurIPS 2024 NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time ACL 2024 Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent EMNLP 2024 NeuralFuse: Learning to Recover the Accuracy of Access-Limited Neural Network Inference in Low-Voltage Regimes NeurIPS 2024