Model Compression

LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models ICLR 2024 LoNAS: Elastic Low-Rank Adapters for Efficient Large Language Models COLING 2024 LoQT: Low-Rank Adapters for Quantized Pretraining NeurIPS 2024 LoRAExit: Empowering Dynamic Modulation of LLMs in Resource-limited Settings using Low-rank Adapters EMNLP 2024 LoRA-GA: Low-Rank Adaptation with Gradient Approximation NeurIPS 2024 LoRAN: Improved Low-Rank Adaptation by a Non-Linear Transformation EMNLP 2024 LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning ACL 2024 LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild ACL 2024 LoRASC: Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning EMNLP 2024 LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models NAACL 2024 Lossless Acceleration of Large Language Model via Adaptive N-gram Parallel Decoding NAACL 2024 Low-Rank Approximation for Sparse Attention in Multi-Modal LLMs CVPR 2024 Low-Rank Prune-And-Factorize for Language Model Compression COLING 2024 LQ-LoRA: Low-rank plus Quantized Matrix Decomposition for Efficient Language Model Finetuning ICLR 2024 LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network AAAI 2024 LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models ICLR 2024 Machine Unlearning via Null Space Calibration IJCAI 2024 MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer CVPR 2024 MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization NeurIPS 2024 MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models NeurIPS 2024 MediSwift: Efficient Sparse Pre-trained Biomedical Language Models ACL 2024 Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models EMNLP 2024 Memory-Efficient Fine-Tuning of Transformers via Token Selection EMNLP 2024 MemoryFormer : Minimize Transformer Computation by Removing Fully-Connected Layers NeurIPS 2024 Mentor-KD: Making Small Language Models Better Multi-step Reasoners EMNLP 2024