Model Compression

Pruning Multilingual Large Language Models for Multilingual Inference EMNLP 2024 Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging EMNLP 2024 PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression NeurIPS 2024 PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference ACL 2024 QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models ICLR 2024 QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning EMNLP 2024 QT-ViT: Improving Linear Attention in ViT with Quadratic Taylor Expansion NeurIPS 2024 Quantization of Large Language Models with an Overdetermined Basis UAI 2024 Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models ACL 2024 QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs NeurIPS 2024 QuIP$#$: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks ICML 2024 Q-VLM: Post-training Quantization for Large Vision-Language Models NeurIPS 2024 RA-LoRA: Rank-Adaptive Parameter-Efficient Fine-Tuning for Accurate 2-bit Quantized Large Language Models ACL 2024 READ-PVLA: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling AAAI 2024 Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing EMNLP 2024 RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs NAACL 2024 Reducing Transformer Key-Value Cache Size with Cross-Layer Attention NeurIPS 2024 Refusal in Language Models Is Mediated by a Single Direction NeurIPS 2024 ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models ICLR 2024 Representation and Generation of Machine Learning Test Functions EACL 2024 ResLoRA: Identity Residual Mapping in Low-Rank Adaption ACL 2024 Resource-Efficient Neural Networks for Embedded Systems JMLR 2024 Resource-Efficient Transformer Pruning for Finetuning of Large Models CVPR 2024 RETAIN: Interactive Tool for Regression Testing Guided LLM Migration EMNLP 2024 Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models ICLR 2024