Model Compression

SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification ACL 2024 SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models EMNLP 2024 SS1: Accelerating Inference with Fast and Expressive Sketch Structured Transform NeurIPS 2024 Stealth edits to large language models NeurIPS 2024 Stochastic Fine-Tuning of Language Models Using Masked Gradients EMNLP 2024 Streamlining Speech Enhancement DNNs: an Automated Pruning Method Based on Dependency Graph with Advanced Regularized Loss Strategies Interspeech 2024 Structured Optimal Brain Pruning for Large Language Models EMNLP 2024 Structured Pruning for Large Language Models Using Coupled Components Elimination and Minor Fine-tuning NAACL 2024 STTATTS: Unified Speech-To-Text And Text-To-Speech Model EMNLP 2024 Style Vectors for Steering Generative Large Language Models EACL 2024 Surgical Feature-Space Decomposition of LLMs: Why, When and How? ACL 2024 SwapMoE: Serving Off-the-shelf MoE-based Large Language Models with Tunable Memory Budget ACL 2024 SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS ICLR 2024 SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning ICLR 2024 Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks NeurIPS 2024 Tangent Transformers for Composition,Privacy and Removal ICLR 2024 Task-agnostic Distillation of Encoder-Decoder Language Models COLING 2024 Task-Agnostic Self-Distillation for Few-Shot Action Recognition IJCAI 2024 Taylor Unswift: Secured Weight Release for Large Language Models via Taylor Expansion EMNLP 2024 The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry ICLR 2024 The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information NeurIPS 2024 The LLM Surgeon ICLR 2024 The Mamba in the Llama: Distilling and Accelerating Hybrid Models NeurIPS 2024 The Need for Speed: Pruning Transformers with One Recipe ICLR 2024 The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction ICLR 2024