Model Compression

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization NeurIPS 2024 SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design CVPR 2024 Simple and Fast Distillation of Diffusion Models NeurIPS 2024 SIRIUS : Contexual Sparisty with Correction for Efficient LLMs NeurIPS 2024 Slicing Vision Transformer for Flexible Inference NeurIPS 2024 SlimSAM: 0.1% Data Makes Segment Anything Slim NeurIPS 2024 SnapKV: LLM Knows What You are Looking for Before Generation NeurIPS 2024 SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead NeurIPS 2024 SparseFit: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations ACL 2024 SparseFlow: Accelerating Transformers by Sparsifying Information Flows ACL 2024 SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers EMNLP 2024 Sparse High Rank Adapters NeurIPS 2024 SparseLLM: Towards Global Pruning of Pre-trained Language Models NeurIPS 2024 Sparse maximal update parameterization: A holistic approach to sparse training dynamics NeurIPS 2024 Spear: Evaluate the Adversarial Robustness of Compressed Neural Models IJCAI 2024 SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation NeurIPS 2024 SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion with Winner-Oriented Spike Shift for Softmax Operation NeurIPS 2024 SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation AAAI 2024 SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification ACL 2024 SS1: Accelerating Inference with Fast and Expressive Sketch Structured Transform NeurIPS 2024 StepbaQ: Stepping backward as Correction for Quantized Diffusion Models NeurIPS 2024 Streamlining Speech Enhancement DNNs: an Automated Pruning Method Based on Dependency Graph with Advanced Regularized Loss Strategies Interspeech 2024 Structured Optimal Brain Pruning for Large Language Models EMNLP 2024 Structured Unrestricted-Rank Matrices for Parameter Efficient Finetuning NeurIPS 2024 Surgical Feature-Space Decomposition of LLMs: Why, When and How? ACL 2024