Model Compression

ServerlessLLM: Low-Latency Serverless Inference for Large Language Models OSDI 2024 SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation NeurIPS 2024 ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models EMNLP 2024 ShareBERT: Embeddings Are Capable of Learning Hidden Layers AAAI 2024 Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood NeurIPS 2024 Shears: Unstructured Sparsity with Neural Low-rank Adapter Search NAACL 2024 ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization NeurIPS 2024 Sign Value Constraint Decomposition for Efficient 1-Bit Quantization of Speech Translation Tasks Interspeech 2024 Sinkhorn Distance Minimization for Knowledge Distillation COLING 2024 SIRIUS : Contexual Sparisty with Correction for Efficient LLMs NeurIPS 2024 SliceGPT: Compress Large Language Models by Deleting Rows and Columns ICLR 2024 Slicing Mutual Information Generalization Bounds for Neural Networks ICML 2024 SlimGPT: Layer-wise Structured Pruning for Large Language Models NeurIPS 2024 SLTrain: a sparse plus low rank approach for parameter and memory efficient pretraining NeurIPS 2024 SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models COLING 2024 Soft Prompt Recovers Compressed LLMs, Transferably ICML 2024 Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference EACL 2024 SparseFlow: Accelerating Transformers by Sparsifying Information Flows ACL 2024 SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers EMNLP 2024 Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency ICML 2024 Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging ICLR 2024 Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN ICLR 2024 Sparsity-Accelerated Training for Large Language Models ACL 2024 Spear: Evaluate the Adversarial Robustness of Compressed Neural Models IJCAI 2024 Spectral Adapter: Fine-Tuning in Spectral Space NeurIPS 2024