Model Compression

SARCAT: Generative Span-Act Guided Response Generation using Copy-enhanced Target Augmentation EMNLP 2024 ScaleKD: Strong Vision Transformers Could Be Excellent Teachers NeurIPS 2024 Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices NeurIPS 2024 Selective Focus: Investigating Semantics Sensitivity in Post-training Quantization for Lane Detection AAAI 2024 Self-Calibrating Vicinal Risk Minimisation for Model Calibration CVPR 2024 Self-Supervised Quantization-Aware Knowledge Distillation AISTATS 2024 Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation IJCAI 2024 SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization NeurIPS 2024 SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation NeurIPS 2024 ShareBERT: Embeddings Are Capable of Learning Hidden Layers AAAI 2024 Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood NeurIPS 2024 ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization NeurIPS 2024 Simple and Fast Distillation of Diffusion Models NeurIPS 2024 SIRIUS : Contexual Sparisty with Correction for Efficient LLMs NeurIPS 2024 Sketching for Distributed Deep Learning: A Sharper Analysis NeurIPS 2024 Slicing Vision Transformer for Flexible Inference NeurIPS 2024 Small Scale Data-Free Knowledge Distillation CVPR 2024 SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training ACL 2024 SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model NeurIPS 2024 Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference EACL 2024 SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead NeurIPS 2024 Sparse High Rank Adapters NeurIPS 2024 SparseLLM: Towards Global Pruning of Pre-trained Language Models NeurIPS 2024 Sparse maximal update parameterization: A holistic approach to sparse training dynamics NeurIPS 2024 SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression ICLR 2024