Model Compression

Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch ICML 2023 ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradient Accumulation ICCV 2023 ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector CVPR 2023 SCConv: Spatial and Channel Reconstruction Convolution for Feature Redundancy CVPR 2023 Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time NeurIPS 2023 Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers EACL 2023 ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer NeurIPS 2023 SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models ICML 2023 Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective CVPR 2023 SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot ICML 2023 Sparse Low-rank Adaptation of Pre-trained Language Models EMNLP 2023 Sparse Training with Lipschitz Continuous Loss Functions and a Weighted Group L0-norm Constraint JMLR 2023 SparseVSR: Lightweight and Noise Robust Visual Speech Recognition Interspeech 2023 Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation EMNLP 2023 Steganography of Steganographic Networks AAAI 2023 STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition ICML 2023 Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning ACL 2023 Structured Pruning for Efficient Generative Pre-trained Language Models ACL 2023 SUBP: Soft Uniform Block Pruning for 1×N Sparse CNNs Multithreading Acceleration NeurIPS 2023 SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient ICML 2023 Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers EACL 2023 Temporal Dynamic Quantization for Diffusion Models NeurIPS 2023 TexQ: Zero-shot Network Quantization with Texture Feature Distribution Calibration NeurIPS 2023 The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models EMNLP 2023 The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter NeurIPS 2023