Model Compression

NAYER: Noisy Layer Data Generation for Efficient and Effective Data-free Knowledge Distillation CVPR 2024 Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction EMNLP 2024 Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models AAAI 2024 Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models ACL 2024 OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models ICLR 2024 Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression CVPR 2024 On-Demand Federated Learning for Arbitrary Target Class Distributions AISTATS 2024 OneBit: Towards Extremely Low-bit Large Language Models NeurIPS 2024 One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls CVPR 2024 One-step Diffusion with Distribution Matching Distillation CVPR 2024 One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training AAAI 2024 On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion NeurIPS 2024 On Sampling Strategies for Spectral Model Sharding NeurIPS 2024 On the Ability of Developers' Training Data Preservation of Learnware NeurIPS 2024 On the Impact of Calibration Data in Post-training Quantization and Pruning ACL 2024 Optimal and Approximate Adaptive Stochastic Quantization NeurIPS 2024 Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs EMNLP 2024 Orthogonal Gradient Boosting for Simpler Additive Rule Ensembles AISTATS 2024 Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation AAAI 2024 Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillation NeurIPS 2024 OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models AAAI 2024 P2D: Plug and Play Discriminator for Accelerating GAN Frameworks WACV 2024 PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher NeurIPS 2024 Parameter Competition Balancing for Model Merging NeurIPS 2024 Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks EMNLP 2024