Model Compression

HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks AAAI 2023 HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation ACL 2023 How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives ACL 2023 Hybrid Inverted Index Is a Robust Accelerator for Dense Retrieval EMNLP 2023 HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks ACL 2023 Information Magnitude Based Dynamic Sub-sampling for Speech-to-text Interspeech 2023 Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models ICML 2023 Integral Neural Networks CVPR 2023 I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference ICCV 2023 Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers CVPR 2023 Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures NeurIPS 2023 Leap-of-Thought: Accelerating Transformers via Dynamic Token Routing EMNLP 2023 Learning Adversarially Robust Sparse Networks via Weight Reparameterization AAAI 2023 Learning Re-sampling Methods with Parameter Attribution for Image Super-resolution NeurIPS 2023 Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference ICLR 2023 Learning To Retain While Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation CVPR 2023 Length-Adaptive Distillation: Customizing Small Language Model for Dynamic Token Pruning EMNLP 2023 Less Is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation CVPR 2023 LightClone: Speaker-guided Parallel Subnet Selection for Few-shot Voice Cloning Interspeech 2023 LightFormer: Light-weight Transformer Using SVD-based Weight Transfer and Parameter Sharing ACL 2023 LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models EMNLP 2023 LLM-FP4: 4-Bit Floating-Point Quantized Transformers EMNLP 2023 LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation ICML 2023 Lossless 4-bit Quantization of Architecture Compressed Conformer ASR Systems on the 300-hr Switchboard Corpus Interspeech 2023 Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost AAAI 2023