Model Compression

LightFormer: Light-weight Transformer Using SVD-based Weight Transfer and Parameter Sharing ACL 2023 LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification ICLR 2023 Lion: Adversarial Distillation of Proprietary Large Language Models EMNLP 2023 LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models EMNLP 2023 LLM-FP4: 4-Bit Floating-Point Quantized Transformers EMNLP 2023 LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models EMNLP 2023 LLM-Pruner: On the Structural Pruning of Large Language Models NeurIPS 2023 LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation ICML 2023 Lossy and Lossless (L2) Post-training Model Size Compression ICCV 2023 Masked Autoencoders Are Stronger Knowledge Distillers ICCV 2023 Masked Spiking Transformer ICCV 2023 Math Word Problem Solving by Generating Linguistic Variants of Problem Statements ACL 2023 MeaeQ: Mount Model Extraction Attacks with Efficient Queries EMNLP 2023 Mem-Rec: Memory Efficient Recommendation System using Alternative Representation ACML 2023 MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers EACL 2023 Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing With Non-Learnable Primitives CVPR 2023 Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models’ Memories ACL 2023 MobileNMT: Enabling Translation in 15MB and 30ms ACL 2023 Model Compression for DNN-based Speaker Verification Using Weight Quantization Interspeech 2023 Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks ACL 2023 Modular Domain Adaptation for Conformer-Based Streaming ASR Interspeech 2023 Module-wise Adaptive Distillation for Multimodality Foundation Models NeurIPS 2023 MoEC: Mixture of Expert Clusters AAAI 2023 MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention ICCV 2023 MST-compression: Compressing and Accelerating Binary Neural Networks with Minimum Spanning Tree ICCV 2023