Knowledge Distillation

On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving CVPR 2024 On the Surprising Effectiveness of Attention Transfer for Vision Transformers NeurIPS 2024 Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation CVPR 2024 Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models Interspeech 2024 Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation AAAI 2024 Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillation NeurIPS 2024 PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning NAACL 2024 PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher NeurIPS 2024 PairDistill: Pairwise Relevance Distillation for Dense Retrieval EMNLP 2024 Parameter-Efficient Multi-Task Model Fusion with Partial Linearization ICLR 2024 PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation CVPR 2024 Pauk at SemEval-2024 Task 4: A Neuro-Symbolic Method for Consistent Classification of Propaganda Techniques in Memes SemEval 2024 PB-LLM: Partially Binarized Large Language Models ICLR 2024 PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor CVPR 2024 PELA: Learning Parameter-Efficient Models with Low-Rank Approximation CVPR 2024 PEMT: Multi-Task Correlation Guided Mixture-of-Experts Enables Parameter-Efficient Transfer Learning ACL 2024 PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees CVPR 2024 Performance-Guided LLM Knowledge Distillation for Efficient Text Classification at Scale EMNLP 2024 Persistence Homology Distillation for Semi-supervised Continual Learning NeurIPS 2024 PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs CVPR 2024 Pinyin Regularization in Error Correction for Chinese Speech Recognition with Large Language Models Interspeech 2024 PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models NeurIPS 2024 PITA: Prompting Task Interaction for Argumentation Mining ACL 2024 PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs ACL 2024 Plug-and-Play Diffusion Distillation CVPR 2024