Knowledge Distillation

MTA4DPR: Multi-Teaching-Assistants Based Iterative Knowledge Distillation for Dense Passage Retrieval EMNLP 2024 Multi-Expert Distillation for Few-Shot Coordination (Student Abstract) AAAI 2024 Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation ACL 2024 Neural Architecture Search Finds Robust Models by Knowledge Distillation UAI 2024 No Head Left Behind – Multi-Head Alignment Distillation for Transformers AAAI 2024 Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation CVPR 2024 Object Attribute Matters in Visual Question Answering AAAI 2024 Omnipotent Distillation with LLMs for Weakly-Supervised Natural Language Video Localization: When Divergence Meets Consistency AAAI 2024 One-Step Effective Diffusion Network for Real-World Image Super-Resolution NeurIPS 2024 On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion NeurIPS 2024 On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm CVPR 2024 On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving CVPR 2024 Optimizing Chinese Lexical Simplification Across Word Types: A Hybrid Approach EMNLP 2024 Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation CVPR 2024 Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation AAAI 2024 Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillation NeurIPS 2024 P2D: Plug and Play Discriminator for Accelerating GAN Frameworks WACV 2024 PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher NeurIPS 2024 PairDistill: Pairwise Relevance Distillation for Dense Retrieval EMNLP 2024 PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation CVPR 2024 PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor CVPR 2024 PELA: Learning Parameter-Efficient Models with Low-Rank Approximation CVPR 2024 Persistence Homology Distillation for Semi-supervised Continual Learning NeurIPS 2024 Pioneering Reliable Assessment in Text-to-Image Knowledge Editing: Leveraging a Fine-Grained Dataset and an Innovative Criterion EMNLP 2024 Plug-and-Play Diffusion Distillation CVPR 2024