Knowledge Distillation

To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation ACL 2024 Token-wise Influential Training Data Retrieval for Large Language Models ACL 2024 Towards Continual Knowledge Graph Embedding via Incremental Distillation AAAI 2024 Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition ACL 2024 Training Graph Transformers via Curriculum-Enhanced Attention Distillation ICLR 2024 Transformer as Linear Expansion of Learngene AAAI 2024 Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Data AAAI 2024 Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging NeurIPS 2024 UDON: Universal Dynamic Online distillatioN for generic image representations NeurIPS 2024 Understanding the Role of the Projector in Knowledge Distillation AAAI 2024 UniADS: Universal Architecture-Distiller Search for Distillation Gap AAAI 2024 Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decoding ACL 2024 Unveiling Implicit Table Knowledge with Question-Then-Pinpoint Reasoner for Insightful Table Summarization EMNLP 2024 Variational Distillation of Diffusion Policies into Mixture of Experts NeurIPS 2024 VE-KD: Vocabulary-Expansion Knowledge-Distillation for Training Smaller Domain-Specific Language Models EMNLP 2024 Video Event Extraction with Multi-View Interaction Knowledge Distillation AAAI 2024 Visible-Infrared Person Re-Indentification via Feature Fusion and Deep Mutual Learning ACML 2024 Vision-Language Models are Strong Noisy Label Detectors NeurIPS 2024 Visual Editing with LLM-based Tool Chaining: An Efficient Distillation Approach for Real-Time Applications EMNLP 2024 Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models CVPR 2024 VkD: Improving Knowledge Distillation using Orthogonal Projections CVPR 2024 Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation NeurIPS 2024 Weak-to-Strong 3D Object Detection with X-Ray Distillation CVPR 2024 Wired Perspectives: Multi-View Wire Art Embraces Generative AI CVPR 2024 XKD: Cross-Modal Knowledge Distillation with Domain Alignment for Video Representation Learning AAAI 2024