Knowledge Distillation

Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models ACL 2025 Quantifying Knowledge Distillation using Partial Information Decomposition AISTATS 2025 SEK: Self-Explained Keywords Empower Large Language Models for Code Generation ACL 2025 Structure-aware Domain Knowledge Injection for Large Language Models ACL 2025 Uncertainty-Instructed Structure Injection for Generalizable HD Map Construction CVPR 2025 Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models Interspeech 2025 Who Taught You That? Tracing Teachers in Model Distillation ACL 2025 “Yes, My LoRD.” Guiding Language Model Extraction with Locality Reinforced Distillation ACL 2025 Active Object Detection with Knowledge Aggregation and Distillation from Large Models CVPR 2024 ADAM: Dense Retrieval Distillation with Adaptive Dark Examples ACL 2024 An Empirical Study of Multilingual Reasoning Distillation for Question Answering EMNLP 2024 An LLM-Enhanced Adversarial Editing System for Lexical Simplification COLING 2024 A Privacy-preserving Approach to Ingest Knowledge from Proprietary Web-based to Locally Run Models for Medical Progress Note Generation ACL 2024 BiKT: Enabling Bidirectional Knowledge Transfer Between Pretrained Models and Sequential Downstream Tasks EMNLP 2024 Chinese Spoken Named Entity Recognition in Real-world Scenarios: Dataset and Approaches ACL 2024 CodeM: Less Data Yields More Versatility via Ability Matrix ACL 2024 Divide-or-Conquer? Which Part Should You Distill Your LLM? EMNLP 2024 Ensemble Distillation for Unsupervised Constituency Parsing ICLR 2024 FedDGL: Federated Dynamic Graph Learning for Temporal Evolution and Data Heterogeneity ACML 2024 Find the Lady: Permutation and Re-synchronization of Deep Neural Networks AAAI 2024 FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation EMNLP 2024 Flipped Classroom: Aligning Teacher Attention with Student in Generalized Category Discovery NeurIPS 2024 InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance EMNLP 2024 LEAD: Exploring Logit Space Evolution for Model Selection CVPR 2024 Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning EMNLP 2024