Knowledge Distillation

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation ACL 2025 DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers ACL 2025 DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder AAAI 2025 Dropout Connects Transformers and CNNs: Transfer General Knowledge for Knowledge Distillation WACV 2025 Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization ICLR 2025 DTDES-KGE: Dual-Teacher Knowledge Distillation with Distinct Embedding Spaces for Knowledge Graph Embeddings EMNLP 2025 Dual Information Purification for Lightweight SAR Object Detection AAAI 2025 Dual-Process Image Generation ICCV 2025 DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers CVPR 2025 Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration AAAI 2025 Dynamic Expansion Diffusion Learning for Lifelong Generative Modelling AAAI 2025 Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation EMNLP 2025 Dynamic Integration of Task-Specific Adapters for Class Incremental Learning CVPR 2025 EA-KD: Entropy-based Adaptive Knowledge Distillation ICCV 2025 EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models EMNLP 2025 EchoDFKD: Data-Free Knowledge Distillation for Cardiac Ultrasound Segmentation using Synthetic Data WACV 2025 Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs COLING 2025 EdTec-ItemGen: Enhancing Retrieval-Augmented Item Generation Through Key Point Extraction ACL 2025 Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement CVPR 2025 Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution AAAI 2025 Efficient Diffusion as Low Light Enhancer CVPR 2025 Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets ACL 2025 Efficient Federated Learning via Clients-to-Server Knowledge Distillation (Student Abstract) AAAI 2025 Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction EMNLP 2025 Efficiently Editing Mixture-of-Experts Models with Compressed Experts EMNLP 2025