Knowledge Distillation

Tail-STEAK: Improve Friend Recommendation for Tail Users via Self-Training Enhanced Knowledge Distillation AAAI 2024 Task-agnostic Distillation of Encoder-Decoder Language Models COLING 2024 Task-Aware Encoder Control for Deep Video Compression CVPR 2024 Teacher as a Lenient Expert: Teacher-Agnostic Data-Free Knowledge Distillation AAAI 2024 Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models ACL 2024 Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios ACL 2024 Teaching Small Language Models Reasoning through Counterfactual Distillation EMNLP 2024 Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering ACL 2024 Teaching Tiny Minds: Exploring Methods to Enhance Knowledge Distillation for Small Language Models CoNLL 2024 Team ISM at CLPsych 2024: Extracting Evidence of Suicide Risk from Reddit Posts with Knowledge Self-Generation and Output Refinement using A Large Language Model EACL 2024 Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024 Text Grouping Adapter: Adapting Pre-trained Text Detector for Layout Analysis CVPR 2024 Text Simplification via Adaptive Teaching ACL 2024 The ALCHEmist: Automated Labeling 500x CHEaper than LLM Data Annotators NeurIPS 2024 The False Promise of Imitating Proprietary Language Models ICLR 2024 The Mamba in the Llama: Distilling and Accelerating Hybrid Models NeurIPS 2024 Three Pillars Improving Vision Foundation Model Distillation for Lidar CVPR 2024 TIKP: Text-to-Image Knowledge Preservation for Continual Semantic Segmentation AAAI 2024 tinyCLAP: Distilling Constrastive Language-Audio Pretrained Models Interspeech 2024 To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation ACL 2024 Towards Backward-Compatible Continual Learning of Image Compression CVPR 2024 Towards Continual Knowledge Graph Embedding via Incremental Distillation AAAI 2024 Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration NeurIPS 2024 Towards Domain-Aware Knowledge Distillation for Continual Model Generalization WACV 2024 Towards Multi-modal Sarcasm Detection via Disentangled Multi-grained Multi-modal Distilling COLING 2024