Knowledge Distillation

Predicting Prerequisite Relations for Unseen Concepts EMNLP 2022 Preservation of the Global Knowledge by Not-True Distillation in Federated Learning NeurIPS 2022 Pretrained Speech Encoders and Efficient Fine-tuning Methods for Speech Translation: UPC at IWSLT 2022 ACL 2022 Pre-Train, Self-Train, Distill: A Simple Recipe for Supersizing 3D Reconstruction CVPR 2022 Preventing Catastrophic Forgetting and Distribution Mismatch in Knowledge Distillation via Synthetic Data WACV 2022 Production federated keyword spotting via distillation, filtering, and joint federated-centralized training Interspeech 2022 Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher COLING 2022 Prompting to Distill: Boosting Data-Free Knowledge Distillation via Reinforced Prompt IJCAI 2022 Proto2Proto: Can You Recognize the Car, the Way I Do? CVPR 2022 Prototypical Contrastive Predictive Coding ICLR 2022 Pseudo-spherical Knowledge Distillation IJCAI 2022 RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation NAACL 2022 Re2G: Retrieve, Rerank, Generate NAACL 2022 Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation EMNLP 2022 Regularizing Brain Age Prediction via Gated Knowledge Distillation MIDL 2022 Reliable Adversarial Distillation with Unreliable Teachers ICLR 2022 Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks NeurIPS 2022 Representation Compensation Networks for Continual Semantic Segmentation CVPR 2022 Respecting Transfer Gap in Knowledge Distillation NeurIPS 2022 Rethinking Resolution in the Context of Efficient Video Recognition NeurIPS 2022 Rethinking Task-Specific Knowledge Distillation: Contextualized Corpus as Better Textbook EMNLP 2022 Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters ACL 2022 Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training EMNLP 2022 Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing? ICML 2022 Rich Feature Construction for the Optimization-Generalization Dilemma ICML 2022