Knowledge Distillation

One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation NAACL 2022 Online Knowledge Distillation by Temporal-Spatial Boosting WACV 2022 Open-Vocabulary One-Stage Detection With Hierarchical Visual-Language Knowledge Distillation CVPR 2022 Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation ACL 2022 Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation CVPR 2022 Parameter Differentiation Based Multilingual Neural Machine Translation AAAI 2022 Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search CVPR 2022 Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks AACL 2022 PILE: Pairwise Iterative Logits Ensemble for Multi-Teacher Labeled Distillation EMNLP 2022 PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient NeurIPS 2022 Plug and Play Knowledge Distillation for kNN-LM with External Logits AACL 2022 Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation CVPR 2022 Predicting Prerequisite Relations for Unseen Concepts EMNLP 2022 Progressive Distillation for Fast Sampling of Diffusion Models ICLR 2022 Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher COLING 2022 Proper Reuse of Image Classification Features Improves Object Detection CVPR 2022 Proto2Proto: Can You Recognize the Car, the Way I Do? CVPR 2022 Pseudo-spherical Knowledge Distillation IJCAI 2022 Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation ACL 2022 Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation EMNLP 2022 Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks NeurIPS 2022 Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification Interspeech 2022 Respecting Transfer Gap in Knowledge Distillation NeurIPS 2022 Rethinking Resolution in the Context of Efficient Video Recognition NeurIPS 2022 Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay AAAI 2022