Knowledge Distillation

Reinforced Multi-Teacher Selection for Knowledge Distillation AAAI 2021 Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation ACL 2021 REPAINT: Knowledge Transfer in Deep Reinforcement Learning ICML 2021 Rethinking Soft Labels for Knowledge Distillation: A Bias–Variance Tradeoff Perspective ICLR 2021 Retrieval Augmentation Reduces Hallucination in Conversation EMNLP 2021 Revealing the Reciprocal Relations Between Self-Supervised Stereo and Monocular Depth Estimation ICCV 2021 Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better ICCV 2021 Revisiting Knowledge Distillation: An Inheritance and Exploration Framework CVPR 2021 Revisiting Pretraining with Adapters ACL 2021 Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation ACML 2021 Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network ACL 2021 Robust Knowledge Transfer via Hybrid Forward on the Teacher-Student Model AAAI 2021 Robust Overfitting may be mitigated by properly learned smoothening ICLR 2021 RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking EMNLP 2021 ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques AAAI 2021 RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation EMNLP 2021 S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-Bit Neural Networks via Guided Distribution Calibration CVPR 2021 Sampling and Filtering of Neural Machine Translation Distillation Data NAACL 2021 Samsung R&D Institute Poland submission to WAT 2021 Indic Language Multilingual Task ACL 2021 SAT: 2D Semantics Assisted Training for 3D Visual Grounding ICCV 2021 Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets NeurIPS 2021 SEED: Self-supervised Distillation For Visual Representation ICLR 2021 Selective Knowledge Distillation for Neural Machine Translation ACL 2021 Self-Adaptive Distillation for Multilingual Speech Recognition: Leveraging Student Independence Interspeech 2021 Self-boosting for Feature Distillation IJCAI 2021