Knowledge Distillation

TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing ACL 2020 The Knowledge Within: Methods for Data-Free Model Compression CVPR 2020 The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding ACL 2020 The NiuTrans System for WNGT 2020 Efficiency Task ACL 2020 TinyBERT: Distilling BERT for Natural Language Understanding EMNLP 2020 TKD: Temporal Knowledge Distillation for Active Perception WACV 2020 Towards Cross-Modality Medical Image Segmentation with Online Mutual Knowledge Distillation AAAI 2020 Towards Making the Most of BERT in Neural Machine Translation AAAI 2020 Towards Oracle Knowledge Distillation with Neural Architecture Search AAAI 2020 Ultrafast Video Attention Prediction with Coupled Knowledge Distillation AAAI 2020 Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification AAAI 2020 Variational Learning of Bayesian Neural Networks via Bayesian Dark Knowledge IJCAI 2020 Volctrans Parallel Corpus Filtering System for WMT 2020 EMNLP 2020 WeChat Neural Machine Translation Systems for WMT20 EMNLP 2020 Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers EMNLP 2020 XtremeDistil: Multi-stage Distillation for Massive Multilingual Models ACL 2020 Amalgamating Knowledge towards Comprehensive Classification AAAI 2019 AWSD: Adaptive Weighted Spatiotemporal Distillation for Video Representation ICCV 2019 Baidu Neural Machine Translation Systems for WMT19 ACL 2019 ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech ICLR 2019 Data-Distortion Guided Self-Distillation for Deep Neural Networks AAAI 2019 Distilled Person Re-Identification: Towards a More Scalable System CVPR 2019 Distilling Discrimination and Generalization Knowledge for Event Detection via Delta-Representation Learning ACL 2019 Distilling Object Detectors With Fine-Grained Feature Imitation CVPR 2019 Distilling Policy Distillation AISTATS 2019