Knowledge Distillation

The Knowledge Within: Methods for Data-Free Model Compression CVPR 2020 The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding ACL 2020 The NiuTrans System for WNGT 2020 Efficiency Task ACL 2020 The University of Helsinki Submission to the IWSLT2020 Offline SpeechTranslation Task ACL 2020 The Volctrans Machine Translation System for WMT20 EMNLP 2020 TinyBERT: Distilling BERT for Natural Language Understanding EMNLP 2020 TKD: Temporal Knowledge Distillation for Active Perception WACV 2020 Towards Cross-Modality Medical Image Segmentation with Online Mutual Knowledge Distillation AAAI 2020 Towards Inheritable Models for Open-Set Domain Adaptation CVPR 2020 Towards Making the Most of BERT in Neural Machine Translation AAAI 2020 Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation AACL 2020 Towards Oracle Knowledge Distillation with Neural Architecture Search AAAI 2020 Ultrafast Video Attention Prediction with Coupled Knowledge Distillation AAAI 2020 Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification AAAI 2020 Using the Past Knowledge to Improve Sentiment Classification EMNLP 2020 Variational Learning of Bayesian Neural Networks via Bayesian Dark Knowledge IJCAI 2020 Weakly-Supervised Video Re-Localization with Multiscale Attention Model AAAI 2020 What It Thinks Is Important Is Important: Robustness Transfers Through Input Gradients CVPR 2020 Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers EMNLP 2020 XtremeDistil: Multi-stage Distillation for Massive Multilingual Models ACL 2020 A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation ICLR 2019 A Comprehensive Overhaul of Feature Distillation ICCV 2019 Acoustic Scene Classification Using Teacher-Student Learning with Soft-Labels Interspeech 2019 Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teachers IJCAI 2019 Amalgamating Knowledge towards Comprehensive Classification AAAI 2019