Knowledge Distillation

Undistillable: Making A Nasty Teacher That CANNOT teach students ICLR 2021 Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation EMNLP 2021 Universal Trading for Order Execution with Oracle Policy Distillation AAAI 2021 Unpaired Learning for Deep Image Deraining With Rain Direction Regularizer ICCV 2021 Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation WACV 2021 Unsupervised Representation Transfer for Small Networks: I Believe I Can Distill On-the-Fly NeurIPS 2021 Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition ICCV 2021 VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer NeurIPS 2021 Wasserstein Contrastive Representation Distillation CVPR 2021 Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching IJCAI 2021 Weight Distillation: Transferring the Knowledge in Neural Network Parameters ACL 2021 When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks CVPR 2021 Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering EMNLP 2021 YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design AAAI 2021 Zero-Shot Knowledge Distillation from a Decision-Based Black-Box Model ICML 2021 Zoo-Tuning: Adaptive Transfer from A Zoo of Models ICML 2021 A2dele: Adaptive and Attentive Depth Distiller for Efficient RGB-D Salient Object Detection CVPR 2020 Acquiring Knowledge from Pre-Trained Model to Neural Machine Translation AAAI 2020 AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search IJCAI 2020 Adapting Grad-CAM for Embedding Networks WACV 2020 Adversarially Robust Distillation AAAI 2020 Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space NeurIPS 2020 A Joint Learning Approach based on Self-Distillation for Keyphrase Extraction from Scientific Documents COLING 2020 A Knowledge-Driven Approach to Classifying Object and Attribute Coreferences in Opinion Mining EMNLP 2020 Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training EMNLP 2020