Knowledge Distillation

Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks AAAI 2021 Structural Adapters in Pretrained Language Models for AMR-to-Text Generation EMNLP 2021 Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor ACL 2021 Student-Teacher Learning From Clean Inputs to Noisy Inputs CVPR 2021 SuperMix: Supervising the Mixing Data Augmentation CVPR 2021 Teachers Do More Than Teach: Compressing Image-to-Image Models CVPR 2021 The Mininglamp Machine Translation System for WMT21 EMNLP 2021 There Is More Than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking With Sound by Distilling Multimodal Knowledge CVPR 2021 Towards Developing a Multilingual and Code-Mixed Visual Question Answering System by Knowledge Distillation EMNLP 2021 Towards Faster Deep Collaborative Filtering via Hierarchical Decision Networks AAAI 2021 Towards Zero-Shot Knowledge Distillation for Natural Language Processing EMNLP 2021 Transformer-Based ASR Incorporating Time-Reduction Layer and Fine-Tuning with Self-Knowledge Distillation Interspeech 2021 Tree-Like Decision Distillation CVPR 2021 Unbiased Mean Teacher for Cross-Domain Object Detection CVPR 2021 Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation EMNLP 2021 Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation WACV 2021 Verb Knowledge Injection for Multilingual Event Processing ACL 2021 Wasserstein Contrastive Representation Distillation CVPR 2021 When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks CVPR 2021 Winnowing Knowledge for Multi-choice Question Answering EMNLP 2021 Zoo-Tuning: Adaptive Transfer from A Zoo of Models ICML 2021 A2dele: Adaptive and Attentive Depth Distiller for Efficient RGB-D Salient Object Detection CVPR 2020 Acquiring Knowledge from Pre-Trained Model to Neural Machine Translation AAAI 2020 ADINet: Attribute Driven Incremental Network for Retinal Image Classification CVPR 2020 Adversarially Robust Distillation AAAI 2020