Knowledge Distillation

Editing Factual Knowledge in Language Models EMNLP 2021 EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation EMNLP 2021 Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation ICLR 2021 Ego-Exo: Transferring Visual Representations From Third-Person to First-Person Videos CVPR 2021 Enhanced Audio Tagging via Multi- to Single-Modal Teacher-Student Mutual Learning AAAI 2021 Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs EMNLP 2021 EvDistill: Asynchronous Events To End-Task Learning via Bidirectional Reconstruction-Guided Cross-Modal Knowledge Distillation CVPR 2021 Exploring Non-Autoregressive Text Style Transfer EMNLP 2021 Fair Feature Distillation for Visual Recognition CVPR 2021 Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification CVPR 2021 Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing NeurIPS 2021 Finetuning Pretrained Transformers into RNNs EMNLP 2021 Flexible Non-Autoregressive Extractive Summarization with Threshold: How to Extract a Non-Fixed Number of Summary Sentences AAAI 2021 Future-Guided Incremental Transformer for Simultaneous Translation AAAI 2021 GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning EMNLP 2021 General Instance Distillation for Object Detection CVPR 2021 Generating Long Financial Report using Conditional Variational Autoencoders with Knowledge Distillation AAAI 2021 GradTS: A Gradient-Based Automatic Auxiliary Task Selection Method Based on Transformer Networks EMNLP 2021 Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation ACL 2021 Harmonized Dense Knowledge Distillation Training for Multi-Exit Architectures AAAI 2021 Hierarchical Self-supervised Augmented Knowledge Distillation IJCAI 2021 How to Select One Among All ? An Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding EMNLP 2021 HW-TSC’s Participation in the WMT 2021 Efficiency Shared Task EMNLP 2021 Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task ACL 2021 Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation EMNLP 2021