Knowledge Distillation

Adversarial Self-Supervised Data-Free Distillation for Text Classification EMNLP 2020 Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space NeurIPS 2020 Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training EMNLP 2020 Analyzing Redundancy in Pretrained Transformer Models EMNLP 2020 Audio-Visual Model Distillation Using Acoustic Images WACV 2020 BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover’s Distance EMNLP 2020 BERT-of-Theseus: Compressing BERT by Progressive Module Replacing EMNLP 2020 Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation CVPR 2020 Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation EMNLP 2020 CASIA’s System for IWSLT 2020 Open Domain Translation ACL 2020 Class-incremental Learning via Deep Model Consolidation WACV 2020 Cloze Distillation: Improving Neural Language Models with Human Next-Word Prediction CoNLL 2020 Collaborative Distillation for Ultra-Resolution Universal Style Transfer CVPR 2020 Compact Speaker Embedding: lrx-Vector Interspeech 2020 CompRess: Self-Supervised Learning by Compressing Representations NeurIPS 2020 Conditional Response Augmentation for Dialogue Using Knowledge Distillation Interspeech 2020 Contrastive Distillation on Intermediate Representations for Language Model Compression EMNLP 2020 Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search NeurIPS 2020 Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing CVPR 2020 D3D: Distilled 3D Networks for Video Action Recognition WACV 2020 Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN CVPR 2020 DeGAN: Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier AAAI 2020 DFQF: Data Free Quantization-aware Fine-tuning ACML 2020 Distill, Adapt, Distill: Training Small, In-Domain Models for Neural Machine Translation ACL 2020 Distill BERT to Traditional Models in Chinese Machine Reading Comprehension (Student Abstract) AAAI 2020