Knowledge Distillation

Similarity-Preserving Knowledge Distillation ICCV 2019 Snapshot Distillation: Teacher-Student Optimization in One Generation CVPR 2019 Structured Knowledge Distillation for Semantic Segmentation CVPR 2019 Structured Pruning of Neural Networks With Budget-Aware Regularization CVPR 2019 Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More CVPR 2019 Teacher Guided Architecture Search ICCV 2019 The NiuTrans Machine Translation Systems for WMT19 ACL 2019 The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding Distillation with Ensemble Learning IJCAI 2019 Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion Interspeech 2019 Towards Understanding Knowledge Distillation ICML 2019 Training Deep Neural Networks in Generations: A More Tolerant Teacher Educates Better Students AAAI 2019 Unified Semantic Parsing with Weak Supervision ACL 2019 Unifying Heterogeneous Classifiers With Distillation CVPR 2019 Unsupervised Image Captioning CVPR 2019 Variational Information Distillation for Knowledge Transfer CVPR 2019 Weakly Supervised Cross-lingual Semantic Relation Classification via Knowledge Distillation EMNLP 2019 Zero-Shot Knowledge Distillation in Deep Networks ICML 2019 Zero-shot Knowledge Transfer via Adversarial Belief Matching NeurIPS 2019 A Multi-task Learning Approach for Image Captioning IJCAI 2018 Analyzing Knowledge Distillation in Neural Machine Translation EMNLP 2018 Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy ICLR 2018 A Study of the Importance of External Knowledge in the Named Entity Recognition Task ACL 2018 A Teacher-Student Framework for Maintainable Dialog Manager EMNLP 2018 Attention-Guided Answer Distillation for Machine Reading Comprehension EMNLP 2018 Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification IJCAI 2018