Knowledge Distillation

Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation CVPR 2019 Relational Knowledge Distillation CVPR 2019 Scalable Syntax-Aware Language Models Using Knowledge Distillation ACL 2019 Sequence-to-Sequence Learning via Attention Transfer for Incremental Speech Recognition Interspeech 2019 Snapshot Distillation: Teacher-Student Optimization in One Generation CVPR 2019 Structured Knowledge Distillation for Semantic Segmentation CVPR 2019 Structured Pruning of Neural Networks With Budget-Aware Regularization CVPR 2019 Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More CVPR 2019 Towards Architecture-Agnostic Neural Transfer: a Knowledge-Enhanced Approach IJCAI 2019 Training Deep Neural Networks in Generations: A More Tolerant Teacher Educates Better Students AAAI 2019 Unified Semantic Parsing with Weak Supervision ACL 2019 Unifying Heterogeneous Classifiers With Distillation CVPR 2019 Variational Information Distillation for Knowledge Transfer CVPR 2019 When does label smoothing help? NeurIPS 2019 Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention ACL 2019 Zero-shot Knowledge Transfer via Adversarial Belief Matching NeurIPS 2019 Analyzing Knowledge Distillation in Neural Machine Translation EMNLP 2018 Boosting Self-Supervised Learning via Knowledge Transfer CVPR 2018 Born Again Neural Networks ICML 2018 Distilling Knowledge for Search-based Structured Prediction ACL 2018 KDGAN: Knowledge Distillation with Generative Adversarial Networks NeurIPS 2018 Knowledge Distillation by On-the-Fly Native Ensemble NeurIPS 2018 Knowledge Distillation for Sequence Model Interspeech 2018 Large scale distributed neural network training through online distillation ICLR 2018 Marian: Cost-effective High-Quality Neural Machine Translation in C++ ACL 2018