Knowledge Distillation

NestedNet: Learning Nested Sparse Structures in Deep Neural Networks CVPR 2018 Neural Natural Language Inference Models Enhanced with External Knowledge ACL 2018 Revisiting the Importance of Encoding Logic Rules in Sentiment Classification EMNLP 2018 WSNet: Compact and Efficient Networks Through Weight Sampling ICML 2018 A Gift From Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning CVPR 2017 A Teacher-Student Framework for Zero-Resource Neural Machine Translation ACL 2017 Cross-lingual Distillation for Text Classification ACL 2017 Efficient Knowledge Distillation from an Ensemble of Teachers Interspeech 2017 Knowledge Distillation for Bilingual Dictionary Induction EMNLP 2017 Learning Efficient Object Detection Models with Knowledge Distillation NeurIPS 2017 Mimicking Very Efficient Network for Object Detection CVPR 2017 Sobolev Training for Neural Networks NeurIPS 2017 Student-Teacher Training with Diverse Decision Tree Ensembles Interspeech 2017 Transfer Learning and Distillation Techniques to Improve the Acoustic Modeling of Low Resource Languages Interspeech 2017 Conditional Generative Moment-Matching Networks NeurIPS 2016 Cross Modal Distillation for Supervision Transfer CVPR 2016 Real-Time Action Recognition With Enhanced Motion Vector CNNs CVPR 2016 Robust Speech Recognition Using Generalized Distillation Framework Interspeech 2016 Do Deep Nets Really Need to be Deep? NeurIPS 2014