Knowledge Distillation

Learning multiple visual domains with residual adapters NeurIPS 2017 Leveraging Knowledge Bases in LSTMs for Improving Machine Reading ACL 2017 Mimicking Very Efficient Network for Object Detection CVPR 2017 NITE: A Neural Inductive Teaching Framework for Domain Specific NER EMNLP 2017 Object Detection Meets Knowledge Graphs IJCAI 2017 SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization ICML 2017 Student-Teacher Training with Diverse Decision Tree Ensembles Interspeech 2017 Transfer Learning and Distillation Techniques to Improve the Acoustic Modeling of Low Resource Languages Interspeech 2017 Cross Modal Distillation for Supervision Transfer CVPR 2016 Distilling Knowledge from Ensembles of Neural Networks for Speech Recognition Interspeech 2016 DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks IJCAI 2016 Dropout distillation ICML 2016 Model Compression Applied to Small-Footprint Keyword Spotting Interspeech 2016 Network Morphism ICML 2016 Robust Speech Recognition Using Generalized Distillation Framework Interspeech 2016 Sequence Student-Teacher Training of Deep Neural Networks Interspeech 2016 Bayesian dark knowledge NeurIPS 2015 Efficient algorithms for learning kernels from multiple similarity matrices with general convex loss functions NeurIPS 2010 Distilling dialogues - A method using natural dialogue corpora for dialogue systems development NAACL 2000