knowledge distillation

Distilling Knowledge from Ensembles of Neural Networks for Speech Recognition Interspeech 2016 DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks IJCAI 2016 Dropout distillation ICML 2016 Model Compression Applied to Small-Footprint Keyword Spotting Interspeech 2016 Network Morphism ICML 2016 Robust Speech Recognition Using Generalized Distillation Framework Interspeech 2016 Sequence Student-Teacher Training of Deep Neural Networks Interspeech 2016 SoundNet: Learning Sound Representations from Unlabeled Video NeurIPS 2016 Bayesian dark knowledge NeurIPS 2015 Do Deep Nets Really Need to be Deep? NeurIPS 2014 Near-Maximum Entropy Models for Binary Neural Representations of Natural Images NeurIPS 2007 Distilling dialogues - A method using natural dialogue corpora for dialogue systems development NAACL 2000