Knowledge Distillation

Boosting Self-Supervised Learning via Knowledge Transfer CVPR 2018 Born Again Neural Networks ICML 2018 ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions NeurIPS 2018 Collaborative Learning for Deep Neural Networks NeurIPS 2018 Compression of End-to-End Models Interspeech 2018 Coupled End-to-End Transfer Learning With Generalized Fisher Information CVPR 2018 Deep Mutual Learning CVPR 2018 Distilling Knowledge for Search-based Structured Prediction ACL 2018 Empirical Analysis of Score Fusion Application to Combined Neural Networks for Open Vocabulary Spoken Term Detection Interspeech 2018 Fast Parametric Learning with Activation Memorization ICML 2018 Feature Representation of Short Utterances Based on Knowledge Distillation for Spoken Language Identification Interspeech 2018 GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking NeurIPS 2018 Improved Training for Online End-to-end Speech Recognition Systems Interspeech 2018 Improving DNNs Trained with Non-Native Transcriptions Using Knowledge Distillation and Target Interpolation Interspeech 2018 Improving Simple Models with Confidence Profiles NeurIPS 2018 KDGAN: Knowledge Distillation with Generative Adversarial Networks NeurIPS 2018 Knowledge Distillation by On-the-Fly Native Ensemble NeurIPS 2018 Knowledge Distillation for Sequence Model Interspeech 2018 Knowledge Transfer with Jacobian Matching ICML 2018 Learning to Specialize with Knowledge Distillation for Visual Question Answering NeurIPS 2018 Learning to Teach ICLR 2018 Lexical and Acoustic Deep Learning Model for Personality Recognition Interspeech 2018 MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models NeurIPS 2018 Model compression via distillation and quantization ICLR 2018 Moonshine: Distilling with Cheap Convolutions NeurIPS 2018