Knowledge Distillation

Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice NeurIPS 2020 Robust Federated Learning via Collaborative Machine Teaching AAAI 2020 SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning CoRL 2020 Scalable Attentive Sentence Pair Modeling via Distilled Sentence Embedding AAAI 2020 Search to Distill: Pearls Are Everywhere but Not the Eyes CVPR 2020 Secretary and Online Matching Problems with Machine Learned Advice NeurIPS 2020 Self-Distillation Amplifies Regularization in Hilbert Space NeurIPS 2020 Self-Distillation as Instance-Specific Label Smoothing NeurIPS 2020 Self-Distillation for Improving CTC-Transformer-Based ASR Systems Interspeech 2020 Self-Supervised Generative Adversarial Compression NeurIPS 2020 Self-Training With Noisy Student Improves ImageNet Classification CVPR 2020 SimulSpeech: End-to-End Simultaneous Speech to Text Translation ACL 2020 Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language ACL 2020 SNOW: Subscribing to Knowledge via Channel Pooling for Transfer & Lifelong Learning of Convolutional Neural Networks ICLR 2020 Spatio-Temporal Graph for Video Captioning With Knowledge Distillation CVPR 2020 Speech to Text Adaptation: Towards an Efficient Cross-Modal Distillation Interspeech 2020 Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task EMNLP 2020 Staged Knowledge Distillation for End-to-End Dysarthric Speech Recognition and Speech Attribute Transcription Interspeech 2020 Structure-Level Knowledge Distillation For Multilingual Sequence Labeling ACL 2020 Sub-Band Knowledge Distillation Framework for Speech Enhancement Interspeech 2020 Task-Oriented Feature Distillation NeurIPS 2020 Teacher-Student Networks with Multiple Decoders for Solving Math Word Problem IJCAI 2020 Temporal-Context Enhanced Detection of Heavily Occluded Pedestrians CVPR 2020 TernaryBERT: Distillation-aware Ultra-low Bit BERT EMNLP 2020 TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing ACL 2020