Knowledge Distillation

Look One and More: Distilling Hybrid Order Relational Knowledge for Cross-Resolution Image Recognition AAAI 2020 Maintaining Discrimination and Fairness in Class Incremental Learning CVPR 2020 Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation EMNLP 2020 Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP EMNLP 2020 MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers NeurIPS 2020 Multi-label Contrastive Predictive Coding NeurIPS 2020 Online Knowledge Distillation via Collaborative Learning CVPR 2020 Online Knowledge Distillation with Diverse Peers AAAI 2020 Partial Label Learning with Batch Label Correction AAAI 2020 Plug-in, Trainable Gate for Streamlining Arbitrary Neural Networks AAAI 2020 Regularizing Class-Wise Predictions via Self-Knowledge Distillation CVPR 2020 Response-Anticipated Memory for On-Demand Knowledge Integration in Response Generation ACL 2020 Scalable Attentive Sentence Pair Modeling via Distilled Sentence Embedding AAAI 2020 Search to Distill: Pearls Are Everywhere but Not the Eyes CVPR 2020 Self-Distillation for Improving CTC-Transformer-Based ASR Systems Interspeech 2020 Self-Training With Noisy Student Improves ImageNet Classification CVPR 2020 SimulSpeech: End-to-End Simultaneous Speech to Text Translation ACL 2020 Spatio-Temporal Graph for Video Captioning With Knowledge Distillation CVPR 2020 Speech to Text Adaptation: Towards an Efficient Cross-Modal Distillation Interspeech 2020 Staged Knowledge Distillation for End-to-End Dysarthric Speech Recognition and Speech Attribute Transcription Interspeech 2020 Stochastic Batch Augmentation with An Effective Distilled Dynamic Soft Label Regularizer IJCAI 2020 StrawNet: Self-Training WaveNet for TTS in Low-Data Regimes Interspeech 2020 Structured Multi-Hashing for Model Compression CVPR 2020 Sub-Band Knowledge Distillation Framework for Speech Enhancement Interspeech 2020 Temporally Distributed Networks for Fast Video Semantic Segmentation CVPR 2020