Knowledge Distillation

Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching NeurIPS 2022 TRILLsson: Distilled Universal Paralinguistic Speech Representations Interspeech 2022 TrustAL: Trustworthy Active Learning Using Knowledge Distillation AAAI 2022 TSPipe: Learn from Teacher Faster with Pipelines ICML 2022 Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network EMNLP 2022 Uncertainty-Aware Hierarchical Refinement for Incremental Implicitly-Refined Classification NeurIPS 2022 Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders EMNLP 2022 Unified Visual Transformer Compression ICLR 2022 Unifying the Convergences in Multilingual Neural Machine Translation EMNLP 2022 Up to 100x Faster Data-Free Knowledge Distillation AAAI 2022 VIRT: Improving Representation-based Text Matching via Virtual Interaction EMNLP 2022 Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation CVPR 2022 Weighted Distillation with Unlabeled Examples NeurIPS 2022 Weighted Mutual Learning with Diversity-Driven Model Compression NeurIPS 2022 What Makes a "Good" Data Augmentation in Knowledge Distillation - A Statistical Perspective NeurIPS 2022 When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation ACL 2022 When the Student Becomes the Master: Learning Better and Smaller Monolingual Models from mBERT COLING 2022 Why Knowledge Distillation Amplifies Gender Bias and How to Mitigate from the Perspective of DistilBERT NAACL 2022 Wider & Closer: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition EMNLP 2022 XTC: Extreme Compression for Pre-trained Transformers Made Simple and Efficient NeurIPS 2022 X-Trans2Cap: Cross-Modal Knowledge Transfer Using Transformer for 3D Dense Captioning CVPR 2022 ZeroGen: Efficient Zero-shot Learning via Dataset Generation EMNLP 2022 3D-to-2D Distillation for Indoor Scene Parsing CVPR 2021 A Comprehensive Comparison of Word Embeddings in Event & Entity Coreference Resolution. EMNLP 2021 Active Learning for Lane Detection: A Knowledge Distillation Approach ICCV 2021