Knowledge Distillation

Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching NeurIPS 2022 TRILLsson: Distilled Universal Paralinguistic Speech Representations Interspeech 2022 TrustAL: Trustworthy Active Learning Using Knowledge Distillation AAAI 2022 TSPipe: Learn from Teacher Faster with Pipelines ICML 2022 Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network EMNLP 2022 Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders EMNLP 2022 Unifying the Convergences in Multilingual Neural Machine Translation EMNLP 2022 VIRT: Improving Representation-based Text Matching via Virtual Interaction EMNLP 2022 ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation NeurIPS 2022 Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation CVPR 2022 Weighted Distillation with Unlabeled Examples NeurIPS 2022 Weighted Mutual Learning with Diversity-Driven Model Compression NeurIPS 2022 Weighted self Distillation for Chinese word segmentation ACL 2022 XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding ACL 2022 X-Trans2Cap: Cross-Modal Knowledge Transfer Using Transformer for 3D Dense Captioning CVPR 2022 3D-to-2D Distillation for Indoor Scene Parsing CVPR 2021 Adam Mickiewicz University’s English-Hausa Submissions to the WMT 2021 News Translation Task EMNLP 2021 Amalgamating Knowledge From Heterogeneous Graph Neural Networks CVPR 2021 Anycost GANs for Interactive Image Synthesis and Editing CVPR 2021 A Peek Into the Reasoning of Neural Networks: Interpreting With Structural Visual Concepts CVPR 2021 A Student-Teacher Architecture for Dialog Domain Adaptation Under the Meta-Learning Setting AAAI 2021 Asymmetric Metric Learning for Knowledge Transfer CVPR 2021 Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation Mechanism ACL 2021 AUTOSUMM: Automatic Model Creation for Text Summarization EMNLP 2021 BinaryBERT: Pushing the Limit of BERT Quantization ACL 2021