Knowledge Distillation

BERT Learns to Teach: Knowledge Distillation with Meta Learning ACL 2022 Better Supervisory Signals by Observing Learning Paths ICLR 2022 Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide Image Classification NeurIPS 2022 Biomedical NER for the Enterprise with Distillated BERN2 and the Kazu Framework EMNLP 2022 Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages EMNLP 2022 BiT: Robustly Binarized Multi-distilled Transformer NeurIPS 2022 BJTU-WeChat’s Systems for the WMT22 Chat Translation Task EMNLP 2022 BMCook: A Task-agnostic Compression Toolkit for Big Models EMNLP 2022 Boosting 3D Object Detection by Simulating Multimodality on Point Clouds CVPR 2022 Boosting Contrastive Learning with Relation Knowledge Distillation AAAI 2022 Boosting Multi-Label Image Classification with Complementary Parallel Self-Distillation IJCAI 2022 Bridging Fairness and Environmental Sustainability in Natural Language Processing EMNLP 2022 Bunched LPCNet2: Efficient Neural Vocoders Covering Devices from Cloud to Edge Interspeech 2022 Calibrating Student Models for Emotion-related Tasks EMNLP 2022 Can Current Explainability Help Provide References in Clinical Notes to Support Humans Annotate Medical Codes? EMNLP 2022 Causal Distillation for Language Models NAACL 2022 CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model EMNLP 2022 Churn Reduction via Distillation ICLR 2022 CILDA: Contrastive Data Augmentation Using Intermediate Layer Knowledge Distillation COLING 2022 Class-Incremental Learning by Knowledge Distillation With Adaptive Feature Consolidation CVPR 2022 Class Incremental Learning for Intent Classification with Limited or No Old Data EMNLP 2022 Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation CVPR 2022 Class-Wise Adaptive Self Distillation for Federated Learning on Non-IID Data (Student Abstract) AAAI 2022 CN-AutoMIC: Distilling Chinese Commonsense Knowledge from Pretrained Language Models EMNLP 2022 Co-Advise: Cross Inductive Bias Distillation CVPR 2022