Knowledge Distillation

Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation EMNLP 2023 Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System ACL 2023 Teacher and Student Models of Offensive Language in Social Media ACL 2023 Teacher Guided Training: An Efficient Framework for Knowledge Transfer ICLR 2023 Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers EACL 2023 Teaching Small Language Models to Reason ACL 2023 The student becomes the master: Outperforming GPT3 on Scientific Factual Error Correction EMNLP 2023 Towards a Smaller Student: Capacity Dynamic Distillation for Efficient Image Retrieval CVPR 2023 Towards Higher Pareto Frontier in Multilingual Machine Translation ACL 2023 Towards Practical Plug-and-Play Diffusion Models CVPR 2023 Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation ACL 2023 Type Enhanced BERT for Correcting NER Errors ACL 2023 Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints ACL 2023 What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations EMNLP 2023 AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning EMNLP 2022 Adaptable Adapters NAACL 2022 Adapters for Enhanced Modeling of Multilingual Knowledge and Text EMNLP 2022 Adaptive Label Smoothing with Self-Knowledge in Natural Language Generation EMNLP 2022 ADD: Frequency Attention and Multi-View Based Knowledge Distillation to Detect Low-Quality Compressed Deepfake Images AAAI 2022 Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers AAAI 2022 Adversarial Knowledge Distillation For Robust Spoken Language Understanding Interspeech 2022 Aligned Weight Regularizers for Pruning Pretrained Neural Networks ACL 2022 Anomaly Detection via Reverse Distillation From One-Class Embedding CVPR 2022 An Unsupervised Multiple-Task and Multiple-Teacher Model for Cross-lingual Named Entity Recognition ACL 2022 A Self-improving Skin Lesions Diagnosis Framework Via Pseudo-labeling and Self-distillation ACML 2022