Knowledge Distillation

Learning Multiple Adverse Weather Removal via Two-Stage Knowledge Learning and Multi-Contrastive Regularization: Toward a Unified Model CVPR 2022 Learning Structured Gaussians To Approximate Deep Ensembles CVPR 2022 Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training NAACL 2022 Legal-Tech Open Diaries: Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models EMNLP 2022 Let Images Give You More: Point Cloud Cross-Modal Training for Shape Analysis NeurIPS 2022 Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora ACL 2022 Localization Distillation for Dense Object Detection CVPR 2022 Low Resource Causal Event Detection from Biomedical Literature ACL 2022 Low-resource Low-footprint Wake-word Detection using Knowledge Distillation Interspeech 2022 MemSR: Training Memory-efficient Lightweight Model for Image Super-Resolution ICML 2022 Metadata Shaping: A Simple Approach for Knowledge-Enhanced Language Models ACL 2022 Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts NeurIPS 2022 Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal ACL 2022 Mixture-of-Experts with Expert Choice Routing NeurIPS 2022 Model Distillation for Faithful Explanations of Medical Code Predictions ACL 2022 Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time ICML 2022 Multi-Knowledge Aggregation and Transfer for Semantic Segmentation AAAI 2022 Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model EMNLP 2022 Multilingual Machine Translation with Hyper-Adapters EMNLP 2022 Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations AACL 2022 Multi-Scale Distillation from Multiple Graph Neural Networks AAAI 2022 NaturalInversion: Data-Free Image Synthesis Improving Real-World Consistency AAAI 2022 Nearest Neighbor Knowledge Distillation for Neural Machine Translation NAACL 2022 Norm-based Noisy Corpora Filtering and Refurbishing in Neural Machine Translation EMNLP 2022 Not to Overfit or Underfit the Source Domains? An Empirical Study of Domain Generalization in Question Answering EMNLP 2022