Knowledge Distillation

Towards Example-Based NMT with Multi-Levenshtein Transformers EMNLP 2023 Towards Lossless Head Pruning through Automatic Peer Distillation for Language Models IJCAI 2023 Towards Practical Plug-and-Play Diffusion Models CVPR 2023 Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation ACL 2023 Towards Understanding Ensemble Distillation in Federated Learning ICML 2023 Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning ICLR 2023 TR0N: Translator Networks for 0-Shot Plug-and-Play Conditional Generation ICML 2023 TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models EMNLP 2023 Tuna: Instruction Tuning using Feedback from Large Language Models EMNLP 2023 Type Enhanced BERT for Correcting NER Errors ACL 2023 UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers EMNLP 2023 Understanding Differential Search Index for Text Retrieval ACL 2023 Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits ICML 2023 Understanding the Role of Mixup in Knowledge Distillation: An Empirical Study WACV 2023 UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye View CVPR 2023 UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors ICCV 2023 Unsupervised Adaptation with Quality-Aware Masking to Improve Target-Speaker Voice Activity Detection for Speaker Diarization Interspeech 2023 Visual Prompt Tuning for Generative Transfer Learning CVPR 2023 Weakly Supervised Knowledge Transfer with Probabilistic Logical Reasoning for Object Detection ICLR 2023 What Knowledge Gets Distilled in Knowledge Distillation? NeurIPS 2023 Whole-slide-imaging Cancer Metastases Detection and Localization with Limited Tumorous Data MIDL 2023 X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detection CVPR 2023 You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Models EACL 2023 Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks ACL 2023 Accelerating Certified Robustness Training via Knowledge Transfer NeurIPS 2022