← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Do Not Forget to Attend to Uncertainty While Mitigating Catastrophic Forgetting WACV 2021
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference EMNLP 2021
Co-Teaching Student-Model through Submission Results of Shared Task EMNLP 2021
Adam Mickiewicz University’s English-Hausa Submissions to the WMT 2021 News Translation Task EMNLP 2021
Training Binary Neural Network without Batch Normalization for Image Super-Resolution AAAI 2021
Grounding Consistency: Distilling Spatial Common Sense for Precise Visual Relationship Detection ICCV 2021
Analyzing the Confidentiality of Undistillable Teachers in Knowledge Distillation NIPS 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout NIPS 2021
Distilling Meta Knowledge on Heterogeneous Graph for Illicit Drug Trafficker Detection on Social Media NIPS 2021
Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme NIPS 2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task EMNLP 2021
Small Model and In-Domain Data Are All You Need EMNLP 2021
Improving Span Representation for Domain-adapted Coreference Resolution EMNLP 2021
Cross-model Back-translated Distillation for Unsupervised Machine Translation ICML 2021
Asynchronous Teacher Guided Bit-wise Hard Mining for Online Hashing AAAI 2021
Channel-Wise Knowledge Distillation for Dense Prediction ICCV 2021
ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques AAAI 2021
RankDistil: Knowledge Distillation for Ranking AISTATS 2021
Isotonic Data Augmentation for Knowledge Distillation IJCAI 2021
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation IJCAI 2021
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor ACL 2021
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation ACL 2021
Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains ACL 2021
Improving Streaming Transformer Based ASR Under a Framework of Self-Supervised Learning INTERSPEECH 2021
Self-Adaptive Distillation for Multilingual Speech Recognition: Leveraging Student Independence INTERSPEECH 2021