← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Even your Teacher Needs Guidance: Ground-Truth Targets Dampen Regularization Imposed by Self-Distillation NIPS 2021
Does Knowledge Distillation Really Work? NIPS 2021
Slow Learning and Fast Inference: Efficient Graph Similarity Computation via Knowledge Distillation NIPS 2021
Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation CORL 2021
Global explanations with decision rules: a co-learning approach UAI 2021
On the distribution of penultimate activations of classification networks UAI 2021
Topic Modeling for Maternal Health Using Reddit EACL 2021
Finding Spoiler Bias in Tweets by Zero-shot Learning and Knowledge Distilling from Neural Text Simplification EACL 2021
Arabic Compact Language Modelling for Resource Limited Devices EACL 2021
AdapterFusion: Non-Destructive Task Composition for Transfer Learning EACL 2021
Annealing Knowledge Distillation EACL 2021
Extremely Small BERT Models from Mixed-Vocabulary Training EACL 2021
DISK-CSV: Distilling Interpretable Semantic Knowledge with a Class Semantic Vector EACL 2021
Dialect Identification through Adversarial Learning and Knowledge Distillation on Romanian BERT EACL 2021
Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network IJCNLP 2021
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders IJCNLP 2021
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation IJCNLP 2021
Selective Knowledge Distillation for Neural Machine Translation IJCNLP 2021
On Knowledge Distillation for Translating Erroneous Speech Transcriptions IJCNLP 2021
Making Pre-trained Language Models Better Few-shot Learners IJCNLP 2021
Unleash GPT-2 Power for Event Detection IJCNLP 2021
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates IJCNLP 2021
Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation IJCNLP 2021
Claim Matching Beyond English to Scale Global Fact-Checking IJCNLP 2021
The USYD-JD Speech Translation System for IWSLT2021 IJCNLP 2021