← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task ACL 2021
G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators NIPS 2021
Few-Shot Class-Incremental Learning via Relation Knowledge Distillation AAAI 2021
Generating Long Financial Report using Conditional Variational Autoencoders with Knowledge Distillation AAAI 2021
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking EMNLP 2021
Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training EMNLP 2021
Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering EMNLP 2021
Improving Tree-Structured Decoder Training for Code Generation via Mutual Learning AAAI 2021
How to Train BERT with an Academic Budget EMNLP 2021
Learning to Augment for Data-scarce Domain BERT Knowledge Distillation AAAI 2021
Data-Free Knowledge Distillation with Soft Targeted Transfer Set Synthesis AAAI 2021
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models ACL 2021
Importance-based Neuron Allocation for Multilingual Neural Machine Translation ACL 2021
SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning NIPS 2021
EvDistill: Asynchronous Events To End-Task Learning via Bidirectional Reconstruction-Guided Cross-Modal Knowledge Distillation CVPR 2021
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation NIPS 2021
NAIST English-to-Japanese Simultaneous Translation System for IWSLT 2021 Simultaneous Text-to-text Task ACL 2021
Learning Distilled Collaboration Graph for Multi-Agent Perception NIPS 2021
An Overview of Uncertainty Calibration for Text Classification and the Role of Distillation ACL 2021
Distilling Relation Embeddings from Pretrained Language Models EMNLP 2021
Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation EMNLP 2021
The USYD-JD Speech Translation System for IWSLT2021 ACL 2021
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning NIPS 2021
MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps NIPS 2021
Robust Knowledge Transfer via Hybrid Forward on the Teacher-Student Model AAAI 2021