← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

DynaBERT: Dynamic BERT with Adaptive Width and Depth NIPS 2020
DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles NIPS 2020
RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning NIPS 2020
WeChat Neural Machine Translation Systems for WMT20 EMNLP 2020
Fast End-to-end Coreference Resolution for Korean EMNLP 2020
Distilling Multiple Domains for Neural Machine Translation EMNLP 2020
Lifelong Language Knowledge Distillation EMNLP 2020
FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction EMNLP 2020
Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers EMNLP 2020
Improving Autoregressive NMT with Non-Autoregressive Model ACL 2020
Xiaomi’s Submissions for IWSLT 2020 Open Domain Translation Task ACL 2020
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification with DocBERT ACL 2020
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding ACL 2020
The NiuTrans System for WNGT 2020 Efficiency Task ACL 2020
Distilling Neural Networks for Greener and Faster Dependency Parsing ACL 2020
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering ACL 2020
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models ACL 2020
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices ACL 2020
Scalable Attentive Sentence Pair Modeling via Distilled Sentence Embedding AAAI 2020
Exploit and Replace: An Asymmetrical Two-Stream Architecture for Versatile Light Field Saliency Detection AAAI 2020
Ultrafast Video Attention Prediction with Coupled Knowledge Distillation AAAI 2020
Online Knowledge Distillation with Diverse Peers AAAI 2020
Distilling Image Dehazing With Heterogeneous Task Imitation CVPR 2020
Distilled Semantics for Comprehensive Scene Understanding from Videos CVPR 2020
What It Thinks Is Important Is Important: Robustness Transfers Through Input Gradients CVPR 2020