← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Finding trainable sparse networks through Neural Tangent Transfer ICML 2020
Query Distillation: BERT-based Distillation for Ensemble Ranking COLING 2020
T-GD: Transferable GAN-generated Images Detection Framework ICML 2020
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks ICML 2020
Train Big, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers ICML 2020
Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation AACL 2020
Does label smoothing mitigate label noise? ICML 2020
Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks ICML 2020
Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation COLING 2020
Extreme Model Compression for On-device Natural Language Understanding COLING 2020
LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression COLING 2020
A Joint Learning Approach based on Self-Distillation for Keyphrase Extraction from Scientific Documents COLING 2020
Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification Using Pre-trained Language Models COLING 2020
Distill and Replay for Continual Language Learning COLING 2020
Class-incremental Learning via Deep Model Consolidation WACV 2020
A "Network Pruning Network'' Approach to Deep Model Compression WACV 2020
Compressing Pre-trained Language Models by Matrix Decomposition AACL 2020
Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification Using Pre-trained Language Models SEMEVAL 2020
Generalized Bayesian Posterior Expectation Distillation for Deep Neural Networks UAI 2020
KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow MIDL 2020
Cloze Distillation: Improving Neural Language Models with Human Next-Word Prediction CONLL 2020
Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search NIPS 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers NIPS 2020
Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space NIPS 2020
UniTrans : Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data IJCAI 2020