← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model NIPS 2017
Distral: Robust multitask reinforcement learning NIPS 2017
Towards Compact and Fast Neural Machine Translation Using a Combined Method EMNLP 2017
Cross-lingual Distillation for Text Classification ACL 2017
Ensembles of Multi-Scale VGG Acoustic Models INTERSPEECH 2017
A Joint Sequential and Relational Model for Frame-Semantic Parsing EMNLP 2017
Sobolev Training for Neural Networks NIPS 2017
Distilling Knowledge from an Ensemble of Models for Punctuation Prediction INTERSPEECH 2017
A Teacher-Student Framework for Zero-Resource Neural Machine Translation ACL 2017
Knowledge Distillation for Bilingual Dictionary Induction EMNLP 2017
Efficient Knowledge Distillation from an Ensemble of Teachers INTERSPEECH 2017
Generalized Distillation Framework for Speaker Normalization INTERSPEECH 2017
Modal Consistency based Pre-Trained Multi-Model Reuse IJCAI 2017
Student-Teacher Training with Diverse Decision Tree Ensembles INTERSPEECH 2017
Adaptively Unified Semi-supervised Learning for Cross-Modal Retrieval IJCAI 2017
How a General-Purpose Commonsense Ontology can Improve Performance of Learning-Based Image Retrieval IJCAI 2017
Transfer Learning and Distillation Techniques to Improve the Acoustic Modeling of Low Resource Languages INTERSPEECH 2017
DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks IJCAI 2016
Network Morphism ICML 2016
Distilling Knowledge from Ensembles of Neural Networks for Speech Recognition INTERSPEECH 2016
Sequence Student-Teacher Training of Deep Neural Networks INTERSPEECH 2016
Model Compression Applied to Small-Footprint Keyword Spotting INTERSPEECH 2016
Dropout distillation ICML 2016
Robust Speech Recognition Using Generalized Distillation Framework INTERSPEECH 2016
Cross Modal Distillation for Supervision Transfer CVPR 2016