conftrace_
← Learning Types
Deep Learning › Learning Types ›

Knowledge Distillation

1,329 papers

Papers per year

1
3
10
12
47
120
155
197
206
255
290
33

Papers

Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation ICML 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time ICML 2022
Self-supervised Models are Good Teaching Assistants for Vision Transformers ICML 2022
MemSR: Training Memory-efficient Lightweight Model for Image Super-Resolution ICML 2022
Distilling Inter-Class Distance for Semantic Segmentation IJCAI 2022
Pseudo-spherical Knowledge Distillation IJCAI 2022
Learning from Students: Online Contrastive Distillation Network for General Continual Learning IJCAI 2022
Plug and Play Knowledge Distillation for kNN-LM with External Logits IJCNLP 2022
TRILLsson: Distilled Universal Paralinguistic Speech Representations INTERSPEECH 2022
Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Models INTERSPEECH 2022
Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation INTERSPEECH 2022
Distilling a Pretrained Language Model to a Multilingual ASR Model INTERSPEECH 2022
Data-augmented cross-lingual synthesis in a teacher-student framework INTERSPEECH 2022
Knowledge Transfer and Distillation from Autoregressive to Non-Autoregessive Speech Recognition INTERSPEECH 2022
Knowledge Distillation For CTC-based Speech Recognition Via Consistent Acoustic Representation Learning INTERSPEECH 2022
Adversarial Knowledge Distillation For Robust Spoken Language Understanding INTERSPEECH 2022
Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification INTERSPEECH 2022
Low-resource Low-footprint Wake-word Detection using Knowledge Distillation INTERSPEECH 2022
Knowledge Distillation via Module Replacing for Automatic Speech Recognition with Recurrent Neural Network Transducer INTERSPEECH 2022
Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation INTERSPEECH 2022
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation NAACL 2022
Adaptable Adapters NAACL 2022
One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation NAACL 2022
Knowledge Inheritance for Pre-trained Language Models NAACL 2022
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models NAACL 2022