← Application Areas
Machine Learning › Application Areas ›

Knowledge Distillation

2907 directly classified papers

Papers per year

Papers

When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation ACL 2022
Aligned Weight Regularizers for Pruning Pretrained Neural Networks ACL 2022
FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction NAACL 2022
Multi-stage Progressive Compression of Conformer Transducer for On-device Speech Recognition INTERSPEECH 2022
Learning from Students: Online Contrastive Distillation Network for General Continual Learning IJCAI 2022
Knowledge distillation for In-memory keyword spotting model INTERSPEECH 2022
CN-AutoMIC: Distilling Chinese Commonsense Knowledge from Pretrained Language Models EMNLP 2022
Calibrating Student Models for Emotion-related Tasks EMNLP 2022
Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation EMNLP 2022
Nearest Neighbor Knowledge Distillation for Neural Machine Translation NAACL 2022
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models NAACL 2022
Learning Cross-Lingual IR from an English Retriever NAACL 2022
Keyword Spotting with Synthetic Data using Heterogeneous Knowledge Distillation INTERSPEECH 2022
Hard Gate Knowledge Distillation - Leverage Calibration for Robust and Reliable Language Model EMNLP 2022
Causal Distillation for Language Models NAACL 2022
Leaner and Faster: Two-Stage Model Compression for Lightweight Text-Image Retrieval NAACL 2022
One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation NAACL 2022
Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models NAACL 2022
Muskits: an End-to-end Music Processing Toolkit for Singing Voice Synthesis INTERSPEECH 2022
FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Models INTERSPEECH 2022
Modular and Parameter-Efficient Fine-Tuning for NLP Models EMNLP 2022
BMCook: A Task-agnostic Compression Toolkit for Big Models EMNLP 2022
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation NAACL 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation NAACL 2022
KNOT: Knowledge Distillation Using Optimal Transport for Solving NLP Tasks COLING 2022