Knowledge Distillation

Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay AAAI 2022 Robust Cross-Modal Representation Learning With Progressive Self-Distillation CVPR 2022 Safe Distillation Box AAAI 2022 Samsung R&D Institute Poland Participation in WMT 2022 EMNLP 2022 Scale-Equivalent Distillation for Semi-Supervised Object Detection CVPR 2022 Section-Aware Commonsense Knowledge-Grounded Dialogue Generation with Pre-trained Language Model COLING 2022 Self-Distillation Based on High-level Information Supervision for Compressing End-to-End ASR Model Interspeech 2022 Self-Distillation From the Last Mini-Batch for Consistency Regularization CVPR 2022 Self-Distillation with Meta Learning for Knowledge Graph Completion EMNLP 2022 Self-distribution distillation: efficient uncertainty estimation UAI 2022 Self-Guidance: Improve Deep Neural Network Generalization via Knowledge Distillation WACV 2022 Self-Supervised Knowledge Transfer via Loosely Supervised Auxiliary Tasks WACV 2022 Self-Supervised Models Are Continual Learners CVPR 2022 Self-supervised Models are Good Teaching Assistants for Vision Transformers ICML 2022 Semantic Segmentation of Legal Documents via Rhetorical Roles EMNLP 2022 Semi-Supervised Domain Adaptation via Sample-to-Sample Self-Distillation WACV 2022 SeqPATE: Differentially Private Text Generation via Knowledge Distillation NeurIPS 2022 Shadow Knowledge Distillation: Bridging Offline and Online Knowledge Transfer NeurIPS 2022 SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages EMNLP 2022 SMASH: Improving SMAll Language Models’ Few-SHot Ability with Prompt-Based Distillation EMNLP 2022 Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models NAACL 2022 Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm ACL 2022 Sparse Teachers Can Be Dense with Knowledge EMNLP 2022 Spatial-Channel Token Distillation for Vision MLPs ICML 2022 Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation Interspeech 2022