Knowledge Distillation

Self-Decoupling and Ensemble Distillation for Efficient Segmentation AAAI 2023 Self-Distillation into Self-Attention Heads for Improving Transformer-based End-to-End Neural Speaker Diarization Interspeech 2023 Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models ACL 2023 Self-Evolved Dynamic Expansion Model for Task-Free Continual Learning ICCV 2023 Self-Improvement of Non-autoregressive Model via Sequence-Level Distillation EMNLP 2023 Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss CVPR 2023 Semantic Human Parsing via Scalable Semantic Transfer Over Multiple Label Domains CVPR 2023 Semantic Scene Completion With Cleaner Self CVPR 2023 Semantic Segmentation of Degraded Images Using Layer-Wise Feature Adjustor WACV 2023 Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR Interspeech 2023 Semi-attention Partition for Occluded Person Re-identification AAAI 2023 Semi-Supervised Learning via Weight-Aware Distillation under Class Distribution Mismatch ICCV 2023 Sentence Embedder Guided Utterance Encoder (SEGUE) for Spoken Language Understanding Interspeech 2023 Sequence-Level Knowledge Distillation for Class-Incremental End-to-End Spoken Language Understanding Interspeech 2023 Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction ACL 2023 SFT-KD-Recon: Learning a Student-friendly Teacher for Knowledge Distillation in Magnetic Resonance Image Reconstruction MIDL 2023 Should Under-parameterized Student Networks Copy or Average Teacher Weights? NeurIPS 2023 SKDBERT: Compressing BERT via Stochastic Knowledge Distillation AAAI 2023 SKD-NER: Continual Named Entity Recognition via Span-based Knowledge Distillation with Reinforcement Learning EMNLP 2023 SmallCap: Lightweight Image Captioning Prompted With Retrieval Augmentation CVPR 2023 Small Pre-trained Language Models Can be Fine-tuned as Large Models via Over-Parameterization ACL 2023 SMOC-Net: Leveraging Camera Pose for Self-Supervised Monocular Object Pose Estimation CVPR 2023 Soft Target-Enhanced Matching Framework for Deep Entity Matching AAAI 2023 Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints ICLR 2023 Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes ICCV 2023