Knowledge Distillation

Scaled Decoupled Distillation CVPR 2024 ScaleKD: Strong Vision Transformers Could Be Excellent Teachers NeurIPS 2024 SDNet: An Extremely Efficient Portrait Matting Model via Self-Distillation WACV 2024 SDPose: Tokenized Pose Estimation via Circulation-Guide Self-Distillation CVPR 2024 SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking CVPR 2024 “Seeing the Big through the Small”: Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations? EMNLP 2024 SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models EMNLP 2024 Segment beyond View: Handling Partially Missing Modality for Audio-Visual Semantic Segmentation AAAI 2024 Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning ACL 2024 SelfCodeAlign: Self-Alignment for Code Generation NeurIPS 2024 Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning ACL 2024 Self-Distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach AAAI 2024 Self-Knowledge Distillation for Knowledge Graph Embedding COLING 2024 Self-Regulated Data-Free Knowledge Amalgamation for Text Classification NAACL 2024 Self-Supervised Learning with Multi-Head Multi-Mode Knowledge Distillation for Speaker Verification Interspeech 2024 Self-Supervised Quantization-Aware Knowledge Distillation AISTATS 2024 Self-Supervised Relation Alignment for Scene Graph Generation WACV 2024 Self-Training Based Few-Shot Node Classification by Knowledge Distillation AAAI 2024 Semi-Supervised Blind Image Quality Assessment through Knowledge Distillation and Incremental Learning AAAI 2024 Semi-Supervised Scene Change Detection by Distillation From Feature-Metric Alignment WACV 2024 Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation IJCAI 2024 Sentence-wise Speech Summarization: Task, Datasets, and End-to-End Modeling with LM Knowledge Distillation Interspeech 2024 ShareBERT: Embeddings Are Capable of Learning Hidden Layers AAAI 2024 Shared-Adapters: A Novel Transformer-based Parameter Efficient Transfer Learning Approach For Children’s Automatic Speech Recognition Interspeech 2024 SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning NeurIPS 2024